The Practical Guide to Deploying the ResNet18 Model with Resizer Input
The overall process of using the PTQ pipeline of the Horizon OpenExplorer toolchain includes multiple phases such as model optimization,
model calibration, model conversion to fixed-point model, model compilation and boarding.
This section takes the Resizer input classification model based on the public version of ResNet18 as a sample(S100 Platform),
and the step-by-step deployment practice is demonstrated in use for your reference.
Prepare the Floating Point Model
To prepare the ResNet18 floating point model, here we use torchvision to export the desired floating point model.
prepare_model.py
import torch
import torchvision
model = torchvision.models.resnet18(pretrained=True)
input_shape = (1, 3, 224, 224)
input_data = torch.randn(input_shape)
output_path = "resnet18.onnx"
torch.onnx.export(model, input_data, output_path,
input_names=["input"], output_names=["output"],
opset_version=10)
Calibration Set Preparation
Information about the public version of the ResNet18 model can be found in The ResNet18 description within the Pytorch documentation,
where it can be seen that the data preprocessing flow for the ResNet18 model is:
- The image short side deflates to 256.
- Resize the image to 224x224 with center crop.
- The data were normalized with mean taking the values [0.485, 0.456, 0.406] and std taking the values [0.229, 0.224, 0.225].
A sample data preprocessing code is shown below:
data_preprocess.py
import os
import cv2
import PIL
import numpy as np
from PIL import Image
ori_dataset_dir = "./calibration_data/imagenet"
calibration_dir = "./calibration_data_rgb"
def resize_transformer(image_data: np.array, short_size: int):
image = Image.fromarray(image_data.astype('uint8'), 'RGB')
# Specify width, height
w, h = image.size
if (w <= h and w == short_size) or (h <= w and h == short_size):
return np.array(image)
# I.e., the width of the image is the short side
if w < h:
resize_size = (short_size, int(short_size * h / w))
# I.e., the height of the image is the short side
else:
resize_size = (int(short_size * w / h), short_size)
# Resize the image
data = np.array(image.resize(resize_size, Image.BILINEAR))
return data
def center_crop_transformer(image_data: np.array, crop_size: int):
image = Image.fromarray(image_data.astype('uint8'), 'RGB')
image_width, image_height = image.size
crop_height, crop_width = (crop_size, crop_size)
crop_top = int(round((image_height - crop_height) / 2.))
crop_left = int(round((image_width - crop_width) / 2.))
image_data = image.crop((crop_left,
crop_top,
crop_left + crop_width,
crop_top + crop_height))
return np.array(image_data).astype(np.float32)
os.mkdir(calibration_dir)
for image_name in os.listdir(ori_dataset_dir):
image_path = os.path.join(ori_dataset_dir, image_name)
# load the image with PIL method
pil_image_data = PIL.Image.open(image_path).convert('RGB')
image_data = np.array(pil_image_data).astype(np.uint8)
# Resize the image
image_data = resize_transformer(image_data, 256)
# Crop the image
image_data = center_crop_transformer(image_data, 224)
# Adjust the data range from [0, 255] to [0, 1]
image_data = image_data * (1 / 255)
# Normalization, (data - mean) / std
mean = [0.485, 0.456, 0.406]
image_data = image_data - mean
std = [0.229, 0.224, 0.225]
image_data = image_data / std
# Convert format from HWC to CHW
image_data = np.transpose(image_data, (2, 0, 1)).astype(np.float32)
# Convert format from CHW to NCHW
image_data = image_data[np.newaxis, :]
# Save the npy file
cali_file_path = os.path.join(calibration_dir, image_name[:-5] + ".npy")
np.save(cali_file_path, image_data)
To support PTQ model calibration, we need to take a small batch dataset from the ImageNet dataset, using the first 100 images as a sample here:
./imagenet
├── ILSVRC2012_val_00000001.JPEG
├── ILSVRC2012_val_00000002.JPEG
├── ILSVRC2012_val_00000003.JPEG
├── ......
├── ILSVRC2012_val_00000099.JPEG
└── ILSVRC2012_val_00000100.JPEG
The catalog structure of the calibration set generated based on the data preprocessing code above is then as follows:
./calibration_data_rgb
├── ILSVRC2012_val_00000001.npy
├── ILSVRC2012_val_00000002.npy
├── ILSVRC2012_val_00000003.npy
├── ......
├── ILSVRC2012_val_00000099.npy
└── ILSVRC2012_val_00000100.npy
Generate Board-side Model
PTQ Conversion Link supports both command line tools and PTQ API for model quantization compilation to generate board-side models, the following is an introduction to the use of the two ways.
Command-line Tool
The command line tool approach only requires you to install horizon_tc_ui(pre-installed in the Docker environment) and create the corresponding yaml file based on the model information configuration,
here we take the yaml file corresponding to the ResNet18 model with resizer input(config.yaml) to show and explain.
config.yaml
model_parameters:
onnx_model: 'resnet18.onnx'
march: "nash-e"
working_dir: 'model_output'
output_model_file_prefix: 'resnet18_224x224_nv12_resizer'
input_parameters:
input_name: ''
input_shape: ''
input_type_rt: 'nv12'
input_type_train: 'rgb'
input_layout_train: 'NCHW'
: 'data_mean_and_scale'
# Formula with [0.485 * 255, 0.456 * 255, 0.406 * 255]
mean_value: "123.675 116.28 103.53"
# Formula with [1 / (0.229*255), 1 / (0.224*255), 1 / (0.225*255)]
scale_value: "0.01712475 0.017507 0.01742919"
calibration_parameters:
cal_data_dir: './calibration_data_rgb'
compiler_parameters:
optimize_level: 'O2'
input_source:
input: resizer
Note
Here, input_name and input_shape are left empty because the tool supports the scenario of single input with no dynamic shape (i.e., the tool internally parses the ONNX model and obtains the name and shape of the input).
When the yaml file configuration is complete, you just need to call The hb_compile Tool to execute the command, the tool executes the command and the key log is as follows:
[horizon@xxx xxx]$ hb_compile -c config.yaml
INFO Start hb_compile...
INFO Start verifying yaml
INFO End verifying yaml
INFO Start to Horizon NN Model Convert.
INFO Start to prepare the onnx model.
INFO End to prepare the onnx model.
INFO Start to optimize the onnx model.
INFO End to optimize the onnx model.
INFO Start to calibrate the model.
INFO End to calibrate the model.
INFO Start to precompile the model.
INFO End to precompile the model.
INFO End to Horizon NN Model Convert.
INFO Successful covert model: /xxx/resnet18_224x224_nv12_resizer_quantized_model.bc
[==================================================]100%
INFO ############# Model input/output info #############
INFO NAME TYPE SHAPE DATA_TYPE
INFO --------- ------ ------------------ ---------
INFO input_y input [1, None, None, 1] UINT8
INFO input_uv input [1, None, None, 2] UINT8
INFO input_roi input [1, 4] INT32
INFO output output [1, 1000] FLOAT32
INFO The hb_compile completes running
After the completion of the command execution, in the yaml file working_dir parameter configuration of the directory (model_output),
will be generated as shown below each stage of the intermediate model, the final model on the board and the model information file,
of which resnet18_224x224_nv12_resizer.hbm that is, the board-side of the inference can be reasoned that the model file:
./model_output
├── resnet18_224x224_nv12_resizer_advice.json
├── resnet18_224x224_nv12_resizer_calibrated_model.onnx
├── resnet18_224x224_nv12_resizer.hbm
├── resnet18_224x224_nv12_resizer.html
├── resnet18_224x224_nv12_resizer.json
├── resnet18_224x224_nv12_resizer_node_info.csv
├── resnet18_224x224_nv12_resizer_optimized_float_model.onnx
├── resnet18_224x224_nv12_resizer_original_float_model.onnx
├── resnet18_224x224_nv12_resizer_ptq_model.onnx
├── resnet18_224x224_nv12_resizer_quant_info.json
└── resnet18_224x224_nv12_resizer_quantized_model.bc
PTQ API
Command line tool provides high ease of use but also bring some flexibility reduction, therefore, when you have the flexibility needs,
you can use the PTQ API way to complete the quantization compilation of the model,
the following for you to introduce the use of the API way to generate the specific process of the board-side of the model.
Attention
Please note that due to the large number of parameters in some interfaces, only the necessary parameters are configured in the sample below to facilitate your overall practice verification,
please refer to HMCT API Refernence and HBDK Tool API Reference for the full parameters of specific interfaces.
Model Optimized Calibration
First, graph optimization and calibration quantization are performed on the floating-point model, a process for which we use the HMCT API, as exemplified below:
calibration.py
import os
import logging
import numpy as np
from hmct.api import build_model
logging.basicConfig(level=logging.INFO)
march = "nash"
onnx_path = "./resnet18.onnx"
cali_data_dir = "./calibration_data_rgb"
model_name = "resnet18_224x224_nv12_resizer"
working_dir = "./model_output/"
cali_data = []
for cali_data_name in os.listdir(cali_data_dir):
data_path = os.path.join(cali_data_dir, cali_data_name)
cali_data.append(np.load(data_path))
ptq_params = {
'cali_dict': {
'calibration_data': {
'input': cali_data
}
},
'debug_methods': [],
'output_nodes': []
}
if not os.path.exists(working_dir):
os.mkdir(working_dir)
build_model(onnx_file=onnx_path,
march=march,
name_prefix=working_dir + model_name,
**ptq_params)
After build_model is executed correctly, the ONNX model for each phase will be generated in the working_dir directory, which has the following directory structure:
./model_output
├── resnet18_224x224_nv12_resizer_calibrated_model.onnx
├── resnet18_224x224_nv12_resizer_optimized_float_model.onnx
├── resnet18_224x224_nv12_resizer_original_float_model.onnx
├── resnet18_224x224_nv12_resizer_ptq_model.onnx
└── resnet18_224x224_nv12_resizer_quant_info.json
The *ptq_model.onnx file here is the ONNX model file after the graph optimization and calibration process.
For a specific description of the ONNX model in the intermediate stages, please refer to the section Post-Training Quantization(PTQ) - PTQ Conversion Steps - Model Quantization and Compilation - Interpret Conversion Output.
Model Turning Fixed Point and Compilation
Next, we need to complete the PTQ model to fixed-point model and model compilation operation, this process we need to complete through the compiler's API, the sample is as follows:
compile.py
import os
import onnx
from hbdk4.compiler.onnx import export
from hbdk4.compiler import convert, compile
march = "nash-e"
working_dir = "./model_output/"
model_name = "resnet18_224x224_nv12_resizer"
ptq_onnx_path = "./model_output/resnet18_224x224_nv12_resizer_ptq_model.onnx"
if not os.path.exists(working_dir):
os.mkdir(working_dir)
# load onnx model
ptq_onnx = onnx.load(ptq_onnx_path)
# Convert onnx model to hbir model
ptq_model = export(proto=ptq_onnx, name=model_name)
func = ptq_model.functions[0]
mean = [0.485, 0.456, 0.406]
std = [0.229, 0.224, 0.225]
# Convert format from NCHW to NHWC
func.inputs[0].insert_transpose([0, 3, 1, 2])
# Insert node for color conversion and normalization
func.inputs[0].insert_image_preprocess(mode="yuvbt601full2rgb",
divisor=255,
mean=mean,
std=std,
is_signed=True)
# Insert node for conversion from nv12 to yuv444
func.inputs[0].insert_roi_resize(mode="nv12")
# Convert type from float to int
quantized_model = convert(m=ptq_model, march=march)
compile(m=quantized_model,
path=working_dir + model_name + ".hbm",
march=march,
progress_bar=True)
After compilation, the working_dir directory will hold the intermediate stage and final model files that can be used on the board, with the following directory structure:
./model_output
├── resnet18_224x224_nv12_resizer_calibrated_model.onnx
├── resnet18_224x224_nv12_resizer_.hbm
├── resnet18_224x224_nv12_resizer_optimized_float_model.onnx
├── resnet18_224x224_nv12_resizer_original_float_model.onnx
├── resnet18_224x224_nv12_resizer_ptq_model.onnx
└── resnet18_224x224_nv12_resizer_quant_info.json
Visualization
After generating the required hbm model, we support you to view it visually with the hb_model_info and hrt_model_exec tools with the following reference commands:
hb_model_info -v resnet18_224x224_nv12_resizer.hbm
hrt_model_exec model_info --model_file resnet18_224x224_nv12_resizer.hbm
Building Board-side Sample
- Dependency libraries for preparing board-side sample.
To build the boardside sample as quickly as possible, we recommend that you use samples/ucp_tutorial/deps_aarch64 directly from the OE package.
Directory as dependent libraries, and the key header files and dynamic libraries that the board-side running sample depends on are listed below:
./deps_aarch64
├── ......
└── ucp
├── include
│ └── hobot
│ ├── dnn
│ │ ├── hb_dnn.h
│ │ ├── hb_dnn_status.h
│ │ └── hb_dnn_v1.h
│ ├── ......
│ ├── hb_sys.h
│ ├── hb_ucp.h
│ ├── hb_ucp_status.h
│ └── hb_ucp_sys.h
└── lib
├── ......
├── libdnn.so
└── libhbucp.so
- Board-side sample development
The following sample shows the process of completing one board-side model inference and obtaining the classification result TOP1 based on the binary file input and the board-side model.
Note
You can refer to the read_image_2_nv12 function in the example code below for how to prepare the input y and uv for the resizer model.
main.cc
#include <fstream>
#include <cstring>
#include <iostream>
#include <map>
#include <vector>
#include "hobot/dnn/hb_dnn.h"
#include "hobot/hb_ucp.h"
#include "hobot/hb_ucp_sys.h"
const char *model_file = "resnet18_224x224_nv12_resizer.hbm";
std::string data_y_path = "ILSVRC2012_val_00000001_y.bin";
std::string data_uv_path = "ILSVRC2012_val_00000001_uv.bin";
typedef struct Roi {
int32_t left;
int32_t top;
int32_t right;
int32_t bottom;
} Roi;
int read_image_2_nv12(std::string &y_path, std::string &uv_path,
std::vector<hbUCPSysMem> &image_mem, int &input_h,
int &input_w);
int prepare_roi_mem(const std::vector<Roi> &rois,
std::vector<hbUCPSysMem> &roi_mem);
int prepare_image_tensor(const std::vector<hbUCPSysMem> &image_mem, int input_h,
int input_w, hbDNNHandle_t dnn_handle,
std::vector<hbDNNTensor> &input_tensor);
int read_binary_file(std::string file_path, char **bin, int *length);
/**
* prepare roi tensor
* @param[in] roi_mem: roi mem info
* @param[in] dnn_handle: dnn handle
* @param[in] roi_tensor_id: tensor id of roi input in model
* @param[out] roi_tensor: roi tensor
*/
int prepare_roi_tensor(const hbUCPSysMem *roi_mem, hbDNNHandle_t dnn_handle,
int32_t roi_tensor_id, hbDNNTensor *roi_tensor);
/**
* prepare out tensor
* @param[in] dnn_handle: dnn handle
* @param[out] output: output tensor
*/
int prepare_output_tensor(hbDNNHandle_t dnn_handle,
std::vector<hbDNNTensor> &output);
int main(int argc, char **argv) {
// load model
hbDNNPackedHandle_t packed_dnn_handle;
hbDNNHandle_t dnn_handle;
const char **model_name_list;
int model_count = 0;
// Step1: get model handle
hbDNNInitializeFromFiles(&packed_dnn_handle, &model_file, 1);
hbDNNGetModelNameList(&model_name_list, &model_count, packed_dnn_handle);
hbDNNGetModelHandle(&dnn_handle, packed_dnn_handle, model_name_list[0]);
// Step2: set input data to nv12
// In the sample, since the input is a same image, can allocate a memory for
// reusing. image_mems is to save image data for y and uv.
std::vector<hbUCPSysMem> image_mems(2);
// image input size
int input_h = 224;
int input_w = 224;
read_image_2_nv12(data_y_path, data_uv_path, image_mems, input_h, input_w);
// Step3: prepare roi mem
/**
* Suppose to infer 2 roi tasks of data, the number of ROIs to be prepared is
* also 2.
*/
// left = 0, top = 0 right = 223, bottom = 223
Roi roi_1 = {0, 0, 223, 223};
// left = 1, top = 1, right = 223, bottom = 223
Roi roi_2 = {1, 1, 223, 223};
std::vector<Roi> rois;
rois.push_back(roi_1);
rois.push_back(roi_2);
int roi_num = 2;
std::vector<hbUCPSysMem> roi_mems(2);
prepare_roi_mem(rois, roi_mems);
// Step4: prepare input and output tensor
std::vector<std::vector<hbDNNTensor>> input_tensors(roi_num);
std::vector<std::vector<hbDNNTensor>> output_tensors(roi_num);
for (int i = 0; i < roi_num; ++i) {
// prepare input tensor
int input_count = 0;
hbDNNGetInputCount(&input_count, dnn_handle);
input_tensors[i].resize(input_count);
// prepare image tensor
/** Tips:
* In the sample, all tasks use the same image, so allocate memory to
* save image. all input tensor can reuse the memory. if your model has
* different input image, please allocate different memory for all inputs.
* */
prepare_image_tensor(image_mems, input_h, input_w,
dnn_handle, input_tensors[i]);
auto roi_tensor_id = 2;
prepare_roi_tensor(&roi_mems[i], dnn_handle, roi_tensor_id,
&input_tensors[i][roi_tensor_id]);
// prepare output tensor
int output_count = 0;
hbDNNGetOutputCount(&output_count, dnn_handle);
output_tensors[i].resize(output_count);
prepare_output_tensor(dnn_handle, output_tensors[i]);
}
// Step5: run inference
hbUCPTaskHandle_t task_handle{nullptr};
/** Tips:
* In the sample, submit multiple tasks at the same time
* when taskHandle is nullptr, here create a new task，and
* when taskHandle is created but not submitted yet, attach new task to the previous which represents multi model task
* */
for (int i = 0; i < roi_num; ++i) {
hbDNNInferV2(&task_handle, output_tensors[i].data(),
input_tensors[i].data(), dnn_handle);
}
// submit multi tasks
hbUCPSchedParam infer_ctrl_param;
HB_UCP_INITIALIZE_SCHED_PARAM(&infer_ctrl_param);
hbUCPSubmitTask(task_handle, &infer_ctrl_param);
// wait task done
hbUCPWaitTaskDone(task_handle, 0);
// Step6: do postprocess with output data for every task
// Find the max score and corresponding label
for (auto roi_idx = 0; roi_idx < roi_num; roi_idx++) {
auto result = reinterpret_cast<float *>(output_tensors[roi_idx][0].sysMem.virAddr);
float max_score = 0.0;
int label = -1;
for (auto i = 0; i < 1000; i++) {
float score = result[i];
if (score > max_score) {
label = i;
max_score = score;
}
}
std::cout << "label: " << label << std::endl;
}
// Step7: release resources
// release task handle
hbUCPReleaseTask(task_handle);
// free input mem
for (auto &mem : image_mems) {
hbUCPFree(&mem);
}
for (auto &mem : roi_mems) {
hbUCPFree(&mem);
}
// free output mem
for (auto &tensors : output_tensors) {
for (auto &tensor : tensors) {
hbUCPFree(&(tensor.sysMem));
}
}
// release model
hbDNNRelease(packed_dnn_handle);
return 0;
}
#define ALIGN(value, alignment) (((value) + ((alignment)-1)) & ~((alignment)-1))
#define ALIGN_32(value) ALIGN(value, 32)
int prepare_image_tensor(const std::vector<hbUCPSysMem> &image_mem, int input_h,
int input_w, hbDNNHandle_t dnn_handle,
std::vector<hbDNNTensor> &input_tensor) {
// y and uv tensor
for (int i = 0; i < 2; i++) {
hbDNNGetInputTensorProperties(&input_tensor[i].properties, dnn_handle, i);
input_tensor[i].sysMem = image_mem[i];
/** Tips:
* roi model should modify input valid shape to input image shape.
* here the struct of y/uv shape is NHWC
* */
input_tensor[i].properties.validShape.dimensionSize[1] = input_h;
input_tensor[i].properties.validShape.dimensionSize[2] = input_w;
if (i == 1) {
// uv input
input_tensor[i].properties.validShape.dimensionSize[1] /= 2;
input_tensor[i].properties.validShape.dimensionSize[2] /= 2;
}
/** Tips:
* For input tensor, stride should be set according to real padding
* of the user's data. And 32 bytes alignment is the requirement of y/uv
* */
input_tensor[i].properties.stride[1] =
ALIGN_32(input_tensor[i].properties.stride[2] *
input_tensor[i].properties.validShape.dimensionSize[2]);
input_tensor[i].properties.stride[0] =
input_tensor[i].properties.stride[1] *
input_tensor[i].properties.validShape.dimensionSize[1];
}
return 0;
}
int prepare_roi_tensor(const hbUCPSysMem *roi_mem, hbDNNHandle_t dnn_handle,
int32_t roi_tensor_id, hbDNNTensor *roi_tensor) {
hbDNNGetInputTensorProperties(&roi_tensor->properties, dnn_handle, roi_tensor_id);
roi_tensor->sysMem = *roi_mem;
return 0;
}
int prepare_output_tensor(hbDNNHandle_t dnn_handle,
std::vector<hbDNNTensor> &output) {
for (size_t i = 0; i < output.size(); i++) {
hbDNNGetOutputTensorProperties(&output[i].properties, dnn_handle, i);
hbUCPMallocCached(&output[i].sysMem, output[i].properties.alignedByteSize, 0);
}
return 0;
}
int read_binary_file(std::string file_path, char **bin, int *length) {
std::ifstream ifs(file_path, std::ios::in | std::ios::binary);
ifs.seekg(0, std::ios::end);
*length = ifs.tellg();
ifs.seekg(0, std::ios::beg);
*bin = new char[sizeof(char) * (*length)];
ifs.read(*bin, *length);
ifs.close();
return 0;
}
/** You can define read_image_2_other_type to prepare your data **/
int read_image_2_nv12(std::string &y_path, std::string &uv_path,
std::vector<hbUCPSysMem> &image_mem, int &input_h,
int &input_w) {
// copy y data
auto w_stride = ALIGN_32(input_w);
int32_t y_mem_size = input_h * w_stride;
hbUCPMallocCached(&image_mem[0], y_mem_size, 0);
uint8_t *y_data_dst = reinterpret_cast<uint8_t *>(image_mem[0].virAddr);
int32_t y_data_length = 0;
char *y_data = nullptr;
read_binary_file(y_path, &y_data, &y_data_length);
memcpy(reinterpret_cast<char *>(image_mem[0].virAddr), y_data, y_mem_size);
// copy uv data
int32_t uv_height = input_h / 2;
int32_t uv_width = input_w / 2;
int32_t uv_mem_size = uv_height * w_stride;
hbUCPMallocCached(&image_mem[1], uv_mem_size, 0);
int32_t uv_data_length = 0;
char *uv_data = nullptr;
read_binary_file(uv_path, &uv_data, &uv_data_length);
memcpy(reinterpret_cast<char *>(image_mem[1].virAddr), uv_data, uv_mem_size);
// make sure cahced mem data is flushed to DDR before inference
hbUCPMemFlush(&image_mem[0], HB_SYS_MEM_CACHE_CLEAN);
hbUCPMemFlush(&image_mem[1], HB_SYS_MEM_CACHE_CLEAN);
free(y_data);
free(uv_data);
return 0;
}
int prepare_roi_mem(const std::vector<Roi> &rois,
std::vector<hbUCPSysMem> &roi_mem) {
auto roi_size = rois.size();
roi_mem.resize(roi_size);
for (auto i = 0; i < roi_size; ++i) {
int32_t mem_size = 4 * sizeof(int32_t);
hbUCPMallocCached(&roi_mem[i], mem_size, 0);
int32_t *roi_data = reinterpret_cast<int32_t *>(roi_mem[i].virAddr);
// The order of filling in the corner points of roi tensor is left, top, right, bottom
roi_data[0] = rois[i].left;
roi_data[1] = rois[i].top;
roi_data[2] = rois[i].right;
roi_data[3] = rois[i].bottom;
// make sure cahced mem data is flushed to DDR before inference
hbUCPMemFlush(&roi_mem[i], HB_SYS_MEM_CACHE_CLEAN);
}
return 0;
}
- Cross-compile to generate board-side executable program
Before cross-compiling, you need to prepare CMakeLists.txt and the sample files.
CMakeLists.txt content is as follows, because the sample does not contain data preprocessing and other operations, so there are fewer dependencies, here is mainly on the compilation parameters of GCC, dependent header files and dynamic library configuration.
Where dnn board-side inference library and hbucp is used to do operations on tensor.
CMakeLists.txt
# CMakeLists.txt
cmake_minimum_required(VERSION 3.0)
project(sample)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11 -Wl,-unresolved-symbols=ignore-in-shared-libs")
message(STATUS "Build type: ${CMAKE_BUILD_TYPE}")
set(CMAKE_CXX_FLAGS_DEBUG "-g -O0")
set(CMAKE_C_FLAGS_DEBUG "-g -O0")
set(CMAKE_CXX_FLAGS_RELEASE " -O3 ")
set(CMAKE_C_FLAGS_RELEASE " -O3 ")
set(CMAKE_BUILD_TYPE ${build_type})
set(DEPS_ROOT ${CMAKE_CURRENT_SOURCE_DIR}/deps_aarch64)
include_directories(${DEPS_ROOT}/ucp/include)
link_directories(${DEPS_ROOT}/ucp/lib)
add_executable(run_sample src/main.cc)
target_link_libraries(run_sample dnn hbucp)
The environment directory structure for compilation is as follows:
.
├── CMakeLists.txt
├── deps_aarch64
│ └── ucp
│ ├── include
│ └── lib
└── src
└── main.cc
When the sample files and CMakeLists.txt are ready, you can compile them. A sample of the compile command is shown below:
Attention
Note that the compilation scripts have to be configured with CC and CXX as the actual paths to cross-compile GCC and G++.
#!/usr/bin/env bash
# Note，please configure according to the actual path
export CC=/arm-gnu-toolchain-12.2.rel1-x86_64-aarch64-none-linux-gnu/bin/aarch64-none-linux-gnu-gcc
export CXX=/arm-gnu-toolchain-12.2.rel1-x86_64-aarch64-none-linux-gnu/bin/aarch64-none-linux-gnu-g++
rm -rf arm_build; mkdir arm_build; cd arm_build
cmake ..; make -j8
cd ..
Once compiled, the board-ready run_sample binary program is generated. At this point, the board-side sample build process is complete.
Preparation for Board-side Operation
When the executable program is compiled, it needs to be prepared for input to the model.
In order to reduce the operation and dependency configuration cost of the practice, here we through the python to do the data processing,
of course, you can also according to the data processing logic in the board side of the sample through the C++ (need to make sure that the data processing logic is the same),
the sample is as follows:
input_data.py
import os
import cv2
import PIL
import numpy as np
from PIL import Image
image_path = "./ILSVRC2012_val_00000001.JPEG"
def resize_transformer(image_data: np.array, short_size: int):
image = Image.fromarray(image_data.astype('uint8'), 'RGB')
# Specify width, height
w, h = image.size
if (w <= h and w == short_size) or (h <= w and h == short_size):
return np.array(image)
# I.e., the width of the image is the short side
if w < h:
resize_size = (short_size, int(short_size * h / w))
# I.e., the height of the image is the short side
else:
resize_size = (int(short_size * w / h), short_size)
# Resize the image
data = np.array(image.resize(resize_size, Image.BILINEAR))
return data
def center_crop_transformer(image_data: np.array, crop_size: int):
image = Image.fromarray(image_data.astype('uint8'), 'RGB')
image_width, image_height = image.size
crop_height, crop_width = (crop_size, crop_size)
crop_top = int(round((image_height - crop_height) / 2.))
crop_left = int(round((image_width - crop_width) / 2.))
image_data = image.crop((crop_left,
crop_top,
crop_left + crop_width,
crop_top + crop_height))
return np.array(image_data).astype(np.float32)
def rgb_to_nv12(image_data: np.array):
r = image_data[:, :, 0]
g = image_data[:, :, 1]
b = image_data[:, :, 2]
y = (0.299 * r + 0.587 * g + 0.114 * b)
u = (-0.169 * r - 0.331 * g + 0.5 * b + 128)[::2, ::2]
v = (0.5 * r - 0.419 * g - 0.081 * b + 128)[::2, ::2]
uv = np.zeros(shape=(u.shape[0], u.shape[1] * 2))
for i in range(0, u.shape[0]):
for j in range(0, u.shape[1]):
uv[i, 2 * j] = u[i, j]
uv[i, 2 * j + 1] = v[i, j]
y = y.astype(np.uint8)
uv = uv.astype(np.uint8)
return y, uv
if __name__ == '__main__':
# load the image with PIL method
pil_image_data = PIL.Image.open(image_path).convert('RGB')
image_data = np.array(pil_image_data).astype(np.uint8)
# Resize the image
image_data = resize_transformer(image_data, 256)
# Crop the image
image_data = center_crop_transformer(image_data, 224)
# Covert format from RGB to nv12
y, uv = rgb_to_nv12(image_data)
y.tofile("ILSVRC2012_val_00000001_y.bin")
uv.tofile("ILSVRC2012_val_00000001_uv.bin")
After completing the preparation of the model input data, which means correctly generating the input file in binary format for the board-side sample inference,
you also need to make sure that you now have the following ready:
-
S100 dev board for the actual execution of board-side program runs.
-
The model(*.hbm) that can be used for board-side inference, the output of Generate Board-side Model.
-
The board-side program(main.cc file and cross-compile to generate board-side executable program), the output of Building Board-side Sample.
-
The board-side program depends on libraries, and in order to reduce deployment costs, you can directly use the contents of the OE package samples/ucp_tutorial/deps_aarch64/ucp/lib folder.
Once ready, we integrate the model file(*.hbm), input data (*.bin files), board-side program and dependent libraries as above, with the following reference directory structure:
horizon
├── ILSVRC2012_val_00000001_y.bin
├── ILSVRC2012_val_00000001_uv.bin
├── lib
├── resnet18_224x224_nv12_resizer.hbm
└── run_sample
Copy this integrated folder as a whole to the board-side environment, refer to the following command:
scp -r horizon/ root@{board_ip}:/map/
Board-side Execution
Finally, you can configure LD_LIBRARY_PATH and run the program as follows:
horizon@hobot:/map/horizon# export LD_LIBRARY_PATH=./lib:$LD_LIBRARY_PATH
horizon@hobot:/map/horizon# ./run_sample
......
label: 65
As you can see, the label: 65 printed in the log is exactly the label corresponding to the ILSVRC2012_val_00000001 image in the ImageNet dataset, i.e., the classification result is correct.
This concludes the full process of practicing the PTQ deployment of the ResNet18 model with Resizer input.
