English

Model Accuracy Analysis

There are inevitable accuracy loss with the post-training model quantization that converting the floating-point models into the fixed-point models based on dozens or hundreds of calibration data. But it has been proofed by a large number of production experience that as long as the most optimized parameter combination can be found out, in most cases, Horizon's conversion tools can keep the accuracy loss about 1%.

This section explains how to correctly analyze model accuracy, the basic flow of its operation is shown below. In case the evaluation results fail your expectations, please refer to the Model Accuracy Optimization section and try to optimize the accuracy. If you still can't solve it, please don't hesitate to contact Horizon and seek for technical support.

You are expected to understand how to evaluate model accuracy when reading this section. This section explains how to run the model inference using the outputs of model conversion. As previously described, successful model conversion consists of the following model outputs:

*_original_float_model.onnx
*_optimized_float_model.onnx
*_calibrated_model.onnx
*_ptq_model.onnx
*_quantized_model.bc
*.hbm

Although the final hbm model is the one that will be deployed to the computing platform, in order to facilitate the accuracy evaluation on Ubuntu development machines. We provide *_quantized_model.bc to complete this accuracy evaluation process. The model has already been quantized and has the same accuracy results as the final hbm model. The basic process for loading the model inference model using the Horizon development library is shown below, and this following illustrative code is not only applicable to quantized model, but also applicable to original and optimized onnx models (just replace the model file). You only need to prepare corresponding data in line according to the input types and layouts of the models.

import numpy as np
# Load Horizon dependency library
from horizon_tc_ui.hb_runtime import HBRuntime

# Prepare the input for model running, here the `input.py` is the processed data
data = np.load("input.npy")  
# Load model file
sess = HBRuntime("model.bc")
# Obtain the model input & output node information
input_names = sess.input_names
output_names = sess.output_names

# Model inference, here we assume the model has only one input
# The return value is a list that corresponds in order to the names specified by output_names
output = sess.run(output_names, {input_names[0]: data})
# Model inference, here we assume the model has multi inputs
# The return value is a list that corresponds in order to the names specified by output_names
output = sess.run(output_names, {input_names[0]: data1, input_names[1]: data2})

PTQ Conversion Tools

The hb_compile Tool

PTQ Conversion Steps

PTQ Conversion Samples

FAQ and Common Failure Resolutions

Appendix

Tutorial

Advanced Tutorial

API Reference

QAT

qconfig_setter

Model Export

Horizon Operator

FAQ and Common Failure Resolutions

Model Inference Development

Model Inference API Introduction

Data Type and Data Structure

Function Interface

Model Inference Tool Introduction

The hrt_model_exec Tool Introduction

The hbm_infer Tool Introduction

UCP Common API Introduction

Data Structure

Functional Interface

UCP Performance Analysis Tools

FAQ and Error code

Model Deployment Principle and Process

Model Deployment Practice Guidance Examples

HMCT API Reference

Toolchain Operator Support Constraint List

Operator Support List

Operator BPU Constraint List

Community Quality Articles

Model Accuracy Analysis

The hb_compile Tool

QAT

qconfig_setter

Model Export

Horizon Operator

Model Inference API Introduction

Data Type and Data Structure

Function Interface

Model Inference Tool Introduction

The hrt_model_exec Tool Introduction

The hbm_infer Tool Introduction

Data Structure

Functional Interface

Operator Support List

Operator BPU Constraint List

#Model Accuracy Analysis

Model Accuracy Analysis