English

Model Checking

In engineering practice, since not all floating-point models can be converted to quantized models, checking is required before conversion. This checking process will go through the process of a model conversion, the basic flow of its operation is shown below:

However, for those more time-consuming procedures, we simplified the checking process. This command will output the check results and the OP deployment on the device when it finishes the checking of the model.

Usage

Attention

Please use the HBRuntime to inference your floating-point onnx model to make sure the model is legal, and then refer to the method below to use hb_compile to check if the model can be converted to a quantized model.

Usage: hb_compile [OPTIONS]

  A tool that maps floating-point models to quantized models and provides
  some additional validation features

Options:
  -h, --help                      Show this message and exit.
  -m, --model PATH                Model to be compiled or modified
  --proto PATH                    Caffe prototxt file
  --march [nash-e|nash-m|nash-p]  BPU's micro architectures
  -i, --input-shape <TEXT TEXT>...
                                  Specify the model input shape, e.g. --input-
                                  shape input1 1x3x224x224

Parameters Introduction

Parameter Name	Parameter Description
`-h, --help`	Show help information and exit.
`--march`	BPU's micro architectures. for S100 processor, you should specify it to `nash-e`. for S100P processor, you should specify it to `nash-m`.
`-m, --model`	Floating-point model file of Caffe/ONNX. Caffe model: the value is the name of the caffemodel file for the Caffe model. ONNX model: the value is the name of the ONNX model file.
`--proto`	Specify the Prototxt file of the Caffe model.
`-i, --input-shape`	Input node and input shape of the node, optional, where the shape should be separated by`x`. The value is `{input_name} {NxHxWxC/NxCxHxW}`, and the `input_name`is separated from the shape by space. For example, if the model input node name is`data`, and the input shape is`[1,3,224,224]`, the value should be `data 1x3x224x224`. If the shape configured here is not the same as the shape information in the model, the configuration here will take precedence. Note that an `--input-shape`only accepts one name and shape combination, if your model has multiple input nodes, just configure the `--input-shape`parameter multiple times in the command.

Note

If you find that the The converted model node information results during model checking does not match the The converted model node information results during model quantized compilation, it may be because there is actually a default yaml configuration during the model checking process, and if you configured the yaml before the conversion, some parameter differences may cause this if you configure yaml before model quantized compilation, some parameters may cause this situation. The yaml configuration parameters that may cause this to happen include: mean_value , scale_value std_value and quant_config.

Usage Example

Single Model Verification

When checking a single-input model using hb_compile, the reference instruction is as follows:

hb_compile --march ${march} \
           --proto ${caffe_proto} \
           --model ${caffe_model/onnx_model} \
           --input-shape ${input_node_name} ${input_shape}

Multi-Input Model Verification

When checking a multi-input model using hb_compile, the reference instruction is as follows:

hb_compile --march ${march} \
           --proto ${caffe_proto} \
           --model ${caffe_model/onnx_model} \
           --input-shape input.0 1x1x224x224
           --input-shape input.1 1x1x224x224
           --input-shape input.2 1x1x224x224

On This Page

PTQ Conversion Tools

The hb_compile Tool

PTQ Conversion Steps

PTQ Conversion Samples

FAQ and Common Failure Resolutions

Appendix

Tutorial

Advanced Tutorial

API Reference

QAT

qconfig_setter

Model Export

Horizon Operator

FAQ and Common Failure Resolutions

Model Inference Development

Model Inference API Introduction

Data Type and Data Structure

Function Interface

Model Inference Tool Introduction

The hrt_model_exec Tool Introduction

The hbm_infer Tool Introduction

UCP Common API Introduction

Data Structure

Functional Interface

UCP Performance Analysis Tools

FAQ and Error code

Model Deployment Principle and Process

Model Deployment Practice Guidance Examples

HMCT API Reference

Toolchain Operator Support Constraint List

Operator Support List

Operator BPU Constraint List

Community Quality Articles

Model Checking

Usage

Parameters Introduction

Usage Example

Single Model Verification

Multi-Input Model Verification

The hb_compile Tool

QAT

qconfig_setter

Model Export

Horizon Operator

Model Inference API Introduction

Data Type and Data Structure

Function Interface

Model Inference Tool Introduction

The hrt_model_exec Tool Introduction

The hbm_infer Tool Introduction

Data Structure

Functional Interface

Operator Support List

Operator BPU Constraint List

#Model Checking

#Usage

#Parameters Introduction

#Usage Example

#Single Model Verification

#Multi-Input Model Verification

Model Checking

Usage

Parameters Introduction

Usage Example

Single Model Verification

Multi-Input Model Verification