English

Model Inference API Overview

This section mainly introduces the API, data, structures, layout, and alignment rules of model inference. By reading this section, you can use APIs to complete operations such as model loading and releasing, model information obtaining, and model inference on dev boards provided by Horizon Robotics.

Data Structure

Name	Description
HB_DNN_TENSOR_MAX_DIMENSIONS	Maximum dimension of the tensor
hbDNNPackedHandle_t	DNN handle, pointing to the packed multiple models
hbDNNHandle_t	DNN handle, pointing to the single model
hbDNNDataType	Tensor type
hbDNNTensorShape	Tensor shape
hbDNNDescType	Description information type
hbDNNQuantiScale	Quantization/dequantization scale data
hbDNNQuantiType	Fixed-point to floating-point conversion quantization/dequantization type
hbDNNTensorProperties	Tensor information
hbDNNTensor	Tensor

Function Interface

Name	Description
hbDNNGetVersion	Get version info of DNN prediction library
hbDNNInitializeFromFiles	Complete the creation and initialization of `dnnPackedHandle` from the file
hbDNNInitializeFromDDR	Complete the creation and initialization of `dnnPackedHandle` from the memory
hbDNNRelease	Release the model that `dnnPackedHandle` points to
hbDNNGetModelNameList	Get the name list and number of the models that `dnnPackedHandle` points to
hbDNNGetModelHandle	Get the handle of a model from the model list that `dnnPackedHandle` points to
hbDNNGetInputCount	Get the number of the input tensors of the model that `dnnHandle` points to
hbDNNGetInputName	Get the names of the input tensors of the model that `dnnHandle` points to
hbDNNGetInputTensorProperties	Get the property of the specific input tensor of the model that `dnnHandle` points to
hbDNNGetOutputCount	Get the number of the output tensors of the model that `dnnHandle` points to
hbDNNGetOutputName	Get the names of the output tensors of the model that `dnnHandle` points to
hbDNNGetOutputTensorProperties	Get the property of the specific output tensor of the model that `dnnHandle` points to
hbDNNGetInputDesc	Get the description information associated with the specific input of the model that `dnnHandle` points to
hbDNNGetOutputDesc	Get the description information associated with the specific output of the model that `dnnHandle` points to
hbDNNGetModelDesc	Get the description information associated with the model that `dnnHandle` points to
hbDNNGetCompileBpuCoreNum	Get the number of BPU cores at compile time associated with the model that `dnnHandle` points to
hbDNNGetHBMDesc	Get the description information associated with the hbm that `dnnPackedHandle` and `index` points to
hbDNNInferV2	Create synchronous/asynchronous inference tasks based on input parameters
hbDNNGetTaskOutputTensorProperties	Get dynamic output property information of the task that `dnnHandle` points to
hbDNNGetErrorDesc	Translate the error codes into natural language

On This Page

PTQ Conversion Tools

The hb_compile Tool

PTQ Conversion Steps

PTQ Conversion Samples

FAQ and Common Failure Resolutions

Appendix

Tutorial

Advanced Tutorial

API Reference

QAT

qconfig_setter

Model Export

Horizon Operator

FAQ and Common Failure Resolutions

Model Inference Development

Model Inference API Introduction

Data Type and Data Structure

Function Interface

Model Inference Tool Introduction

The hrt_model_exec Tool Introduction

The hbm_infer Tool Introduction

UCP Common API Introduction

Data Structure

Functional Interface

UCP Performance Analysis Tools

FAQ and Error code

Model Deployment Principle and Process

Model Deployment Practice Guidance Examples

HMCT API Reference

Toolchain Operator Support Constraint List

Operator Support List

Operator BPU Constraint List

Community Quality Articles

Model Inference API Overview

Data Structure

Function Interface

The hb_compile Tool

QAT

qconfig_setter

Model Export

Horizon Operator

Model Inference API Introduction

Data Type and Data Structure

Function Interface

Model Inference Tool Introduction

The hrt_model_exec Tool Introduction

The hbm_infer Tool Introduction

Data Structure

Functional Interface

Operator Support List

Operator BPU Constraint List

#Model Inference API Overview

#Data Structure

#Function Interface

Model Inference API Overview

Data Structure

Function Interface