J6P/H ONNX Operator BPU Constraint List
In the table below:
lhs: left-hand side, the left-hand side in the operation.
rhs: right-hand side, the right-hand side in the operation.
Note
- The DeformConv operator itself has certain special characteristics.
We have implemented direct support for
torchvision::deformconvthrough a specific custom operator method. Please contact Horizon for more information. - The GroupNormalization operator, when exported from the Pytorch framework to ONNX, will be automatically split by the framework into multiple suboperators into the original ONNX model, and we support BPU computation for these suboperators, thus implementing support for the GroupNormalization operator.
| ONNX Operator Name | BPU Support Constraints |
|---|---|
| Abs | input: Type: int8, int16, float16 output: Type: int8, int16, float16. Only support (int8->int8), (int16->int8,int16), (float16->float16) |
| Acos | input: Type: int8, int16, float16 output: Type: int8, int16, float16. Only support (int8->int8), (int16->int8,int16), (float16->float16) |
| Acosh | input: Type: int8, int16, float16 output: Type: int8, int16, float16. Only support (int8->int8), (int16->int8,int16), (float16->float16) |
| Add | lhs: Type: int8, int16, int32, float16, float32 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: The input and output types need to be the same. Shape: [*] others: When inputs are of integer or quantized type, if the result overflows, it will saturate to the maximum or minimum value of the result data type. |
| And | lhs: Type: int8, int16, bool8, float16 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: Type: bool8 Shape: [*] |
| ArgMax | input: Type: int8, int16, int32, float16, float32 Shape: [*] Dim: if type is int8, int16 or float16, reduce axis dim size ∈ [1, 32767] Element : if type is int8, int16 or float16, reduce Elements size ∈ [1, 65535] dims: If type is int32, float32, only support one dim. output: Type: int8, int16, int32 Shape: reduce dim will be 1 or fused depend on keepDim |
| ArgMin | input: Type: int8, int16, int32, float16, float32 Shape: [*] Dim: if type is int8, int16 or float16, reduce axis dim size ∈ [1, 32767] Element : if type is int8, int16 or float16, reduce Elements size ∈ [1, 65535] dims: If type is int32, float32, only support one dim. output: Type: int8, int16, int32 Shape: reduce dim will be 1 or fused depend on keepDim |
| Asin | input: Type: int8, int16, float16 output: Type: int8, int16, float16. Only support (int8->int8), (int16->int8,int16), (float16->float16) |
| Asinh | input: Type: int8, int16, float16 output: Type: int8, int16, float16. Only support (int8->int8), (int16->int8,int16), (float16->float16) |
| Atan | input: Type: int8, int16, float16 output: Type: int8, int16, float16. Only support (int8->int8), (int16->int8,int16), (float16->float16) |
| Atanh | input: Type: int8, int16, float16 output: Type: int8, int16, float16. Only support (int8->int8), (int16->int8,int16), (float16->float16) |
| AveragePool | input: Type: int8, int16 Shape: [*,H,W,C] or [*,L,C] output: The input and output types need to be the same. kernel: Shape: [KL] or [KH,KW], only support 1d or 2d now Dim: 1d: KL ∈ [1, 256], KL*bitWidth/8 <= 24576; 2d: KH, KW ∈ [1, 256], KH*KW*bitWidth/8 <= 24576 stride: Shape: [SH,SW] or [SL] Dim: SH, SW, SL ∈ [1, 256] pad: Shape: [PH_BEGIN,PW_BEGIN,PH_END,PW_END] or [PL_BEGIN,PL_END] PH_BEGIN,PW_BEGIN,PL_BEGIN,PH_END,PW_END,PL_END ∈ [-255, 256] dilation: Shape: 1d: [DW]; 2d: [DH, DW] Dim: 1d: DW ∈ {1}; 2d: DH, DW ∈ {1} |
| BatchNormalization | N/A, collapsed in graph optimization phase |
| BitwiseAnd | lhs: Type: int8, int16 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: The input and output types need to be the same. Shape: [*] |
| BitwiseNot | input: Type: int8, int16 Shape: [*] output: The input and output types need to be the same. Shape: [*] |
| BitwiseOr | lhs: Type: int8, int16 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: The input and output types need to be the same. Shape: [*] |
| BitwiseXor | lhs: Type: int8, int16 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: The input and output types need to be the same. Shape: [*] |
| Cast | input: Type: int8, int16, int32, float16, float32, bool8 Shape: [*] output: Type: int8, int16, int32, float16, float32, bool8. only support combinations (int8→int8,int16,int32,float16,float32,bool8), (int16→int8,int16,int32,float16,float32,bool8), (int32→int16,int32,float16,float32), (float16→int8,int16,int32,float16, float32), (float32→int8,int16,int32,float16,float32), (bool8→int8,int16,int32,float16,float32,bool8) Shape: [*] |
| Ceil | input: Type: int8, int16, float16 Shape: [*] output: The input and output types need to be the same. Shape: [*] |
| Celu | input: Type: int8, int16, float16 output: Type: int8, int16, float16. Only support (int8->int8), (int16->int8,int16), (float16->float16) |
| Clip | input: Type: int8, int16, float16 output: Type: int8, int16, float16. Only support (int8->int8), (int16->int8,int16), (float16->float16) |
| Concat | input: Type: No limits Arg Number: input number ∈ [1, 1024] Dim: all dims < 131072 Size: size < 2G output: The input and output types need to be the same. Dim: all dims < 131072 Size: size < 2G |
| Constant | N/A, collapsed in graph optimization phase |
| ConstantOfShape | N/A, collapsed in graph optimization phase |
| Conv | input: --conv 1d-- Type: int8, int16 Dim: * ∈ [1, 4096]; L,C ∈ [1, 65536] --conv 2d-- Type: int8, int16 Shape: [*,H,W,C] Dim: * ∈ [1, 4096]; H,W,C ∈ [1, 65536] --conv 3d-- Type: int8 Shape: [*,D,H,W,C] Dim: * ∈ [1, 128]; D,H,W ∈ [1, 65536]; C ∈ [1, 4096]; weight: --conv 1d-- Type: int8, int16 Shape: [N,KL,C] Dim: C ∈ [1, 8192]; KL ∈ [1, 31]; N ∈ [1, 1048576] if fout is the last layer of conv else [1, 8192] Size: KL × C ∈ [1, 65535] --conv 2d-- Type: int8, int16 Shape: [N,KH,KW,C] Dim: C ∈ [1, 8192]; KH,KW ∈ [1, 31]; N ∈ [1, 1048576] if fout is the last layer of conv else [1, 8192] Size: KH × KW × C ∈ [1, 65535] --conv 3d Type: int8 Shape: [N,KD,KH,KW,C] N ∈ [1, 65535]; KD,KH,KW ∈ [1, 9]; Dim: C ∈ [1, 4096]; Size: KD × KH × KW × C ∈ [1, 131072] bias: Type: float32 output: --conv 1d-- Type: int8, int16, int32, float32, float16 Shape: [*,L,C] Dim: * ∈ [1, 4096]; L,C ∈ [1, 65536] --conv 2d-- Type: int8, int16, int32, float32, float16 If Type is float32, float16, input and weight only support int8 Shape: [*,H,W,C] Dim: * ∈ [1, 4096]; H,W,C ∈ [1, 65536] --conv 3d-- Type: int8, int16, int32, float32, float16 Shape: [*,D,H,W,C] Dim: * ∈ [1, 128]; D,H,W ∈ [1, 65536]; C ∈ [1, 4096]; stride: --conv 1d-- Shape: [SL] Dim: SL ∈ [1, 256]; SL ∈ {1} if dilation > 1 --conv 2d-- Shape: [SH,SW] Dim: SH,SW ∈ [1, 256]; SH,SW ∈ {1} if dilation > 1 --conv 3d-- Shape: [SD,SH,SW] Dim: SD,SH,SW must be 1 or 2 and equal to each other. pad: --conv 1d-- Shape: [P_left,P_right] Dim: P_left,P_right ∈ (-L, 256] --conv 2d-- Shape: [P_top,P_left,P_bottom,P_right] Dim: P_top,P_bottom ∈ (-H, 256], P_left,P_right ∈ (-W, 256] --conv 3d-- Shape: [P_front, P_top, P_left, P_back, P_bottom, P_right] Dim: P_front,P_back ∈ [0, KD/2], P_top,P_bottom ∈ [0, KH/2], P_left,P_right ∈ [0, KW/2] groupNum: Fin.c is divisible by group number, conv 3d only support 1 dilation: --conv 1d-- Shape: [DL] Dim: DL ∈ [1, 18] --conv 2d-- Shape: [DH,DW] Dim: DH,DW ∈ [1, 18] --conv 3d-- Shape: [DD,DH,DW] DD,DH,DW = 1 others: --conv 1d-- Stride only support odd number and 2 when conv is a int16 depthwise conv If groupNum > 1, for each group, fin.c' ∈ [1, 65535], KL × fin.c' ∈ [1, 65535] --conv 2d-- Stride only support odd number and 2 when conv is a int16 depthwise conv If groupNum > 1, for each group, fin.c' ∈ [1, 65535], KH × KW × fin.c' ∈ [1, 65535] Fin.c' = fin.c × min(lcm(fout.c × (lcm(fin.c, 4) / fin.c), 8) / fout.c, groupNum) |
| ConvTranspose | input: --conv 1d/2d-- Type: int8, int16; input and weight cannot both be int16 1d_Shape: [*,W,C] 1d_Dim: * ∈ [1, 128]; W ∈ [1, 65536]; C ∈ [1, 2048] 2d_Shape: [*,H,W,C] 2d_Dim: * ∈ [1, 128]; H,W ∈ [1, 65536]; C ∈ [1, 2048] --conv 3d-- Type: int8 3d_Shape: [*,D,H,W,C] 3d_Dim: * ∈ [1, 128]; D,H,W ∈ [1, 65536]; C ∈ [1, 2048] weight: --conv 1d/2d-- Type: int8, int16; input and weight cannot both be int16 1d_Shape: [N,KW,C] 1d_Dim: N,C ∈ [1, 2048]; KW ∈ [1, 14] 1d_Size: KW × C ∈ [1, 65535] 2d_Shape: [N,KH,KW,C] 2d_Dim: N,C ∈ [1, 2048]; KH,KW ∈ [1, 14]; KH,KW cannot both be 1 2d_Size: KH × KW × C ∈ [1, 65535] --conv 3d-- Type: int8 3d_Shape: [N,KD,KH,KW,C] 3d_Dim: N,C ∈ [1, 2048]; KD,KH,KW ∈ [1, 14]; KD,KH,KW cannot all be 1 3d_Size: KH × KW × C ∈ [1, 65535] bias: Type: float32 output: Type: int8, int16, int32, float16, float32 stride: 1d_Shape: [SW] 1d_Dim: SW ∈ [1, 14]; 2d_Shape: [SH,SW] 2d_Dim: SH,SW ∈ [1, 14]; 3d_Shape: [SD,SH,SW] 3d_Dim: SD,SH,SW ∈ [1, 14]; pad: 1d_Shape: [P_left,P_bottom] 1d_Dim: P_left,P_bottom ∈ [0, 256] 2d_Shape: [P_top,P_left,P_bottom,P_right] 2d_Dim: P_top,P_left,P_bottom,P_right ∈ [0, 256] 3d_Shape: [P_front,P_top,P_left,P_back,P_bottom,P_right] 3d_Dim: P_front,P_top,P_left,P_back,P_bottom,P_right ∈ [0, 256] groupNum: Fin.c is divisible by group number, conv 3d only support 1 dilation: 1d_Shape: [DW] 1d_Dim: DW ∈ {1} 2d_Shape: [DH,DW] 2d_Dim: DH,DW ∈ {1} 3d_Shape: [DD,DH,DW] 3d_Dim: DD,DH,DW ∈ {1} |
| Cos | input: Type: int8, int16, float16 output: Type: int8, int16, float16. Only support (int8->int8), (int16->int8,int16), (float16->float16) |
| Cosh | input: Type: int8, int16, float16 output: Type: int8, int16, float16. Only support (int8->int8), (int16->int8,int16), (float16->float16) |
| CumSum | input: Type: int8, int16, float16, float32 Shape: [*, dim[axis], *] Dim: * ∈ [1, 65536]; dim[axis] ∈ [1, 8192] output: Type: int8, int16, int32, float16, float32. only support combinations (int8→int8,int16,int32), (int16→int16,int32), (float16→float16), (float32→float32) The Shape and Dim is same as input exclusive: If type is float16, float32, only support exclusive == 0, otherwise, exclusive is 0 or 1 reverse: If type is float16, float32, only support reverse == 0, otherwise, exclusive is 0 or 1 |
| DeformConv | input: Type: int8 Shape: [*,H,W,C] Dim: H,W ∈ [1, 1024]; H × W ≤ 720 × 1024; other dims ∈ [1, 65536] offset: Type: int16 Shape: [*,OH,OW,2 × offsetGroupNum × KH × KW] Size: 2 × offsetGroupNum × KH × KW ∈ [2, 256], OH × KH × OW × KW ≤ 720 × 1024 mask: Type: int8 Shape: [*,OH,OW,offsetGroupNum × KH × KW] Size: offsetGroupNum × KH × KW ∈ [1, 128] The value of mask is usually [0, 1] weight: Type: int8 Shape: [N,KH,KW,C] Dim: C ∈ [1, 8192]; KH,KW ∈ [1, 8]; N ∈ [1, 4096] Size: KH × KW × C ∈ [1, 65535] bias: Type: float32 output: Type: int8, int16, int32 Other constraints: Same as fin stride: Shape: [SH,SW] Dim: SH,SW ∈ [1] pad: Shape: [P_top,P_left,P_bottom,P_right] Dim: P_top,P_bottom ∈ [-H/2, 256], P_left,P_right ∈ [-W/2, 256] groupNum: Fin.c is divisible by group number offsetGroupNum: Fin.c is divisible by offset group number Size: offsetGroupNum ∈ [1, 2] dilation: Shape: [DH,DW] Dim: DH,DW ∈ [1] others: For each group, fin.c ∈ [1, 8192], KH × KW × fin.c ∈ [1, 65535], fin.c = C when group = 1 |
| DepthToSpace | input: Type: No limits output: The input and output types need to be the same. |
| Div | input: Type: int8, int16, float16, float32 output: Type: int8, int16, float16, float32. Only support (int8->int8), (int16->int8,int16), (float16->float16), (float32->float32) |
| Dropout | N/A, collapsed in graph optimization phase |
| Elu | input: Type: int8, int16, float16 output: Type: int8, int16, float16. Only support (int8->int8), (int16->int8,int16), (float16->float16) |
| Equal | lhs: Type: int8, int16, int32, float16, float32, bool8 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: Type: bool8 Shape: [*] |
| Erf | input: Type: int8, int16, float16 output: Type: int8, int16, float16. Only support (int8->int8), (int16->int8,int16), (float16->float16) |
| Exp | input: Type: int8, int16, float16 output: Type: int8, int16, float16. Only support (int8->int8), (int16->int8,int16), (float16->float16) |
| Expand | input: Type: No limits output: The input and output types need to be the same. |
| EyeLike | input: Type: int8, int16, int32, float16, float32, bool8 Shape: [*] output: Type: int8, int16, int32, float16, float32, bool8. only support combinations (int8→int8,int16,int32,float16,float32,bool8), (int16→int8,int16,int32,float16,float32,bool8), (int32→int16,int32,float16,float32), (float16→int8,int16,int32,float16, float32), (float32→int8,int16,int32,float16,float32), (bool8→int8,int16,int32,float16,float32,bool8) Shape: [*] |
| Flatten | input: Type: No limits output: The input and output types need to be the same. |
| Floor | input: Type: int8, int16, float16 Shape: [*] output: The input and output types need to be the same. Shape: [*] |
| GRU | input: Type: int8, int16 Dim: all dims < 2097152 size < 2G output: Same as input |
| Gather | input: Type: int8, int16, int32, float16, float32 Shape: [*] Input will transpose to [N, W, C]. W is inputShape[dim], N is the product of inputShape[:dim], C is the product of inputShape[dim+1:]. N, C ∈ [1, 1048576]. When C <= 4, W ∈ [1, 1048576]; when C > 4, W ∈ [1, 131072]. If input type is int32, float32, W ∈ [1, 65536]. index: Type: int8, int16, int32; Unsupported negative indices. Shape: [*] index value should not be larger than 1048576 output: The input and output types need to be the same. |
| GatherElements | input: Type: int8, int16, int32, float16, float32 Shape: [*] Input will transpose to [N, W, C]. W is inputShape[dim], N is the product of inputShape[:dim], C is the product of inputShape[dim+1:]. N, C ∈ [1, 1048576]. N × C should not be larger than 1048576 When C <= 4, W ∈ [1, 1048576]; when C > 4, W ∈ [1, 131072]. If input type is int32, float32, W ∈ [1, 65536]. indices: Type: int8, int16, int32; Unsupported negative indices. Shape: [*] indices value should not be larger than 32767 Indices will transpose to [N, D, C]. D is indicesShape[dim], N is the product of indicesShape[:dim], C is the product of indicesShape[dim+1:]. N, C ∈ [1, 1048576]. IndicesShape[i] <= inputShape[i] for all dimensions i != dim. output: The input and output types need to be the same. |
| GatherND | input: Type: int8, int16, int32, float16, float32 Shape: [*] If gather1d, W = inputShape[batchDim], when C <= 4, W ∈ [1, 1048576]; when C > 4, W ∈ [1, 131072]. If gather2d, H = inputShape[batchDim], W = inputShape[batchDim+1], H ∈ [1, 65536], W ∈ [1, 1048576]. If input type is int32, float32, W ∈ [1, 65536]. B is product of inputShape[0: batchDim], B ∈ [1, 1048576]. C is product of inputShape[batchDim+D:], C ∈ [1, 1048576]. indices: Type: int8, int16, int32; Unsupported negative indices. Shape: [*, D] indices value should not be larger than 1048576. D ∈ [1, 2]. output: Shape: [*] The input and output types need to be the same. batchDim: The number of batch dimensions. The gather of indexing starts from dimension of input[batchDim:] |
| Gelu | input: Type: int8, int16, float16 output: Type: int8, int16, float16. Only support (int8->int8), (int16->int8,int16), (float16->float16) |
| Gemm | input: --conv 1d-- Type: int8, int16 Dim: * ∈ [1, 4096]; L,C ∈ [1, 65536] --conv 2d-- Type: int8, int16 Shape: [*,H,W,C] Dim: * ∈ [1, 4096]; H,W,C ∈ [1, 65536] --conv 3d-- Type: int8 Shape: [*,D,H,W,C] Dim: * ∈ [1, 128]; D,H,W ∈ [1, 65536]; C ∈ [1, 4096]; weight: --conv 1d-- Type: int8, int16 Shape: [N,KL,C] Dim: C ∈ [1, 8192]; KL ∈ [1, 31]; N ∈ [1, 1048576] if fout is the last layer of conv else [1, 8192] Size: KL × C ∈ [1, 65535] --conv 2d-- Type: int8, int16 Shape: [N,KH,KW,C] Dim: C ∈ [1, 8192]; KH,KW ∈ [1, 31]; N ∈ [1, 1048576] if fout is the last layer of conv else [1, 8192] Size: KH × KW × C ∈ [1, 65535] --conv 3d Type: int8 Shape: [N,KD,KH,KW,C] N ∈ [1, 65535]; KD,KH,KW ∈ [1, 9]; Dim: C ∈ [1, 4096]; Size: KD × KH × KW × C ∈ [1, 131072] bias: Type: float32 output: --conv 1d-- Type: int8, int16, int32, float32, float16 Shape: [*,L,C] Dim: * ∈ [1, 4096]; L,C ∈ [1, 65536] --conv 2d-- Type: int8, int16, int32, float32, float16 If Type is float32, float16, input and weight only support int8 Shape: [*,H,W,C] Dim: * ∈ [1, 4096]; H,W,C ∈ [1, 65536] --conv 3d-- Type: int8, int16, int32, float32, float16 Shape: [*,D,H,W,C] Dim: * ∈ [1, 128]; D,H,W ∈ [1, 65536]; C ∈ [1, 4096]; stride: --conv 1d-- Shape: [SL] Dim: SL ∈ [1, 256]; SL ∈ {1} if dilation > 1 --conv 2d-- Shape: [SH,SW] Dim: SH,SW ∈ [1, 256]; SH,SW ∈ {1} if dilation > 1 --conv 3d-- Shape: [SD,SH,SW] Dim: SD,SH,SW must be 1 or 2 and equal to each other. pad: --conv 1d-- Shape: [P_left,P_right] Dim: P_left,P_right ∈ (-L, 256] --conv 2d-- Shape: [P_top,P_left,P_bottom,P_right] Dim: P_top,P_bottom ∈ (-H, 256], P_left,P_right ∈ (-W, 256] --conv 3d-- Shape: [P_front, P_top, P_left, P_back, P_bottom, P_right] Dim: P_front,P_back ∈ [0, KD/2], P_top,P_bottom ∈ [0, KH/2], P_left,P_right ∈ [0, KW/2] groupNum: Fin.c is divisible by group number, conv 3d only support 1 dilation: --conv 1d-- Shape: [DL] Dim: DL ∈ [1, 18] --conv 2d-- Shape: [DH,DW] Dim: DH,DW ∈ [1, 18] --conv 3d-- Shape: [DD,DH,DW] DD,DH,DW = 1 others: --conv 1d-- Stride only support odd number and 2 when conv is a int16 depthwise conv If groupNum > 1, for each group, fin.c' ∈ [1, 65535], KL × fin.c' ∈ [1, 65535] --conv 2d-- Stride only support odd number and 2 when conv is a int16 depthwise conv If groupNum > 1, for each group, fin.c' ∈ [1, 65535], KH × KW × fin.c' ∈ [1, 65535] Fin.c' = fin.c × min(lcm(fout.c × (lcm(fin.c, 4) / fin.c), 8) / fout.c, groupNum) |
| GlobalAveragePool | input: Type: int8, int16, float16, float32 Shape: [*] Dim: reduce axis dim size ∈ [1, 65535] Element : reduce Elements size ∈ [1, 65535] output: Type: int8, int16, float16, float32. only support combinations (int8→int8,int16), (int16→int16), (float16→float16,float32), (float32→float32) Shape: reduce dim will be 1 or fused depend on keepDim dims: If type is float32, only support one dim. |
| GlobalMaxPool | input: Type: int8, int16, int32, float16, float32 Shape: [*] Dim: if type is int8, int16 or float16, reduce axis dim size ∈ [1, 65535] Element : if type is int8, int16 or float16, reduce Elements size ∈ [1, 65535] output: Type: The input and output types need to be the same. Shape: reduce dim will be 1 or fused depend on keepDim |
| Greater | lhs: Type: int8, int16, int32, float16, float32 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: Type: bool8 Shape: [*] |
| GreaterOrEqual | lhs: Type: int8, int16, int32, float16, float32 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: Type: bool8 Shape: [*] |
| GridSample | input: Type: int8, int16, float16 Shape: [*,H,W,C] Dim: If type is int8, H,W ∈ [1, 32768], H and W cannot both be greater than 4096 at the same time, other dims ∈ [1, 65536]. If type is int16 or float16, H,W ∈ [1, 2048], H × W <= 1048576, other dims ∈ [1, 65536]. grid: Type: int16, float32. If gird type is float32, input type should be int16 or float16. Shape: [*,H,W,2] output: The input and output types need to be the same except Dim constraints mode: If input type is int8, support nearest and bilinear If input type is int16 or float16, only support bilinear. padding_mode: If input type is int8, support zeros and border. If input type is int16 or float16, only support zeros. |
| GroupNormalization | input: Type: int8, int16 Shape: [*] Dim: reduce axis dim size ∈ [1, 65535] Element : reduce Elements size ∈ [1, 65535] output: Same as input |
| HardSigmoid | input: Type: int8, int16, float16 output: Type: int8, int16, float16. Only support (int8->int8), (int16->int8,int16), (float16->float16) |
| HardSwish | input: Type: int8, int16, float16 output: Type: int8, int16, float16. Only support (int8->int8), (int16->int8,int16), (float16->float16) |
| Identity | N/A, collapsed in graph optimization phase |
| If | N/A, collapsed in graph optimization phase |
| InstanceNormalization | input: Type: int8, int16 Shape: [*] Dim: reduce axis dim size ∈ [1, 65535] Element : reduce Elements size ∈ [1, 65535] output: Same as input |
| LSTM | input: Type: int8, int16 Dim: all dims < 2097152 size < 2G output: Same as input |
| LayerNormalization | input: Type: int8, int16, float16 Shape: [*] Dim: reduce axis dim size ∈ [1, 65535] Element : reduce Elements size ∈ [1, 65535] output: Same as input |
| LeakyRelu | input: Type: int8, int16, float16 output: Type: int8, int16, float16. Only support (int8->int8), (int16->int8,int16), (float16->float16) |
| Less | lhs: Type: int8, int16, int32, float16, float32 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: Type: bool8 Shape: [*] |
| LessOrEqual | lhs: Type: int8, int16, int32, float16, float32 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: Type: bool8 Shape: [*] |
| Log | input: Type: int8, int16, float16 output: Type: int8, int16, float16. Only support (int8->int8), (int16->int8,int16), (float16->float16) |
| LogSoftmax | input: Type: int8, int16 Shape: [*] Dim: reduce axis dim size ∈ [1, 65535] Element : reduce Elements size ∈ [1, 65535] output: Same as input |
| MatMul | lhs: Type: int8, int16 Shape: [*,M,C] Dim: * ∈ [1, 4096], M,C ∈ [1, 8192] rhl: Type: int8, int16 Shape: [*,C,N] Dim: * ∈ [1, 4096]; C ∈ [1, 8192], N ∈ [1, 1048576] output: Type: int8, int16, int32, float16, float32 Shape: [*,M,N] Other constraints: Same as lhs and rhs |
| Max | lhs: Type: int8, int16, float16, float32 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: The input and output types need to be the same. Shape: [*] |
| MaxPool | input: Type: int8, int16, float16 Shape: [*,H,W,C] output: The input and output types need to be the same. kernel: Shape: [KL] or [KH,KW], only support 1d or 2d now Dim: 1d: KL ∈ [1, 256], KL*bitWidth/8 <= 24576; 2d: KH, KW ∈ [1, 256], KH*KW*bitWidth/8 <= 24576 stride: Shape: [SH,SW] Dim: SH, SW ∈ [1, 256] pad: Shape: [PH_BEGIN,PW_BEGIN,PH_END,PW_END] PH_BEGIN,PW_BEGIN,PH_END,PW_END ∈ [-4, 256] dilation: Shape: 1d: [DW]; 2d: [DH, DW] Dim: 1d: DW ∈ {1}; 2d: DH, DW ∈ {1} |
| Min | lhs: Type: int8, int16, float16, float32 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: The input and output types need to be the same. Shape: [*] |
| Mish | input: Type: int8, int16, float16 output: Type: int8, int16, float16. Only support (int8->int8), (int16->int8,int16), (float16->float16) |
| Mod | lhs: Type: int16, int32. Not support quantized type. Shape: [*] Value range: must be none-negative. rhs: The lhs and rhs types need to be the same. Shape: [*] Value range: must be positive. output: The input and output types need to be the same. Shape: [*] |
| Mul | lhs: Type: int8, int16, int32, float16, float32 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: Type: int8, int16, int32, float16, float32. only support (int8,int16→int8,int16,int32), (int32→int32), (float16→float16), (float32→float32). Shape: [*] others: When inputs are of integer or quantized type, if the result overflows, it will saturate to the maximum or minimum value of the result data type. |
| Neg | input: Type: int8, int16, int32, float16 Shape: [*] output: The input and output types need to be the same. Shape: [*] |
| Not | input: Type: int8, int16, bool8, float16 Shape: [*] output: Type: bool8 Shape: [*] |
| Or | lhs: Type: int8, int16, bool8, float16 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: Type: bool8 Shape: [*] |
| PRelu | lhs: Type: int8, int16 Shape: [*] rhs: The type and shape is same as lhs output: The type and shape is same as lhs |
| Pad | N/A, collapsed in graph optimization phase |
| Pow | If the first input is feature and the second input is initializer with a single element: input: Type: int8, int16, float16 output: Type: int8, int16, float16. Only support (int8->int8), (int16->int8,int16), (float16->float16) Otherwise: input: Type: float16 output: Type: float16 |
| Reciprocal | input: Type: int8, int16, float16 output: Type: int8, int16, float16. Only support (int8->int8), (int16->int8,int16), (float16->float16) |
| ReduceL2 | input: Type: int8, int16 Shape: [*] Dim: reduce axis dim size ∈ [1, 65535] Element : reduce Elements size ∈ [1, 65535] output: Same as input |
| ReduceMax | input: Type: int8, int16, int32, float16, float32 Shape: [*] Dim: if type is int8, int16 or float16, reduce axis dim size ∈ [1, 65535] Element : if type is int8, int16 or float16, reduce Elements size ∈ [1, 65535] output: Type: The input and output types need to be the same. Shape: reduce dim will be 1 or fused depend on keepDim |
| ReduceMean | input: Type: int8, int16, float16, float32 Shape: [*] Dim: reduce axis dim size ∈ [1, 65535] Element : reduce Elements size ∈ [1, 65535] output: Type: int8, int16, float16, float32. only support combinations (int8→int8,int16), (int16→int16), (float16→float16,float32), (float32→float32) Shape: reduce dim will be 1 or fused depend on keepDim dims: If type is float32, only support one dim. |
| ReduceMin | input: Type: int8, int16, int32, float16, float32 Shape: [*] Dim: if type is int8, int16 or float16, reduce axis dim size ∈ [1, 65535] Element : if type is int8, int16 or float16, reduce Elements size ∈ [1, 65535] output: Type: The input and output types need to be the same. Shape: reduce dim will be 1 or fused depend on keepDim |
| ReduceSum | input: Type: int8, int16, float16, float32 Shape: [*] Dim: reduce axis dim size ∈ [1, 65535] Element : reduce Elements size ∈ [1, 65535] output: Type: int8, int16, float16, float32. only support combinations (int8→int8,int16), (int16→int16), (float16→float16,float32), (float32→float32) Shape: reduce dim will be 1 or fused depend on keepDim dims: If type is float32, only support one dim. |
| Relu | N/A, collapsed in graph optimization phase |
| Reshape | input: Type: No limits output: The input and output types need to be the same. |
| Resize | input: Type: int8, float16 output: Type: int8, float16. Only support (int8->int8), (float16->float16) |
| Round | input: Type: int8, int16, float16 output: Type: int8, int16, float16. Only support (int8->int8), (int16->int8,int16), (float16->float16) |
| ScatterElements | input: Type: float16, float32, if type is float32, scatterReduceMode must be add Shape: [*] indices: Type: int16, int32 Shape: data and indices must be the same shape, except for scatterAxis updates: Type: float16 Shape: [*] output: The input and output types need to be the same. scatterReduceMode: Mode ∈ [none, add, max, min] |
| ScatterND | input: Type: float16, float32, if type is float32, scatterReduceMode must be add Shape: [*] indices: Type: int16, int32 Shape: [*,C] Dim: *, C ∈ [1, 2] updates: Type: float16 Shape: [*] output: The input and output types need to be the same. scatterReduceMode: Mode ∈ [none, add, max, min] |
| Selu | input: Type: int8, int16, float16 output: Type: int8, int16, float16. Only support (int8->int8), (int16->int8,int16), (float16->float16) |
| Shape | N/A, collapsed in graph optimization phase |
| Sigmoid | input: Type: int8, int16, float16 output: Type: int8, int16, float16. Only support (int8->int8), (int16->int8,int16), (float16->float16) |
| Sign | input: Type: int8, int16, float16 output: Type: int8, int16, float16. Only support (int8->int8), (int16->int8,int16), (float16->float16) |
| Sin | input: Type: int8, int16, float16 output: Type: int8, int16, float16. Only support (int8->int8), (int16->int8,int16), (float16->float16) |
| Sinh | input: Type: int8, int16, float16 output: Type: int8, int16, float16. Only support (int8->int8), (int16->int8,int16), (float16->float16) |
| Size | N/A, collapsed in graph optimization phase |
| Softmax | input: Type: int8, int16, float16 Shape: [*] Dim: reduce axis dim size ∈ [1, 65535] Element : reduce Elements size ∈ [1, 65535] output: Same as input |
| Softplus | input: Type: int8, int16, float16 output: Type: int8, int16, float16. Only support (int8->int8), (int16->int8,int16), (float16->float16) |
| Softsign | input: Type: int8, int16, float16 output: Type: int8, int16, float16. Only support (int8->int8), (int16->int8,int16), (float16->float16) |
| SpaceToDepth | input: Type: No limits output: The input and output types need to be the same. |
| Split | input: Type: No limits Dim: all dims < 2097152 output: The input and output types need to be the same. The input and output constraints are the same. |
| Sqrt | input: Type: int8, int16, float16 output: Type: int8, int16, float16. Only support (int8->int8), (int16->int8,int16), (float16->float16) |
| Squeeze | input: Type: No limits output: The input and output types need to be the same. |
| Sub | lhs: Type: int8, int16, int32, float16, float32 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: The input and output types need to be the same. Shape: [*] others: When inputs are of integer or quantized type, if the result overflows, it will saturate to the maximum or minimum value of the result data type. |
| Sum | lhs: Type: int8, int16, int32, float16, float32 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: The input and output types need to be the same. Shape: [*] others: When inputs are of integer or quantized type, if the result overflows, it will saturate to the maximum or minimum value of the result data type. |
| Tan | input: Type: int8, int16, float16 output: Type: int8, int16, float16. Only support (int8->int8), (int16->int8,int16), (float16->float16) |
| Tanh | input: Type: int8, int16, float16 output: Type: int8, int16, float16. Only support (int8->int8), (int16->int8,int16), (float16->float16) |
| ThresholdedRelu | input: Type: int8, int16, float16 output: Type: int8, int16, float16. Only support (int8->int8), (int16->int8,int16), (float16->float16) |
| Tile | input: Type: No limits output: The input and output types need to be the same. |
| TopK | input: Type: int8, int16, int32, float16, float32 output: The type of output value is same as input, and indices is integer type. k: K <= 1024 others: The combined size in bytes of the op's operands and outputs must be no more than 4.8MB. |
| Transpose | input: Type: No limits output: The input and output types need to be the same. |
| Trilu | lhs: Type: int8, int16, int32, float16, float32 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: Type: int8, int16, int32, float16, float32. only support (int8,int16→int8,int16,int32), (int32→int32), (float16→float16), (float32→float32). Shape: [*] others: When inputs are of integer or quantized type, if the result overflows, it will saturate to the maximum or minimum value of the result data type. |
| Unsqueeze | input: Type: No limits output: The input and output types need to be the same. |
| Where | condition: Type: bool8 Shape: [*] lhs: Type: int8, int16, int32, float16, float32 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: Type: float16, float32 Shape: [*] input: Type: int8,int16,int32 Shape: [*] |
| Xor | lhs: Type: int8, int16, bool8, float16 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: Type: bool8 Shape: [*] |