S100 ONNX Operator BPU Constraint List
In the table below:
lhs: left-hand side, the left-hand side in the operation.
rhs: right-hand side, the right-hand side in the operation.
Note
- The DeformConv operator itself has certain special characteristics.
We have implemented direct support for
torchvision::deformconvthrough a specific custom operator method. Please contact Horizon for more information. - The GroupNormalization operator, when exported from the Pytorch framework to ONNX, will be automatically split by the framework into multiple suboperators into the original ONNX model, and we support BPU computation for these suboperators, thus implementing support for the GroupNormalization operator.
| ONNX Operator Name | BPU Support Constraints |
|---|---|
| Abs | input: Type: int8, int16 output: Type: int8, int16. Only support (int8->int8), (int16->int8,int16) |
| Acos | input: Type: int8, int16 output: Type: int8, int16. Only support (int8->int8), (int16->int8,int16) |
| Acosh | input: Type: int8, int16 output: Type: int8, int16. Only support (int8->int8), (int16->int8,int16) |
| Add | lhs: Type: int8, int16, int32, float16, float32 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: The input and output types need to be the same. Shape: [*] others: When inputs are of integer or quantized type, if the result overflows, it will saturate to the maximum or minimum value of the result data type. |
| And | lhs: Type: int8, int16, bool8, float16 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: Type: bool8 Shape: [*] |
| ArgMax | input: Type: int8, int16, int32, float16, float32 Shape: [*] Dim: if type is int8, int16 or float16, reduce axis dim size ∈ [1, 32767] Element : if type is int8 or int16, reduce Elements size ∈ [1, 65535] dims: If type is int32, float16, float32, only support one dim. output: Type: int8, int16, int32 Shape: reduce dim will be 1 or fused depend on keepDims |
| ArgMin | input: Type: int8, int16, int32, float16, float32 Shape: [*] Dim: if type is int8, int16 or float16, reduce axis dim size ∈ [1, 32767] Element : if type is int8 or int16, reduce Elements size ∈ [1, 65535] dims: If type is int32, float16, float32, only support one dim. output: Type: int8, int16, int32 Shape: reduce dim will be 1 or fused depend on keepDims |
| Asin | input: Type: int8, int16 output: Type: int8, int16. Only support (int8->int8), (int16->int8,int16) |
| Asinh | input: Type: int8, int16 output: Type: int8, int16. Only support (int8->int8), (int16->int8,int16) |
| Atan | input: Type: int8, int16 output: Type: int8, int16. Only support (int8->int8), (int16->int8,int16) |
| Atanh | input: Type: int8, int16 output: Type: int8, int16. Only support (int8->int8), (int16->int8,int16) |
| AveragePool | input: Type: int8, int16 Shape: [*,H,W,C] or [*,L,C] output: The input and output types need to be the same. kernel: Shape: [KL] or [KH,KW], only support 1d or 2d now Dim: 1d: KL ∈ [1, 256], KL*bitWidth/8 <= 24576; 2d: KH, KW ∈ [1, 256], KH*KW*bitWidth/8 <= 24576 stride: Shape: [SH,SW] or [SL] Dim: SH, SW, SL ∈ [1, 256] pad: Shape: [PH_BEGIN,PW_BEGIN,PH_END,PW_END] or [PL_BEGIN,PL_END] PH_BEGIN,PW_BEGIN,PL_BEGIN,PH_END,PW_END,PL_END ∈ [-255, 256] dilation: Shape: 1d: [DW]; 2d: [DH, DW] Dim: 1d: DW ∈ {1}; 2d: DH, DW ∈ {1} |
| BatchNormalization | N/A, collapsed in graph optimization phase |
| BitwiseAnd | lhs: Type: int8, int16 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: The input and output types need to be the same. Shape: [*] |
| BitwiseNot | input: Type: int8, int16 Shape: [*] output: The input and output types need to be the same. Shape: [*] |
| BitwiseOr | lhs: Type: int8, int16 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: The input and output types need to be the same. Shape: [*] |
| BitwiseXor | lhs: Type: int8, int16 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: The input and output types need to be the same. Shape: [*] |
| Cast | input: Type: int8, int16, int32, float16, float32, bool8 Shape: [*] output: Type: int8, int16, int32, float16, float32, bool8. only support combinations (int8→int8,int16,int32,float16,float32,bool8), (int16→int8,int16,int32,float16,float32,bool8), (int32→int16,int32,float16,float32), (float16→int8,int16,int32,float16, float32), (float32→int8,int16,int32,float16,float32), (bool8→int8,int16,int32,float16,float32,bool8) Shape: [*] |
| Ceil | input: Type: int8, int16, float16 Shape: [*] output: The input and output types need to be the same. Shape: [*] |
| Celu | input: Type: int8, int16 output: Type: int8, int16. Only support (int8->int8), (int16->int8,int16) |
| Clip | input: Type: int8, int16 output: Type: int8, int16. Only support (int8->int8), (int16->int8,int16) |
| Concat | input: Type: No limits Arg Number: input number ∈ [1, 1024] Dim: all dims < 131072 Size: size < 2G output: The input and output types need to be the same. Dim: all dims < 131072 Size: size < 2G |
| Constant | N/A, collapsed in graph optimization phase |
| ConstantOfShape | N/A, collapsed in graph optimization phase |
| Conv | input: --conv 1d-- Type: int8, int16 Shape: [*,L,C] Dim: * ∈ [1, 4096]; L,C ∈ [1, 65536] --conv 2d-- Type: int8, int16 Shape: [*,H,W,C] Dim: * ∈ [1, 4096]; H,W,C ∈ [1, 65536] --conv 3d-- Type: int8 Shape: [*,D,H,W,C] Dim: * ∈ [1, 128]; D,H,W ∈ [1, 65536]; C ∈ [1, 4096]; weight: --conv 1d-- Type: int8, int16 Shape: [N,KL,C] Dim: C ∈ [1, 8192]; KL ∈ [1, 31]; N ∈ [1, 65536] if fout is the last layer of conv else [1, 8192] Size: KL × C ∈ [1, 65535] --conv 2d-- Type: int8, int16 Shape: [N,KH,KW,C] Dim: C ∈ [1, 8192]; KH,KW ∈ [1, 31]; N ∈ [1, 65536] if fout is the last layer of conv else [1, 8192] Size: KH × KW × C ∈ [1, 65535] --conv 3d-- Type: int8 Shape: [N,KD,KH,KW,C] N ∈ [1, 65535]; KD,KH,KW ∈ [1, 9]; Dim: C ∈ [1, 4096]; Size: KD × KH × KW × C ∈ [1, 131072] bias: Type: float32 output: --conv 1d-- Type: int8, int16, int32 Shape: [*,L,C] Dim: * ∈ [1, 4096]; L,C ∈ [1, 65536] --conv 2d-- Type: int8, int16, int32 Shape: [*,H,W,C] Dim: * ∈ [1, 4096]; H,W,C ∈ [1, 65536] --conv 3d-- Type: int8, int16, int32 Shape: [*,D,H,W,C] Dim: * ∈ [1, 128]; D,H,W ∈ [1, 65536]; C ∈ [1, 4096]; stride: --conv 1d-- Shape: [SL] Dim: SL ∈ [1, 256]; SL ∈ {1} if dilation > 1 --conv 2d-- Shape: [SH,SW] Dim: SH,SW ∈ [1, 256]; SH,SW ∈ {1} if dilation > 1 --conv 3d-- Shape: [SD,SH,SW] Dim: SD,SH,SW must be 1 or 2 and equal to each other. pad: --conv 1d-- Shape: [P_left,P_right] Dim: P_left,P_right ∈ (-L, 256] --conv 2d-- Shape: [P_top,P_left,P_bottom,P_right] Dim: P_top,P_bottom ∈ (-H, 256], P_left,P_right ∈ (-W, 256] --conv 3d-- Shape: [P_front, P_top, P_left, P_back, P_bottom, P_right] Dim: P_front,P_back ∈ [0, KD/2], P_top,P_bottom ∈ [0, KH/2], P_left,P_right ∈ [0, KW/2] groupNum: Fin.c is divisible by group number, conv 3d only support 1 dilation: --conv 1d-- Shape: [DL] Dim: DL ∈ [1, 18] --conv 2d-- Shape: [DH,DW] Dim: DH,DW ∈ [1, 18] --conv 3d-- Shape: [DD,DH,DW] DD,DH,DW = 1 others: --conv 1d-- Stride only support odd number and 2 when conv is a int16 depthwise conv If groupNum > 1, for each group, fin.c' ∈ [1, 65535], KL × fin.c' ∈ [1, 65535] --conv 2d-- Stride only support odd number and 2 when conv is a int16 depthwise conv If groupNum > 1, for each group, fin.c' ∈ [1, 65535], KH × KW × fin.c' ∈ [1, 65535] Fin.c' = fin.c × min(lcm(fout.c × (lcm(fin.c, 4) / fin.c), 8) / fout.c, groupNum) |
| ConvTranspose | input: --conv 1d/2d-- Type: int8, int16; input and weight cannot both be int16 1d_Shape: [*,W,C] 1d_Dim: * ∈ [1, 128]; W ∈ [1, 65536]; C ∈ [1, 2048] 2d_Shape: [*,H,W,C] 2d_Dim: * ∈ [1, 128]; H,W ∈ [1, 65536]; C ∈ [1, 2048] --conv 3d-- Type: int8 3d_Shape: [*,D,H,W,C] 3d_Dim: * ∈ [1, 128]; D,H,W ∈ [1, 65536]; C ∈ [1, 2048] weight: --conv 1d/2d-- Type: int8, int16; input and weight cannot both be int16 1d_Shape: [N,KW,C] 1d_Dim: N,C ∈ [1, 2048]; KW ∈ [1, 14] 1d_Size: KW × C ∈ [1, 65535] 2d_Shape: [N,KH,KW,C] 2d_Dim: N,C ∈ [1, 2048]; KH,KW ∈ [1, 14]; KH,KW cannot both be 1 2d_Size: KH × KW × C ∈ [1, 65535] --conv 3d-- Type: int8 3d_Shape: [N,KD,KH,KW,C] 3d_Dim: N,C ∈ [1, 2048]; KD,KH,KW ∈ [1, 14]; KD,KH,KW cannot all be 1 3d_Size: KH × KW × C ∈ [1, 65535] bias: Type: float32 output: Type: int8, int16, int32 stride: 1d_Shape: [SW] 1d_Dim: SW ∈ [1, 14]; 2d_Shape: [SH,SW] 2d_Dim: SH,SW ∈ [1, 14]; 3d_Shape: [SD,SH,SW] 3d_Dim: SD,SH,SW ∈ [1, 14]; pad: 1d_Shape: [P_left,P_bottom] 1d_Dim: P_left,P_bottom ∈ [0, 256] 2d_Shape: [P_top,P_left,P_bottom,P_right] 2d_Dim: P_top,P_left,P_bottom,P_right ∈ [0, 256] 3d_Shape: [P_front,P_top,P_left,P_back,P_bottom,P_right] 3d_Dim: P_front,P_top,P_left,P_back,P_bottom,P_right ∈ [0, 256] groupNum: Fin.c is divisible by group number, conv 3d only support 1 dilation: 1d_Shape: [DW] 1d_Dim: DW ∈ {1} 2d_Shape: [DH,DW] 2d_Dim: DH,DW ∈ {1} 3d_Shape: [DD,DH,DW] 3d_Dim: DD,DH,DW ∈ {1} |
| Cos | input: Type: int8, int16 output: Type: int8, int16. Only support (int8->int8), (int16->int8,int16) |
| Cosh | input: Type: int8, int16 output: Type: int8, int16. Only support (int8->int8), (int16->int8,int16) |
| CumSum | input: Type: int8, int16, float16, float32 Shape: [*, dim[axis], *] Dim: * ∈ [1, 65536]; dim[axis] ∈ [1, 8192] output: Type: int8, int16, int32, float16, float32. only support combinations (int8→int8,int16,int32), (int16→int16,int32), (float16→float16), (float32→float32) The Shape and Dim is same as input exclusive: If type is float16, float32, only support exclusive == 0, otherwise, exclusive is 0 or 1 reverse: If type is float16, float32, only support reverse == 0, otherwise, exclusive is 0 or 1 |
| DeformConv | input: Type: int8 Shape: [*,H,W,C] Dim: H,W ∈ [1, 1024]; H × W ≤ 720 × 1024; other dims ∈ [1, 65536] offset: Type: int16 Shape: [*,OH,OW,2 × offsetGroupNum × KH × KW] Size: 2 × offsetGroupNum × KH × KW ∈ [2, 256], OH × KH × OW × KW ≤ 720 × 1024 mask: Type: int8 Shape: [*,OH,OW,offsetGroupNum × KH × KW] Size: offsetGroupNum × KH × KW ∈ [1, 128] The value of mask is usually [0, 1] weight: Type: int8 Shape: [N,KH,KW,C] Dim: C ∈ [1, 8192]; KH,KW ∈ [1, 8]; N ∈ [1, 4096] Size: KH × KW × C ∈ [1, 65535] bias: Type: float32 output: Type: int8, int16, int32 Other constraints: Same as fin stride: Shape: [SH,SW] Dim: SH,SW ∈ [1] pad: Shape: [P_top,P_left,P_bottom,P_right] Dim: P_top,P_bottom ∈ [-H/2, 256], P_left,P_right ∈ [-W/2, 256] groupNum: Fin.c is divisible by group number offsetGroupNum: Fin.c is divisible by offset group number Size: offsetGroupNum ∈ [1, 2] dilation: Shape: [DH,DW] Dim: DH,DW ∈ [1] others: For each group, fin.c ∈ [1, 8192], KH × KW × fin.c ∈ [1, 65535], fin.c = C when group = 1 |
| DepthToSpace | input: Type: No limits output: The input and output types need to be the same. |
| Div | input: Type: int8, int16, float16, float32 output: Type: int8, int16, float16, float32. Only support (int8->int8), (int16->int8,int16), (float16->float16), (float32->float32) |
| Dropout | N/A, collapsed in graph optimization phase |
| Elu | input: Type: int8, int16 output: Type: int8, int16. Only support (int8->int8), (int16->int8,int16) |
| Equal | lhs: Type: int8, int16, int32, float16, float32, bool8 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: Type: bool8 Shape: [*] |
| Erf | input: Type: int8, int16 output: Type: int8, int16. Only support (int8->int8), (int16->int8,int16) |
| Exp | input: Type: int8, int16 output: Type: int8, int16. Only support (int8->int8), (int16->int8,int16) |
| Expand | input: Type: No limits output: The input and output types need to be the same. |
| EyeLike | input: Type: int8, int16, int32, float16, float32, bool8 Shape: [*] output: Type: int8, int16, int32, float16, float32, bool8. only support combinations (int8→int8,int16,int32,float16,float32,bool8), (int16→int8,int16,int32,float16,float32,bool8), (int32→int16,int32,float16,float32), (float16→int8,int16,int32,float16, float32), (float32→int8,int16,int32,float16,float32), (bool8→int8,int16,int32,float16,float32,bool8) Shape: [*] |
| Flatten | input: Type: No limits output: The input and output types need to be the same. |
| Floor | input: Type: int8, int16, float16 Shape: [*] output: The input and output types need to be the same. Shape: [*] |
| GRU | input: Type: int8, int16 Dim: all dims < 2097152 size < 2G output: Same as input |
| Gather | input: Type: int8, int16, int32, float16, float32 Shape: [*] Input will transpose to [N, W, C]. W is inputShape[dim], N is the product of inputShape[:dim], C is the product of inputShape[dim+1:]. N, C ∈ [1, 1048576], W ∈ [1, 4096]. If input type is int8, int16, W ∈ [1, 32768]. index: Type: int8, int16, int32; Unsupported negative indices. Shape: [*] index value should not be larger than 32768. And the reduce multiple of all index dims of shape should in range [1, 737280(720*1024)], because all dims Will be reduced to W dim of indices and output. If W of fout is larger than 737280, this op will be split too many sub-ops. output: The input and output types need to be the same. |
| GatherElements | input: Type: int8, int16, int32, float16, float32 Shape: [*] Input will transpose to [N, W, C]. W is inputShape[dim], N is the product of inputShape[:dim], C is the product of inputShape[dim+1:]. N, C ∈ [1, 1048576]. N × C should not be larger than 1048576 W ∈ [1, 4096]. If input type is int8, int16, W ∈ [1, 32768]. indices: Type: int8, int16, int32; Unsupported negative indices. Shape: [*] indices value should not be larger than 32768 Indices will transpose to [N, D, C]. D is indicesShape[dim], N is the product of indicesShape[:dim], C is the product of indicesShape[dim+1:]. N, C ∈ [1, 1048576], D ∈ [1, 737280(720*1024)]. IndicesShape[i] <= inputShape[i] for all dimensions i != dim. output: The input and output types need to be the same. |
| GatherND | input: Type: int8, int16, int32, float16, float32 Shape: [*] If gather1d, W = inputShape[batchDim], W ∈ [1, 4096]. If input type is int8, int16 W ∈ [1, 32768] If gather2d, H = inputShape[batchDim], W = inputShape[batchDim+1], H, W ∈ [1, 4096]. If input type is int8, int16, H, W ∈ [1, 32768]. H and W cannot both be greater than 4096 at the same time. B is product of inputShape[0: batchDim], B ∈ [1, 1048576]. C is product of inputShape[batchDim+D:], C ∈ [1, 1048576]. indices: Type: int8, int16, int32; Unsupported negative indices. Shape: [*, D] indices value should not be larger than 32768. D ∈ [1, 2]. Size: I is product of indicesShape[batchDim: batchDim+D], I ∈ [1, 737280]. output: Shape: [*] The input and output types need to be the same. batchDim: The number of batch dimensions. The gather of indexing starts from dimension of input[batchDim:] |
| Gelu | input: Type: int8, int16 output: Type: int8, int16. Only support (int8->int8), (int16->int8,int16) |
| Gemm | input: --conv 1d-- Type: int8, int16 Shape: [*,L,C] Dim: * ∈ [1, 4096]; L,C ∈ [1, 65536] --conv 2d-- Type: int8, int16 Shape: [*,H,W,C] Dim: * ∈ [1, 4096]; H,W,C ∈ [1, 65536] --conv 3d-- Type: int8 Shape: [*,D,H,W,C] Dim: * ∈ [1, 128]; D,H,W ∈ [1, 65536]; C ∈ [1, 4096]; weight: --conv 1d-- Type: int8, int16 Shape: [N,KL,C] Dim: C ∈ [1, 8192]; KL ∈ [1, 31]; N ∈ [1, 65536] if fout is the last layer of conv else [1, 8192] Size: KL × C ∈ [1, 65535] --conv 2d-- Type: int8, int16 Shape: [N,KH,KW,C] Dim: C ∈ [1, 8192]; KH,KW ∈ [1, 31]; N ∈ [1, 65536] if fout is the last layer of conv else [1, 8192] Size: KH × KW × C ∈ [1, 65535] --conv 3d-- Type: int8 Shape: [N,KD,KH,KW,C] N ∈ [1, 65535]; KD,KH,KW ∈ [1, 9]; Dim: C ∈ [1, 4096]; Size: KD × KH × KW × C ∈ [1, 131072] bias: Type: float32 output: --conv 1d-- Type: int8, int16, int32 Shape: [*,L,C] Dim: * ∈ [1, 4096]; L,C ∈ [1, 65536] --conv 2d-- Type: int8, int16, int32 Shape: [*,H,W,C] Dim: * ∈ [1, 4096]; H,W,C ∈ [1, 65536] --conv 3d-- Type: int8, int16, int32 Shape: [*,D,H,W,C] Dim: * ∈ [1, 128]; D,H,W ∈ [1, 65536]; C ∈ [1, 4096]; stride: --conv 1d-- Shape: [SL] Dim: SL ∈ [1, 256]; SL ∈ {1} if dilation > 1 --conv 2d-- Shape: [SH,SW] Dim: SH,SW ∈ [1, 256]; SH,SW ∈ {1} if dilation > 1 --conv 3d-- Shape: [SD,SH,SW] Dim: SD,SH,SW must be 1 or 2 and equal to each other. pad: --conv 1d-- Shape: [P_left,P_right] Dim: P_left,P_right ∈ (-L, 256] --conv 2d-- Shape: [P_top,P_left,P_bottom,P_right] Dim: P_top,P_bottom ∈ (-H, 256], P_left,P_right ∈ (-W, 256] --conv 3d-- Shape: [P_front, P_top, P_left, P_back, P_bottom, P_right] Dim: P_front,P_back ∈ [0, KD/2], P_top,P_bottom ∈ [0, KH/2], P_left,P_right ∈ [0, KW/2] groupNum: Fin.c is divisible by group number, conv 3d only support 1 dilation: --conv 1d-- Shape: [DL] Dim: DL ∈ [1, 18] --conv 2d-- Shape: [DH,DW] Dim: DH,DW ∈ [1, 18] --conv 3d-- Shape: [DD,DH,DW] DD,DH,DW = 1 others: --conv 1d-- Stride only support odd number and 2 when conv is a int16 depthwise conv If groupNum > 1, for each group, fin.c' ∈ [1, 65535], KL × fin.c' ∈ [1, 65535] --conv 2d-- Stride only support odd number and 2 when conv is a int16 depthwise conv If groupNum > 1, for each group, fin.c' ∈ [1, 65535], KH × KW × fin.c' ∈ [1, 65535] Fin.c' = fin.c × min(lcm(fout.c × (lcm(fin.c, 4) / fin.c), 8) / fout.c, groupNum) |
| GlobalAveragePool | input: Type: int8, int16 float16, float32 Shape: [*] Dim: reduce axis dim size ∈ [1, 65535] Element : reduce Elements size ∈ [1, 65535] output: Type: int8, int16, float16, float32. only support combinations (int8→int8,int16), (int16→int16), (float16→float16), (float32→float32) Shape: reduce dim will be 1 or fused depend on keepDim dims: If type is float16 or float32, only support one dim. |
| GlobalMaxPool | input: Type: int8, int16, int32, float16, float32 Shape: [*] Dim: if type is int8 or int16, reduce axis dim size ∈ [1, 65535] Element : if type is int8 or int16, reduce Elements size ∈ [1, 65535] output: Type: The input and output types need to be the same. Shape: reduce dim will be 1 or fused depend on keepDim |
| Greater | lhs: Type: int8, int16, int32, float16, float32 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: Type: bool8 Shape: [*] |
| GreaterOrEqual | lhs: Type: int8, int16, int32, float16, float32 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: Type: bool8 Shape: [*] |
| GridSample | input: Type: int8 Shape: [*,H,W,C] Dim: H ∈ [1, 32768], W ∈ [1, 32768], other dims ∈ [1, 65536]. NOTE: H and W cannot both be greater than 4096 at the same time. grid: Type: int16 Shape: [*,H,W,2] output: Same as input except Dim constraints mode: Only support bilinear and nearest padding_mode: Only support zeros and border |
| GroupNormalization | input: Type: int8, int16 Shape: [*] Dim: reduce axis dim size ∈ [1, 65535] Element : reduce Elements size ∈ [1, 65535] output: Same as input |
| HardSigmoid | input: Type: int8, int16 output: Type: int8, int16. Only support (int8->int8), (int16->int8,int16) |
| HardSwish | input: Type: int8, int16 output: Type: int8, int16. Only support (int8->int8), (int16->int8,int16) |
| Identity | N/A, collapsed in graph optimization phase |
| If | N/A, collapsed in graph optimization phase |
| InstanceNormalization | input: Type: int8, int16 Shape: [*] Dim: reduce axis dim size ∈ [1, 65535] Element : reduce Elements size ∈ [1, 65535] output: Same as input |
| LSTM | input: Type: int8, int16 Dim: all dims < 2097152 size < 2G output: Same as input |
| LayerNormalization | input: Type: int8, int16, float16 Shape: [*] Dim: reduce axis dim size ∈ [1, 65535] Element : reduce Elements size ∈ [1, 65535] output: Same as input |
| LeakyRelu | input: Type: int8, int16 output: Type: int8, int16. Only support (int8->int8), (int16->int8,int16) |
| Less | lhs: Type: int8, int16, int32, float16, float32 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: Type: bool8 Shape: [*] |
| LessOrEqual | lhs: Type: int8, int16, int32, float16, float32 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: Type: bool8 Shape: [*] |
| Log | input: Type: int8, int16 output: Type: int8, int16. Only support (int8->int8), (int16->int8,int16) |
| LogSoftmax | input: Type: int8, int16 Shape: [*] Dim: reduce axis dim size ∈ [1, 65535] Element : reduce Elements size ∈ [1, 65535] output: Same as input |
| MatMul | lhs: Type: int8, int16 Shape: [*,M,C] Dim: * ∈ [1, 4096], M,C ∈ [1, 8192] rhl: Type: int8, int16 Shape: [*,C,N] Dim: * ∈ [1, 4096]; C ∈ [1, 8192], N ∈ [1, 1048576] output: Type: int8, int16, int32 Shape: [*,M,N] Other constraints: Same as lhs and rhs |
| Max | lhs: Type: int8, int16, float16, float32 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: The input and output types need to be the same. Shape: [*] |
| MaxPool | input: Type: int8, int16 Shape: [*,H,W,C] output: The input and output types need to be the same. kernel: Shape: [KL] or [KH,KW], only support 1d or 2d now Dim: 1d: KL ∈ [1, 256], KL*bitWidth/8 <= 24576; 2d: KH, KW ∈ [1, 256], KH*KW*bitWidth/8 <= 24576 stride: Shape: [SH,SW] or [SL] Dim: SH, SW, SL ∈ [1, 256] pad: Shape: [PH_BEGIN,PW_BEGIN,PH_END,PW_END] or [PL_BEGIN,PL_END] PH_BEGIN,PW_BEGIN,PL_BEGIN,PH_END,PW_END,PL_END ∈ [-255, 256] dilation: Shape: 1d: [DW]; 2d: [DH, DW] Dim: 1d: DW ∈ {1}; 2d: DH, DW ∈ {1} |
| Min | lhs: Type: int8, int16, float16, float32 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: The input and output types need to be the same. Shape: [*] |
| Mish | input: Type: int8, int16 output: Type: int8, int16. Only support (int8->int8), (int16->int8,int16) |
| Mod | lhs: Type: int16, int32. Not support quantized type. Shape: [*] Value range: must be none-negative. rhs: The lhs and rhs types need to be the same. Shape: [*] Value range: must be positive. output: The input and output types need to be the same. Shape: [*] |
| Mul | lhs: Type: int8, int16, float16, float32 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: Type: int8, int16, int32, float16, float32. only support (int8,int16→int8,int16,int32), (float16→float16), (float32→float32). Shape: [*] others: When inputs are of integer or quantized type, if the result overflows, it will saturate to the maximum or minimum value of the result data type. |
| Neg | input: Type: int8, int16, int32, float16 Shape: [*] output: The input and output types need to be the same. Shape: [*] |
| Not | input: Type: int8, int16, bool8, float16 Shape: [*] output: Type: bool8 Shape: [*] |
| Or | lhs: Type: int8, int16, bool8, float16 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: Type: bool8 Shape: [*] |
| PRelu | lhs: Type: int8, int16 Shape: [*] rhs: The type and shape is same as lhs output: The type and shape is same as lhs |
| Pad | N/A, collapsed in graph optimization phase |
| Pow | If the first input is feature and the second input is initializer with a single element: input: Type: int8, int16 output: Type: int8, int16. Only support (int8->int8), (int16->int8,int16) Otherwise: input: Type: float16 output: Type: float16 |
| Reciprocal | input: Type: int8, int16 output: Type: int8, int16. Only support (int8->int8), (int16->int8,int16) |
| ReduceL2 | input: Type: int8, int16 Shape: [*] Dim: reduce axis dim size ∈ [1, 65535] Element : reduce Elements size ∈ [1, 65535] output: Same as input |
| ReduceMax | input: Type: int8, int16, int32, float16, float32 Shape: [*] Dim: if type is int8 or int16, reduce axis dim size ∈ [1, 65535] Element : if type is int8 or int16, reduce Elements size ∈ [1, 65535] output: Type: The input and output types need to be the same. Shape: reduce dim will be 1 or fused depend on keepDim |
| ReduceMean | input: Type: int8, int16 float16, float32 Shape: [*] Dim: reduce axis dim size ∈ [1, 65535] Element : reduce Elements size ∈ [1, 65535] output: Type: int8, int16, float16, float32. only support combinations (int8→int8,int16), (int16→int16), (float16→float16), (float32→float32) Shape: reduce dim will be 1 or fused depend on keepDim dims: If type is float16 or float32, only support one dim. |
| ReduceMin | input: Type: int8, int16, int32, float16, float32 Shape: [*] Dim: if type is int8 or int16, reduce axis dim size ∈ [1, 65535] Element : if type is int8 or int16, reduce Elements size ∈ [1, 65535] output: Type: The input and output types need to be the same. Shape: reduce dim will be 1 or fused depend on keepDim |
| ReduceSum | input: Type: int8, int16 float16, float32 Shape: [*] Dim: reduce axis dim size ∈ [1, 65535] Element : reduce Elements size ∈ [1, 65535] output: Type: int8, int16, float16, float32. only support combinations (int8→int8,int16), (int16→int16), (float16→float16), (float32→float32) Shape: reduce dim will be 1 or fused depend on keepDim dims: If type is float16 or float32, only support one dim. |
| Relu | N/A, collapsed in graph optimization phase |
| Reshape | input: Type: No limits output: The input and output types need to be the same. |
| Resize | input: Type: int8, float16 output: Type: int8, float16. Only support (int8->int8), (float16->float16) |
| Round | input: Type: int8, int16 output: Type: int8, int16. Only support (int8->int8), (int16->int8,int16) |
| Selu | input: Type: int8, int16 output: Type: int8, int16. Only support (int8->int8), (int16->int8,int16) |
| Shape | N/A, collapsed in graph optimization phase |
| Sigmoid | input: Type: int8, int16 output: Type: int8, int16. Only support (int8->int8), (int16->int8,int16) |
| Sign | input: Type: int8, int16 output: Type: int8, int16. Only support (int8->int8), (int16->int8,int16) |
| Sin | input: Type: int8, int16 output: Type: int8, int16. Only support (int8->int8), (int16->int8,int16) |
| Sinh | input: Type: int8, int16 output: Type: int8, int16. Only support (int8->int8), (int16->int8,int16) |
| Size | N/A, collapsed in graph optimization phase |
| Softmax | input: Type: int8, int16 Shape: [*] Dim: reduce axis dim size ∈ [1, 65535] Element : reduce Elements size ∈ [1, 65535] output: Same as input |
| Softplus | input: Type: int8, int16 output: Type: int8, int16. Only support (int8->int8), (int16->int8,int16) |
| Softsign | input: Type: int8, int16 output: Type: int8, int16. Only support (int8->int8), (int16->int8,int16) |
| SpaceToDepth | input: Type: No limits output: The input and output types need to be the same. |
| Split | input: Type: No limits Dim: all dims < 2097152 output: The input and output types need to be the same. The input and output constraints are the same. |
| Sqrt | input: Type: int8, int16 output: Type: int8, int16. Only support (int8->int8), (int16->int8,int16) |
| Squeeze | input: Type: No limits output: The input and output types need to be the same. |
| Sub | lhs: Type: int8, int16, float16 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: The input and output types need to be the same. Shape: [*] others: When inputs are of integer or quantized type, if the result overflows, it will saturate to the maximum or minimum value of the result data type. |
| Sum | lhs: Type: int8, int16, int32, float16, float32 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: The input and output types need to be the same. Shape: [*] others: When inputs are of integer or quantized type, if the result overflows, it will saturate to the maximum or minimum value of the result data type. |
| Tan | input: Type: int8, int16 output: Type: int8, int16. Only support (int8->int8), (int16->int8,int16) |
| Tanh | input: Type: int8, int16 output: Type: int8, int16. Only support (int8->int8), (int16->int8,int16) |
| ThresholdedRelu | input: Type: int8, int16 output: Type: int8, int16. Only support (int8->int8), (int16->int8,int16) |
| Tile | input: Type: No limits output: The input and output types need to be the same. |
| TopK | input: Type: int8, int16, int32, float16, float32 output: The type of output value is same as input, and indices is integer type. k: K <= 1024 others: The combined size in bytes of the op's operands and outputs must be no more than 4.8MB. |
| Transpose | input: Type: No limits output: The input and output types need to be the same. |
| Trilu | lhs: Type: int8, int16, float16, float32 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: Type: int8, int16, int32, float16, float32. only support (int8,int16→int8,int16,int32), (float16→float16), (float32→float32). Shape: [*] others: When inputs are of integer or quantized type, if the result overflows, it will saturate to the maximum or minimum value of the result data type. |
| Unsqueeze | input: Type: No limits output: The input and output types need to be the same. |
| Where | condition: Type: bool8 Shape: [*] lhs: Type: int8, int16 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: Type: float16, float32 Shape: [*] input: Type: int8,int16,int32 Shape: [*] |
| Xor | lhs: Type: int8, int16, bool8, float16 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: Type: bool8 Shape: [*] |