#Benchmark of Model Performance

#Descriptions

• Test Conditions:

  • Test Board: S100.

  • Number of Test Cores: Single core.

  • Frequency to obtain model performance data: Average of performance parameters over a 5-minute period.

  • Python version: Python 3.10.

  • Models: Models under the samples/ucp_tutorial/dnn/ai_benchmark/s100 path in the OE package.

  • Runtime Environment: Linux.

• Table Header Acronyms:

  • C = Computation, in GOPs (i.e., billion operations per second), obtained by calling the hbm_perf interface.

  • FPS = Frame(s) Per Second, obtained by running the fps.sh script with multi thread of different models in the ai_benchmark sample package/script on the dev board. Post-processing included.

  • ITC = Inference Time Consumption, in ms (millisecond), obtained by running the latency.sh script with single thread of different models in the ai_benchmark sample package/script on the dev board. Post-processing not included.

  • TCPP = Postprocess Time Consumption, in ms (millisecond), obtained by running the latency.sh script with single thread of different models in the ai_benchmark sample package/script on the dev board.

  • RV = Read Volume in a single inference, in mb (Mbit), obtained by calling the hbm_perf interface.

  • WV = Write Volume in a single inference, in mb (Mbit), obtained by calling the hbm_perf interface.

#Model Key Performance Data

#Model Full Performance Data

#MobileNetv1

• INPUT SIZE: 1x3x224x224
• C(GOPs): 1.14
• FPS: 4594.40
• ITC(ms): 0.466
• TCPP(ms): 0.033
• RV(mb): 4.56
• WV(mb): 0.0041
• Dataset: ImageNet
• ACCURACY: Top1: 0.7374(FLOAT)/0.7295(INT8)

#MobileNetv2

• INPUT SIZE: 1x3x224x224
• C(GOPs): 0.63
• FPS: 4591.90
• ITC(ms): 0.487
• TCPP(ms): 0.033
• RV(mb): 3.95
• WV(mb): 0.0041
• Dataset: ImageNet
• ACCURACY: Top1: 0.7218(FLOAT)/0.7146(INT8)

#ResNet50

• INPUT SIZE: 1x3x224x224
• C(GOPs): 7.72
• FPS: 1152.80
• ITC(ms): 1.161
• TCPP(ms): 0.033
• RV(mb): 26.08
• WV(mb): 0.0041
• Dataset: ImageNet
• ACCURACY: Top1: 0.7704(FLOAT)/0.7673(INT8)

#GoogleNet

• INPUT SIZE: 1x3x224x224
• C(GOPs): 3.00
• FPS: 2858.20
• ITC(ms): 0.636
• TCPP(ms): 0.032
• RV(mb): 7.16
• WV(mb): 0.0041
• Dataset: ImageNet
• ACCURACY: Top1: 0.7018(FLOAT)/0.6998(INT8)

#EfficientNet_Lite0

• INPUT SIZE: 1x224x224x3
• C(GOPs): 0.77
• FPS: 4068.20
• ITC(ms): 0.559
• TCPP(ms): 0.032
• RV(mb): 5.44
• WV(mb): 0.0041
• Dataset: ImageNet
• ACCURACY: Top1: 0.7479(FLOAT)/0.7454(INT8)

#EfficientNet_Lite1

• INPUT SIZE: 1x240x240x3
• C(GOPs): 1.20
• FPS: 3231.60
• ITC(ms): 0.624
• TCPP(ms): 0.032
• RV(mb): 6.34
• WV(mb): 0.0041
• Dataset: ImageNet
• ACCURACY: Top1: 0.7652(FLOAT)/0.7614(INT8)

#EfficientNet_Lite2

• INPUT SIZE: 1x260x260x3
• C(GOPs): 1.72
• FPS: 2478.80
• ITC(ms): 0.723
• TCPP(ms): 0.032
• RV(mb): 7.18
• WV(mb): 0.0041
• Dataset: ImageNet
• ACCURACY: Top1: 0.7734(FLOAT)/0.7697(INT8)

#EfficientNet_Lite3

• INPUT SIZE: 1x280x280x3
• C(GOPs): 2.77
• FPS: 1898.20
• ITC(ms): 0.841
• TCPP(ms): 0.032
• RV(mb): 9.62
• WV(mb): 0.0041
• Dataset: ImageNet
• ACCURACY: Top1: 0.7917(FLOAT)/0.7896(INT8)

#EfficientNet_Lite4

• INPUT SIZE: 1x300x300x3
• C(GOPs): 5.11
• FPS: 1300.50
• ITC(ms): 1.082
• TCPP(ms): 0.032
• RV(mb): 14.70
• WV(mb): 0.0041
• Dataset: ImageNet
• ACCURACY: Top1: 0.8063(FLOAT)/0.8043(INT8)

#Vargconvnet

• INPUT SIZE: 1x3x224x224
• C(GOPs): 9.06
• FPS: 1464.80
• ITC(ms): 0.974
• TCPP(ms): 0.032
• RV(mb): 10.67
• WV(mb): 0.0041
• Dataset: ImageNet
• ACCURACY: Top1: 0.7793(FLOAT)/0.7770(INT8)

#Efficientnasnet_m

• INPUT SIZE: 1x3x300x300
• C(GOPs): 4.53
• FPS: 1477.10
• ITC(ms): 0.971
• TCPP(ms): 0.032
• RV(mb): 13.99
• WV(mb): 0.0041
• Dataset: ImageNet
• ACCURACY: Top1: 0.7935(FLOAT)/0.7923(INT8)

#Efficientnasnet_s

• INPUT SIZE: 1x3x280x280
• C(GOPs): 1.44
• FPS: 3327.70
• ITC(ms): 0.590
• TCPP(ms): 0.031
• RV(mb): 5.68
• WV(mb): 0.0041
• Dataset: ImageNet
• ACCURACY: Top1: 0.7441(FLOAT)/0.7524(INT8)

#ResNet18

• INPUT SIZE: 1x3x224x224
• C(GOPs): 3.63
• FPS: 2567.80
• ITC(ms): 0.671
• TCPP(ms): 0.032
• RV(mb): 11.87
• WV(mb): 0.0041
• Dataset: ImageNet
• ACCURACY: Top1: 0.7170(FLOAT)/0.7162(INT8)

#YOLOv2_Darknet19

• INPUT SIZE: 1x3x608x608
• C(GOPs): 62.94
• FPS: 227.64
• ITC(ms): 4.712
• TCPP(ms): 0.293
• RV(mb): 51.86
• WV(mb): 0.77
• Dataset: COCO
• ACCURACY: [IoU=0.50:0.95]= 0.2760(FLOAT)/0.2700(INT8)

#YOLOv3_Darknet53

• INPUT SIZE: 1x3x416x416
• C(GOPs): 65.86
• FPS: 210.85
• ITC(ms): 5.101
• TCPP(ms): 1.611
• RV(mb): 68.43
• WV(mb): 9.38
• Dataset: COCO
• ACCURACY: [IoU=0.50:0.95]= 0.3370(FLOAT)/0.3360(INT8)

#YOLOv5x_v2.0

• INPUT SIZE: 1x3x672x672
• C(GOPs): 243.85
• FPS: 62.44
• ITC(ms): 16.437
• TCPP(ms): 5.683
• RV(mb): 140.18
• WV(mb): 50.31
• Dataset: COCO
• ACCURACY: [IoU=0.50:0.95]= 0.4810(FLOAT)/0.4670(INT8)

#SSD_MobileNetv1

• INPUT SIZE: 1x3x300x300
• C(GOPs): 2.30
• FPS: 3223.40
• ITC(ms): 0.687
• TCPP(ms): 0.184
• RV(mb): 6.24
• WV(mb): 0.21
• Dataset: VOC
• ACCURACY: mAP: 0.7345(FLOAT)/0.7269(INT8)

#Centernet_resnet101

• INPUT SIZE: 1x3x512x512
• C(GOPs): 90.53
• FPS: 186.75
• ITC(ms): 5.714
• TCPP(ms): 0.928
• RV(mb): 59.17
• WV(mb): 11.80
• Dataset: COCO
• ACCURACY: [IoU=0.50:0.95]= 0.3420(FLOAT)/0.3270(INT8)

#YOLOv3_VargDarknet

• INPUT SIZE: 1x3x416x416
• C(GOPs): 42.82
• FPS: 307.08
• ITC(ms): 3.616
• TCPP(ms): 1.575
• RV(mb): 46.52
• WV(mb): 4.91
• Dataset: COCO
• ACCURACY: [IoU=0.50:0.95]= 0.3280(FLOAT)/0.3270(INT8)

#Deeplabv3plus_efficientnetb0

• INPUT SIZE: 1x3x1024x2048
• C(GOPs): 30.77
• FPS: 152.01
• ITC(ms): 6.968
• TCPP(ms): 0.254
• RV(mb): 22.09
• WV(mb): 11.80
• Dataset: Cityscapes
• ACCURACY: mIoU: 0.7630(FLOAT)/0.7571(INT8)

#Fastscnn_efficientnetb0

• INPUT SIZE: 1x3x1024x2048
• C(GOPs): 12.48
• FPS: 253.93
• ITC(ms): 4.278
• TCPP(ms): 0.264
• RV(mb): 11.58
• WV(mb): 9.24
• Dataset: Cityscapes
• ACCURACY: mIoU: 0.6997(FLOAT)/0.6914(INT8)

#Deeplabv3plus_efficientnetm1

• INPUT SIZE: 1x3x1024x2048
• C(GOPs): 77.04
• FPS: 92.28
• ITC(ms): 11.184
• TCPP(ms): 0.264
• RV(mb): 86.77
• WV(mb): 55.57
• Dataset: Cityscapes
• ACCURACY: mIoU: 0.7794(FLOAT)/0.7754(INT8)

#Deeplabv3plus_efficientnetm2

• INPUT SIZE: 1x3x1024x2048
• C(GOPs): 124.15
• FPS: 64.83
• ITC(ms): 15.818
• TCPP(ms): 0.255
• RV(mb): 155.15
• WV(mb): 89.13
• Dataset: Cityscapes
• ACCURACY: mIoU: 0.7882(FLOAT)/0.7854(INT8)

#Bev_gkt_mixvargenet_multitask

• INPUT SIZE: image: 6x3x512x960 points(0-8): 6x64x64x2
• C(GOPs): 207.16
• FPS: 68.54
• ITC(ms): 15.425
• TCPP(ms): 4.204
• RV(mb): 121.70
• WV(mb): 97.64
• Dataset: Nuscenes
• ACCURACY: NDS: 0.2810(FLOAT)/0.2782(INT8) MeanIOU: 0.4852(FLOAT)/0.4836(INT8) mAP: 0.1990(FLOAT)/0.1997(INT8)

#Bev_ipm_4d_efficientnetb0_multitask

• INPUT SIZE: image: 6x3x512x960 points: 6x128x128x2 prev_feat: 1x164x28x128 prev_point: 1x128x128x2
• C(GOPs): 53.58
• FPS: 112.15
• ITC(ms): 9.898
• TCPP(ms): 4.163
• RV(mb): 65.63
• WV(mb): 51.93
• Dataset: Nuscenes
• ACCURACY: NDS: 0.3721(FLOAT)/0.3728(INT8) MeanIOU: 0.5287(FLOAT)/0.5388(INT8) mAP: 0.2200(FLOAT)/0.2216(INT8)

#Bev_ipm_efficientnetb0_multitask

• INPUT SIZE: image: 6x3x512x960 points: 6x128x128x2
• C(GOPs): 52.97
• FPS: 115.33
• ITC(ms): 9.443
• TCPP(ms): 4.250
• RV(mb): 62.33
• WV(mb): 49.84
• Dataset: Nuscenes
• ACCURACY: NDS: 0.3054(FLOAT)/0.3041(INT8) MeanIOU: 0.5145(FLOAT)/0.5104(INT8) mAP: 0.2170(FLOAT)/0.2163(INT8)

#Bev_lss_efficientnetb0_multitask

• INPUT SIZE: image: 6x3x256x704 points(0&1): 10x128x128x2
• C(GOPs): 24.06
• FPS: 186.93
• ITC(ms): 6.116
• TCPP(ms): 4.191
• RV(mb): 25.42
• WV(mb): 19.02
• Dataset: Nuscenes
• ACCURACY: NDS: 0.3007(FLOAT)/0.2991(INT8) MeanIOU: 0.5180(FLOAT)/0.5147(INT8) mAP: 0.2062(FLOAT)/0.2040(INT8)

#Detr3d_efficientnetb3

• INPUT SIZE: coords(0-3): 6x4x256x2 image: 6x3x512x1408 masks: 1x4x256x24
• C(GOPs): 227.71
• FPS: 32.09
• ITC(ms): 31.709
• TCPP(ms): 1.111
• RV(mb): 330.69
• WV(mb): 176.11
• Dataset: Nuscenes
• ACCURACY: NDS: 0.3304(FLOAT)/0.3285(INT8) mAP: 0.2753(FLOAT)/0.2708(INT8)

#Petr_efficientnetb3

• INPUT SIZE: image: 6x3x512x1408 pos_embed: 1x96x44x256
• C(GOPs): 219.17
• FPS: 18.89
• ITC(ms): 53.507
• TCPP(ms): 1.127
• RV(mb): 298.77
• WV(mb): 184.89
• Dataset: Nuscenes
• ACCURACY: NDS: 0.3765(FLOAT)/0.3731(INT8) mAP: 0.3038(FLOAT)/0.2925(INT8)

#Bevformer_tiny_resnet50_detection

• INPUT SIZE: img: 6x3x480x800 prev_bev: 1x2500x256 prev_bev_ref: 1x50x50x2 queries_rebatch_grid: 6x20x32x2 restore_bev_grid: 1x100x50x2 reference_points_rebatch: 6x640x4x2 bev_pillar_counts: 1x2500x1
• C(GOPs): 387.29
• FPS: 31.29
• ITC(ms): 41.686
• TCPP(ms): 1.200
• RV(mb): 266.60
• WV(mb): 176.56
• Dataset: Nuscenes
• ACCURACY: NDS: 0.3713(FLOAT)/0.3680(INT8) mAP: 0.2673(FLOAT)/0.2619(INT8)

#Flashocc_henet_lss_occ3d_nuscenes

• INPUT SIZE: img: 6x3x512x960 points: 10x128x128x2 points_depth: 10x128x128x2
• C(GOPs): 126.75
• FPS: 96.38
• ITC(ms): 11.232
• TCPP(ms): 40.562
• RV(mb): 87.45
• WV(mb): 55.84
• Dataset: Nuscenes
• ACCURACY: mIoU: 0.3674(FLOAT)/0.3693(INT8)

#Horizon_swin_transformer

• INPUT SIZE: 1x3x224x224
• C(GOPs): 8.98
• FPS: 308.77
• ITC(ms): 3.549
• TCPP(ms): 0.032
• RV(mb): 46.54
• WV(mb): 7.51
• Dataset: ImageNet
• ACCURACY: Top1: 0.8024(FLOAT)/0.7959(INT8)

#Mixvargenet

• INPUT SIZE: 1x3x224x224
• C(GOPs): 2.07
• FPS: 4742.70
• ITC(ms): 0.467
• TCPP(ms): 0.033
• RV(mb): 2.51
• WV(mb): 0.0041
• Dataset: ImageNet
• ACCURACY: Top1: 0.7075(FLOAT)/0.7049(INT8)

#Vargnetv2

• INPUT SIZE: 1x3x224x224
• C(GOPs): 0.72
• FPS: 4423.60
• ITC(ms): 0.520
• TCPP(ms): 0.033
• RV(mb): 4.68
• WV(mb): 0.0041
• Dataset: ImageNet
• ACCURACY: Top1: 0.7342(FLOAT)/0.7326(INT8)

#Vit_small

• INPUT SIZE: 1x3x224x224
• C(GOPs): 9.20
• FPS: 568.63
• ITC(ms): 2.054
• TCPP(ms): 0.032
• RV(mb): 26.15
• WV(mb): 0.0041
• Dataset: ImageNet
• ACCURACY: Top1: 0.7950(FLOAT)/0.7937(INT8)

#Centerpoint_pointpillar

• INPUT SIZE: points: 300000x5 voxel_feature: 1x5x20x40000 coors: 40000x4
• C(GOPs): 127.73
• FPS: 125.15
• ITC(ms): 15.344
• TCPP(ms): 12.889
• RV(mb): 51.36
• WV(mb): 25.83
• Dataset: Nuscenes
• ACCURACY: NDS: 0.5832(FLOAT)/0.5816(INT8) mAP: 0.4804(FLOAT)/0.4784(INT8)

#Detr_efficientnetb3

• INPUT SIZE: 1x3x800x1333
• C(GOPs): 67.39
• FPS: 51.95
• ITC(ms): 19.619
• TCPP(ms): 0.351
• RV(mb): 261.38
• WV(mb): 137.07
• Dataset: MS COCO
• ACCURACY: [IoU=0.50:0.95]= 0.3720(FLOAT)/0.3600(INT8)

#Detr_resnet50

• INPUT SIZE: 1x3x800x1333
• C(GOPs): 203.07
• FPS: 40.20
• ITC(ms): 25.334
• TCPP(ms): 0.345
• RV(mb): 352.78
• WV(mb): 224.48
• Dataset: MS COCO
• ACCURACY: [IoU=0.50:0.95]= 0.3569(FLOAT)/0.3160(INT8)

#FCOS3D_efficientnetb0

• INPUT SIZE: 1x3x512x896
• C(GOPs): 19.94
• FPS: 448.94
• ITC(ms): 2.987
• TCPP(ms): 2.723
• RV(mb): 11.23
• WV(mb): 4.17
• Dataset: nuscenes
• ACCURACY: NDS: 0.3061(FLOAT)/0.3030(INT8) mAP: 0.2133(FLOAT)/0.2069(INT8)

#Fcos_efficientnetb0

• INPUT SIZE: 1x3x512x512
• C(GOPs): 5.02
• FPS: 1079.10
• ITC(ms): 1.513
• TCPP(ms): 0.053
• RV(mb): 6.09
• WV(mb): 2.68
• Dataset: MS COCO
• ACCURACY: [IoU=0.50:0.95]= 0.3626(FLOAT)/0.3553(INT8)

#Ganet_mixvargenet

• INPUT SIZE: 1x3x320x800
• C(GOPs): 10.74
• FPS: 1574.50
• ITC(ms): 0.950
• TCPP(ms): 0.206
• RV(mb): 2.16
• WV(mb): 0.52
• Dataset: CuLane
• ACCURACY: F1Score: 0.7949(FLOAT)/0.7883(INT8)

#Keypoint_efficientnetb0

• INPUT SIZE: 1x3x128x128
• C(GOPs): 0.45
• FPS: 4533.80
• ITC(ms): 0.500
• TCPP(ms): 0.071
• RV(mb): 4.62
• WV(mb): 0.01
• Dataset: Carfusion
• ACCURACY: PCK(alpha=0.1): 0.9433(FLOAT)/0.9432(INT8)

#Pointpillars_kitti_car

• INPUT SIZE: 150000x4
• C(GOPs): 66.82
• FPS: 147.90
• ITC(ms): 30.921
• TCPP(ms): 0.431
• RV(mb): 69.93
• WV(mb): 30.00
• Dataset: Kitti3d
• ACCURACY: APDet= 0.7731(FLOAT)/0.7678(INT8)

#Deformable_detr_resnet50

• INPUT SIZE: 1x3x800x1333
• C(GOPs): 408.94
• FPS: 5.38
• ITC(ms): 186.460
• TCPP(ms): 15.689
• RV(mb): 3476.02
• WV(mb): 2497.51
• Dataset: MS COCO
• ACCURACY: [IoU=0.50:0.95]= 0.4413(FLOAT)/0.4194(INT8)

#Stereonetplus_mixvargenet

• INPUT SIZE: 2x3x544x960
• C(GOPs): 48.57
• FPS: 223.90
• ITC(ms): 4.863
• TCPP(ms): 1.874
• RV(mb): 30.64
• WV(mb): 28.07
• Dataset: SceneFlow
• ACCURACY: EPE: 1.1270(FLOAT)/1.1341(INT8)

#Centerpoint_mixvargnet_multitask

• INPUT SIZE: points: 300000x5 voxel_feature: 1x5x20x40000 coors: 40000x4
• C(GOPs): 51.45
• FPS: 186.72
• ITC(ms): 13.003
• TCPP(ms): 11.736
• RV(mb): 32.08
• WV(mb): 16.79
• Dataset: Nuscenes
• ACCURACY: NDS: 0.5809(FLOAT)/0.5753(INT8) MeanIOU: 0.9128(FLOAT)/0.9121(INT8) mAP: 0.4727(FLOAT)/0.4626(INT8)

#Unet_mobilenetv1

• INPUT SIZE: 1x3x1024x2048
• C(GOPs): 7.36
• FPS: 811.04
• ITC(ms): 1.676
• TCPP(ms): 0.116
• RV(mb): 13.27
• WV(mb): 7.60
• Dataset: Cityscapes
• ACCURACY: mIoU: 0.6802(FLOAT)/0.6758(INT8)

#Motr_efficientnetb3

• INPUT SIZE: image: 1x800x1422x3 track_query: 1x2x128x156 ref_points: 1x2x128x4 mask_query: 1x1x256x1
• C(GOPs): 64.43
• FPS: 73.73
• ITC(ms): 13.737
• TCPP(ms): 4.958
• RV(mb): 115.40
• WV(mb): 47.22
• Dataset: Mot17
• ACCURACY: MOTA: 0.5798(FLOAT)/0.5762(INT8)

#Densetnt_vectornet

• INPUT SIZE: goals_2d: 30x1x2048x2 goals_2d_mask: 30x1x2048x1 instance_mask: 30x1x96x1 lane_feat: 30x9x64x11 traj_feat: 30x19x32x9
• C(GOPs): 12.50
• FPS: 149.20
• ITC(ms): 7.543
• TCPP(ms): 2.248
• RV(mb): 56.79
• WV(mb): 34.86
• Dataset: Argoverse 1
• ACCURACY: minFDA: 1.2975(FLOAT)/1.3023(INT8)

#Maptroe_henet_tinym_bevformer

• INPUT SIZE: img: 6x3x480x800 osm_mask: 1x1x50x100 queries_rebatch_grid: 6x20x100x2 restore_bev_grid: 1x100x100x2 reference_points_rebatch: 6x2000x4x2 bev_pillar_counts: 1x5000x1
• C(GOPs): 134.57
• FPS: 75.21
• ITC(ms): 13.826
• TCPP(ms): 0.258
• RV(mb): 119.91
• WV(mb): 35.37
• Dataset: Nuscenes
• ACCURACY: mAP: 0.6633(FLOAT)/0.6565(INT8)

#Qcnet_oe

• INPUT SIZE: valid_mask: 1x30x10 valid_mask_a2a: 1x10x30x30 agent_type: 1x30x1 x_a_cur: 1x1x30x1,1x1x30x1,1x1x30x1,1x1x30x1 r_pl2a_cur: 1x1x30x80,1x1x30x80,1x1x30x80 r_t_cur: 1x1x30x6,1x1x30x6,1x1x30x6,1x1x30x6 r_a2a_cur: 1x1x30x30,1x1x30x30,1x1x30x30 x_a_mid_emb: 1x30x2x128 x_a: 1x30x6x128 pl_type,is_intersection: 1x80 r_pl2pl: 1x1x80x80,1x1x80x80,1x1x80x80 r_pt2pl: 1x1x80x50,1x1x80x50,1x1x80x50 mask_pl2pl: 1x80x80 magnitude,pt_type,side,mask: 1x80x50 mask_a2m: 1x30x30 mask_dst: 1x30x1 type_pl2pl: 1x80x80
• C(GOPs): 7.85
• FPS: 234.88
• ITC(ms): 5.384
• TCPP(ms): 0.897
• RV(mb): 36.03
• WV(mb): 17.51
• Dataset: Argoverse 2
• ACCURACY: hitrate: 0.8026(FLOAT)/0.7953(INT8)