Skip to content

Benchmarks 2024 11 21 TVM LLVM O3 spike_rv64

GitHub Action edited this page Nov 21, 2024 · 1 revision

Setup

Simulator

  • Spike (riscv-isa-sim ) (ISS, CPI=1)
    • Spike : 0bc176b3fca43560b9e8586cdbc41cfde073e17a
    • Spike PK : 7e9b671c0415dfd7b562ac934feb9380075d4aa2

Toolchains

Models

Frameworks

  • MLonMCU : develop

  • TVM : Nightly Pre-Build

Miscellaneous

  • Used -Os flag for compilation.
  • Benchmarks generated using MLonMCU deployment tool with minimal efforts.
  • Memory metrics are reported in Bytes

Results (Framework: tvm, Backend: tvmaot, Toolchain: llvm, Flags: -O3, Target: spike_rv64 )

Audio Wake Words (aww)

Cycles (Speedup) Total ROM (rel.) Total RAM (rel.) VLEN Layout Kernels Mode Arch Unroll Auto-Vectorization
13151076
( 1.2x )
123276
( 1.444 )
61288
( 2.912 )
0 NCHW TVM Fallback RV64GC 0 -
20156731
( 0.8x )
106608
( 1.249 )
61280
( 2.911 )
0 NHWC TVM Fallback RV64GC 0 -
25188588
( 0.6x )
177380
( 2.078 )
61280
( 2.911 )
128 NCHW TVM Fallback RV64GCV 0 Loop+SLP
25190699
( 0.6x )
177380
( 2.078 )
61280
( 2.911 )
256 NCHW TVM Fallback RV64GCV 0 Loop+SLP
25193467
( 0.6x )
177380
( 2.078 )
61280
( 2.911 )
512 NCHW TVM Fallback RV64GCV 0 Loop+SLP
25199018
( 0.6x )
177380
( 2.078 )
61280
( 2.911 )
1024 NCHW TVM Fallback RV64GCV 0 Loop+SLP
25211658
( 0.6x )
177380
( 2.078 )
61280
( 2.911 )
2048 NCHW TVM Fallback RV64GCV 0 Loop+SLP
25235903
( 0.6x )
177380
( 2.078 )
61280
( 2.911 )
4096 NCHW TVM Fallback RV64GCV 0 Loop+SLP
3633236
( 4.3x )
125408
( 1.469 )
61280
( 2.911 )
128 NHWC TVM Fallback RV64GCV 0 Loop+SLP
2200422
( 7.1x )
125408
( 1.469 )
61280
( 2.911 )
256 NHWC TVM Fallback RV64GCV 0 Loop+SLP
1484795
( 10.6x )
125408
( 1.469 )
61280
( 2.911 )
512 NHWC TVM Fallback RV64GCV 0 Loop+SLP
1637211
( 9.6x )
125408
( 1.469 )
61280
( 2.911 )
1024 NHWC TVM Fallback RV64GCV 0 Loop+SLP
20562293
( 0.8x )
125408
( 1.469 )
61280
( 2.911 )
2048 NHWC TVM Fallback RV64GCV 0 Loop+SLP
20564158
( 0.8x )
125408
( 1.469 )
61280
( 2.911 )
4096 NHWC TVM Fallback RV64GCV 0 Loop+SLP
15676964
( Base )
85356
( Base )
21048
( Base )
0 NHWC muRISCV-NN Scalar RV64GC 0 -
15585850
( 1.0x )
85248
( 0.999 )
25512
( 1.212 )
0 NHWC muRISCV-NN Vector (Portable) RV64GC 0 -
7078502
( 2.2x )
93966
( 1.101 )
21048
( 1.0 )
128 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
6492170
( 2.4x )
93966
( 1.101 )
21048
( 1.0 )
256 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
6202043
( 2.5x )
93966
( 1.101 )
21048
( 1.0 )
512 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
6057912
( 2.6x )
93966
( 1.101 )
21048
( 1.0 )
1024 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
14530525
( 1.1x )
93966
( 1.101 )
21048
( 1.0 )
2048 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
14532390
( 1.1x )
93966
( 1.101 )
21048
( 1.0 )
4096 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
6000272
( 2.6x )
87942
( 1.03 )
25512
( 1.212 )
128 NHWC muRISCV-NN Vector RV64GCV 0 -
4866264
( 3.2x )
87942
( 1.03 )
25512
( 1.212 )
256 NHWC muRISCV-NN Vector RV64GCV 0 -
4307992
( 3.6x )
87942
( 1.03 )
25512
( 1.212 )
512 NHWC muRISCV-NN Vector RV64GCV 0 -
4268873
( 3.7x )
87942
( 1.03 )
25512
( 1.212 )
1024 NHWC muRISCV-NN Vector RV64GCV 0 -
4268873
( 3.7x )
87942
( 1.03 )
25512
( 1.212 )
2048 NHWC muRISCV-NN Vector RV64GCV 0 -
4270738
( 3.7x )
87942
( 1.03 )
25512
( 1.212 )
4096 NHWC muRISCV-NN Vector RV64GCV 0 -
8198450
( 1.9x )
94596
( 1.108 )
25512
( 1.212 )
128 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
7612118
( 2.1x )
94596
( 1.108 )
25512
( 1.212 )
256 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
7321991
( 2.1x )
94596
( 1.108 )
25512
( 1.212 )
512 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
7177860
( 2.2x )
94596
( 1.108 )
25512
( 1.212 )
1024 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
15650473
( 1.0x )
94596
( 1.108 )
25512
( 1.212 )
2048 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
15654203
( 1.0x )
94596
( 1.108 )
25512
( 1.212 )
4096 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP

Image Classification (resnet)

Cycles (Speedup) Total ROM (rel.) Total RAM (rel.) VLEN Layout Kernels Mode Arch Unroll Auto-Vectorization
58410657
( 1.0x )
224164
( 1.689 )
110208
( 1.921 )
0 NCHW TVM Fallback RV64GC 0 -
67859808
( 0.8x )
221728
( 1.671 )
110208
( 1.921 )
0 NHWC TVM Fallback RV64GC 0 -
110153895
( 0.5x )
291112
( 2.194 )
110208
( 1.921 )
128 NCHW TVM Fallback RV64GCV 0 Loop+SLP
110118149
( 0.5x )
291108
( 2.194 )
110208
( 1.921 )
256 NCHW TVM Fallback RV64GCV 0 Loop+SLP
110327550
( 0.5x )
291108
( 2.194 )
110208
( 1.921 )
512 NCHW TVM Fallback RV64GCV 0 Loop+SLP
110333107
( 0.5x )
291112
( 2.194 )
110208
( 1.921 )
1024 NCHW TVM Fallback RV64GCV 0 Loop+SLP
110341764
( 0.5x )
291112
( 2.194 )
110208
( 1.921 )
2048 NCHW TVM Fallback RV64GCV 0 Loop+SLP
110360412
( 0.5x )
291108
( 2.194 )
110208
( 1.921 )
4096 NCHW TVM Fallback RV64GCV 0 Loop+SLP
33941538
( 1.6x )
219572
( 1.655 )
110208
( 1.921 )
128 NHWC TVM Fallback RV64GCV 0 Loop+SLP
30816398
( 1.8x )
219558
( 1.654 )
110208
( 1.921 )
256 NHWC TVM Fallback RV64GCV 0 Loop+SLP
29337411
( 1.9x )
219560
( 1.654 )
110208
( 1.921 )
512 NHWC TVM Fallback RV64GCV 0 Loop+SLP
53821895
( 1.0x )
219572
( 1.655 )
110208
( 1.921 )
1024 NHWC TVM Fallback RV64GCV 0 Loop+SLP
70125313
( 0.8x )
219558
( 1.654 )
110208
( 1.921 )
2048 NHWC TVM Fallback RV64GCV 0 Loop+SLP
70127178
( 0.8x )
219558
( 1.654 )
110208
( 1.921 )
4096 NHWC TVM Fallback RV64GCV 0 Loop+SLP
55595602
( Base )
132708
( Base )
57368
( Base )
0 NHWC muRISCV-NN Scalar RV64GC 0 -
71339536
( 0.8x )
132376
( 0.997 )
57368
( 1.0 )
0 NHWC muRISCV-NN Vector (Portable) RV64GC 0 -
13519312
( 4.1x )
142208
( 1.072 )
57368
( 1.0 )
128 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
11083068
( 5.0x )
142208
( 1.072 )
57368
( 1.0 )
256 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
13889346
( 4.0x )
142208
( 1.072 )
57368
( 1.0 )
512 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
15369489
( 3.6x )
142208
( 1.072 )
57368
( 1.0 )
1024 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
16042061
( 3.5x )
142208
( 1.072 )
57368
( 1.0 )
2048 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
35236028
( 1.6x )
142208
( 1.072 )
57368
( 1.0 )
4096 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
15156941
( 3.7x )
135434
( 1.021 )
57368
( 1.0 )
128 NHWC muRISCV-NN Vector RV64GCV 0 -
9573741
( 5.8x )
135434
( 1.021 )
57368
( 1.0 )
256 NHWC muRISCV-NN Vector RV64GCV 0 -
7027901
( 7.9x )
135434
( 1.021 )
57368
( 1.0 )
512 NHWC muRISCV-NN Vector RV64GCV 0 -
5785773
( 9.6x )
135434
( 1.021 )
57368
( 1.0 )
1024 NHWC muRISCV-NN Vector RV64GCV 0 -
4859374
( 11.4x )
135434
( 1.021 )
57368
( 1.0 )
2048 NHWC muRISCV-NN Vector RV64GCV 0 -
4612147
( 12.1x )
135434
( 1.021 )
57368
( 1.0 )
4096 NHWC muRISCV-NN Vector RV64GCV 0 -
16882943
( 3.3x )
141530
( 1.066 )
57368
( 1.0 )
128 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
15848275
( 3.5x )
141530
( 1.066 )
57368
( 1.0 )
256 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
16600784
( 3.3x )
141530
( 1.066 )
57368
( 1.0 )
512 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
16309299
( 3.4x )
141530
( 1.066 )
57368
( 1.0 )
1024 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
15837039
( 3.5x )
141530
( 1.066 )
57368
( 1.0 )
2048 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
42621406
( 1.3x )
141530
( 1.066 )
57368
( 1.0 )
4096 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP

Anomaly Detection (toycar)

Cycles (Speedup) Total ROM (rel.) Total RAM (rel.) VLEN Layout Kernels Mode Arch Unroll Auto-Vectorization
1628946
( 1.0x )
580582
( 1.87 )
7400
( 1.121 )
0 NCHW TVM Fallback RV64GC 0 -
1628946
( 1.0x )
580582
( 1.87 )
7400
( 1.121 )
0 NHWC TVM Fallback RV64GC 0 -
821720
( 2.0x )
583906
( 1.881 )
7400
( 1.121 )
128 NCHW TVM Fallback RV64GCV 0 Loop+SLP
763231
( 2.2x )
583906
( 1.881 )
7400
( 1.121 )
256 NCHW TVM Fallback RV64GCV 0 Loop+SLP
742171
( 2.2x )
583906
( 1.881 )
7400
( 1.121 )
512 NCHW TVM Fallback RV64GCV 0 Loop+SLP
729642
( 2.3x )
583906
( 1.881 )
7400
( 1.121 )
1024 NCHW TVM Fallback RV64GCV 0 Loop+SLP
722445
( 2.3x )
583906
( 1.881 )
7400
( 1.121 )
2048 NCHW TVM Fallback RV64GCV 0 Loop+SLP
862318
( 1.9x )
583906
( 1.881 )
7400
( 1.121 )
4096 NCHW TVM Fallback RV64GCV 0 Loop+SLP
821720
( 2.0x )
583906
( 1.881 )
7400
( 1.121 )
128 NHWC TVM Fallback RV64GCV 0 Loop+SLP
763231
( 2.2x )
583906
( 1.881 )
7400
( 1.121 )
256 NHWC TVM Fallback RV64GCV 0 Loop+SLP
742171
( 2.2x )
583906
( 1.881 )
7400
( 1.121 )
512 NHWC TVM Fallback RV64GCV 0 Loop+SLP
729642
( 2.3x )
583906
( 1.881 )
7400
( 1.121 )
1024 NHWC TVM Fallback RV64GCV 0 Loop+SLP
722445
( 2.3x )
583906
( 1.881 )
7400
( 1.121 )
2048 NHWC TVM Fallback RV64GCV 0 Loop+SLP
862318
( 1.9x )
583906
( 1.881 )
7400
( 1.121 )
4096 NHWC TVM Fallback RV64GCV 0 Loop+SLP
1663942
( Base )
310454
( Base )
6600
( Base )
0 NHWC muRISCV-NN Scalar RV64GC 0 -
1663942
( 1.0x )
310454
( 1.0 )
6600
( 1.0 )
0 NHWC muRISCV-NN Vector (Portable) RV64GC 0 -
577144
( 2.9x )
311718
( 1.004 )
6600
( 1.0 )
128 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
491116
( 3.4x )
311718
( 1.004 )
6600
( 1.0 )
256 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
445900
( 3.7x )
311718
( 1.004 )
6600
( 1.0 )
512 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
423292
( 3.9x )
311718
( 1.004 )
6600
( 1.0 )
1024 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
411988
( 4.0x )
311718
( 1.004 )
6600
( 1.0 )
2048 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
1350517
( 1.2x )
311718
( 1.004 )
6600
( 1.0 )
4096 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
618047
( 2.7x )
311876
( 1.005 )
6600
( 1.0 )
128 NHWC muRISCV-NN Vector RV64GCV 0 -
499679
( 3.3x )
311876
( 1.005 )
6600
( 1.0 )
256 NHWC muRISCV-NN Vector RV64GCV 0 -
440495
( 3.8x )
311876
( 1.005 )
6600
( 1.0 )
512 NHWC muRISCV-NN Vector RV64GCV 0 -
411239
( 4.0x )
311876
( 1.005 )
6600
( 1.0 )
1024 NHWC muRISCV-NN Vector RV64GCV 0 -
407555
( 4.1x )
311876
( 1.005 )
6600
( 1.0 )
2048 NHWC muRISCV-NN Vector RV64GCV 0 -
405671
( 4.1x )
311876
( 1.005 )
6600
( 1.0 )
4096 NHWC muRISCV-NN Vector RV64GCV 0 -
577144
( 2.9x )
311718
( 1.004 )
6600
( 1.0 )
128 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
491116
( 3.4x )
311718
( 1.004 )
6600
( 1.0 )
256 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
445900
( 3.7x )
311718
( 1.004 )
6600
( 1.0 )
512 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
423292
( 3.9x )
311718
( 1.004 )
6600
( 1.0 )
1024 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
411988
( 4.0x )
311718
( 1.004 )
6600
( 1.0 )
2048 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
1350517
( 1.2x )
311718
( 1.004 )
6600
( 1.0 )
4096 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP

Visual Wake Words (vww)

Cycles (Speedup) Total ROM (rel.) Total RAM (rel.) VLEN Layout Kernels Mode Arch Unroll Auto-Vectorization
37388763
( 1.2x )
572690
( 1.799 )
182872
( 2.09 )
0 NCHW TVM Fallback RV64GC 0 -
53420041
( 0.8x )
548724
( 1.724 )
182816
( 2.089 )
0 NHWC TVM Fallback RV64GC 0 -
67859943
( 0.6x )
710370
( 2.232 )
182824
( 2.089 )
128 NCHW TVM Fallback RV64GCV 0 Loop+SLP
67815248
( 0.6x )
710370
( 2.232 )
182824
( 2.089 )
256 NCHW TVM Fallback RV64GCV 0 Loop+SLP
67817113
( 0.6x )
710370
( 2.232 )
182824
( 2.089 )
512 NCHW TVM Fallback RV64GCV 0 Loop+SLP
67822708
( 0.6x )
710370
( 2.232 )
182824
( 2.089 )
1024 NCHW TVM Fallback RV64GCV 0 Loop+SLP
67833897
( 0.6x )
710368
( 2.232 )
182824
( 2.089 )
2048 NCHW TVM Fallback RV64GCV 0 Loop+SLP
67856280
( 0.6x )
710374
( 2.232 )
182824
( 2.089 )
4096 NCHW TVM Fallback RV64GCV 0 Loop+SLP
11903426
( 3.7x )
620378
( 1.949 )
182816
( 2.089 )
128 NHWC TVM Fallback RV64GCV 0 Loop+SLP
8466268
( 5.2x )
620380
( 1.949 )
182816
( 2.089 )
256 NHWC TVM Fallback RV64GCV 0 Loop+SLP
9189144
( 4.8x )
620372
( 1.949 )
182816
( 2.089 )
512 NHWC TVM Fallback RV64GCV 0 Loop+SLP
17558774
( 2.5x )
620372
( 1.949 )
182816
( 2.089 )
1024 NHWC TVM Fallback RV64GCV 0 Loop+SLP
25851857
( 1.7x )
620380
( 1.949 )
182816
( 2.089 )
2048 NHWC TVM Fallback RV64GCV 0 Loop+SLP
50337939
( 0.9x )
620378
( 1.949 )
182816
( 2.089 )
4096 NHWC TVM Fallback RV64GCV 0 Loop+SLP
43962962
( Base )
318326
( Base )
87504
( Base )
0 NHWC muRISCV-NN Scalar RV64GC 0 -
44776163
( 1.0x )
318352
( 1.0 )
87504
( 1.0 )
0 NHWC muRISCV-NN Vector (Portable) RV64GC 0 -
16957252
( 2.6x )
327884
( 1.03 )
87504
( 1.0 )
128 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
16536868
( 2.7x )
327886
( 1.03 )
87504
( 1.0 )
256 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
17695660
( 2.5x )
327886
( 1.03 )
87504
( 1.0 )
512 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
20847469
( 2.1x )
327886
( 1.03 )
87504
( 1.0 )
1024 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
24372650
( 1.8x )
327886
( 1.03 )
87504
( 1.0 )
2048 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
38337437
( 1.1x )
327884
( 1.03 )
87504
( 1.0 )
4096 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
13369553
( 3.3x )
321874
( 1.011 )
87504
( 1.0 )
128 NHWC muRISCV-NN Vector RV64GCV 0 -
10048568
( 4.4x )
321872
( 1.011 )
87504
( 1.0 )
256 NHWC muRISCV-NN Vector RV64GCV 0 -
8757149
( 5.0x )
321876
( 1.011 )
87504
( 1.0 )
512 NHWC muRISCV-NN Vector RV64GCV 0 -
8255028
( 5.3x )
321874
( 1.011 )
87504
( 1.0 )
1024 NHWC muRISCV-NN Vector RV64GCV 0 -
8207262
( 5.4x )
321876
( 1.011 )
87504
( 1.0 )
2048 NHWC muRISCV-NN Vector RV64GCV 0 -
8209127
( 5.4x )
321876
( 1.011 )
87504
( 1.0 )
4096 NHWC muRISCV-NN Vector RV64GCV 0 -
20504097
( 2.1x )
328556
( 1.032 )
87504
( 1.0 )
128 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
20226561
( 2.2x )
328556
( 1.032 )
87504
( 1.0 )
256 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
21744490
( 2.0x )
328556
( 1.032 )
87504
( 1.0 )
512 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
24896299
( 1.8x )
328556
( 1.032 )
87504
( 1.0 )
1024 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
28421479
( 1.5x )
328552
( 1.032 )
87504
( 1.0 )
2048 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
42388131
( 1.0x )
328552
( 1.032 )
87504
( 1.0 )
4096 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP

Original data

Click here to download the raw files for this benchmark.

2024-11-26
2024-11-21
2024-11-19
2024-11-18
2024-07-12
2024-06-29
2024-03-02
2024-02-26
2024-02-23
2024-02-22
2024-02-20
2024-02-11
2023-12-22
Clone this wiki locally