Skip to content

Benchmarks 2024 11 26 TVM LLVM Os spike_rv32

GitHub Action edited this page Nov 26, 2024 · 1 revision

Setup

Simulator

  • Spike (riscv-isa-sim ) (ISS, CPI=1)
    • Spike : eb0a3e2b0a7c57522928be39de95cd9f8c6dc636
    • Spike PK : fix-gcc14-rvv

Toolchains

Models

Frameworks

  • MLonMCU : develop

  • TVM : Nightly Pre-Build

Miscellaneous

  • Used -Os flag for compilation.
  • Benchmarks generated using MLonMCU deployment tool with minimal efforts.
  • Memory metrics are reported in Bytes

Results (Framework: tvm, Backend: tvmaot, Toolchain: llvm, Flags: -Os, Target: spike_rv32 )

Audio Wake Words (aww)

Cycles (Speedup) Total ROM (rel.) Total RAM (rel.) VLEN Layout Kernels Mode Arch Unroll Auto-Vectorization
29352385.0
( 0.5x )
112074
( 1.208 )
59444
( 3.094 )
0 NCHW TVM Fallback RV32GC 0 -
24905657.0
( 0.6x )
105418
( 1.136 )
59444
( 3.094 )
0 NHWC TVM Fallback RV32GC 0 -
6062412.0
( 2.5x )
108746
( 1.172 )
59444
( 3.094 )
128 NCHW TVM Fallback RV32GCV 0 Loop+SLP
5953622.0
( 2.5x )
108430
( 1.168 )
59444
( 3.094 )
256 NCHW TVM Fallback RV32GCV 0 Loop+SLP
5950021.0
( 2.6x )
108382
( 1.168 )
59444
( 3.094 )
512 NCHW TVM Fallback RV32GCV 0 Loop+SLP
5998841.0
( 2.5x )
108514
( 1.169 )
59444
( 3.094 )
1024 NCHW TVM Fallback RV32GCV 0 Loop+SLP
5998041.0
( 2.5x )
108472
( 1.169 )
59444
( 3.094 )
2048 NCHW TVM Fallback RV32GCV 0 Loop+SLP
6062500.0
( 2.5x )
108806
( 1.172 )
59444
( 3.094 )
4096 NCHW TVM Fallback RV32GCV 0 Loop+SLP
10045091.0
( 1.5x )
105956
( 1.142 )
59444
( 3.094 )
128 NHWC TVM Fallback RV32GCV 0 Loop+SLP
8397190.0
( 1.8x )
106016
( 1.142 )
59444
( 3.094 )
256 NHWC TVM Fallback RV32GCV 0 Loop+SLP
7578215.0
( 2.0x )
105982
( 1.142 )
59444
( 3.094 )
512 NHWC TVM Fallback RV32GCV 0 Loop+SLP
7608739.0
( 2.0x )
114746
( 1.236 )
59444
( 3.094 )
1024 NHWC TVM Fallback RV32GCV 0 Loop+SLP
7618909.0
( 2.0x )
114736
( 1.236 )
59444
( 3.094 )
2048 NHWC TVM Fallback RV32GCV 0 Loop+SLP
7642767.0
( 2.0x )
114770
( 1.237 )
59444
( 3.094 )
4096 NHWC TVM Fallback RV32GCV 0 Loop+SLP
15175868.0
( Base )
92800
( Base )
19212
( Base )
0 NHWC muRISCV-NN Scalar RV32GC 0 -
14592937.0
( 1.0x )
91906
( 0.99 )
23676
( 1.232 )
0 NHWC muRISCV-NN Vector (Portable) RV32GC 0 -
7274038.0
( 2.1x )
95848
( 1.033 )
19284
( 1.004 )
128 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
6431759.0
( 2.4x )
95916
( 1.034 )
19212
( 1.0 )
256 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
6010703.0
( 2.5x )
95916
( 1.034 )
19212
( 1.0 )
512 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
5800175.0
( 2.6x )
95916
( 1.034 )
19212
( 1.0 )
1024 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
5803566.0
( 2.6x )
95916
( 1.034 )
19212
( 1.0 )
2048 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
5806957.0
( 2.6x )
95916
( 1.034 )
19212
( 1.0 )
4096 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
5126075.0
( 3.0x )
93856
( 1.011 )
23676
( 1.232 )
128 NHWC muRISCV-NN Vector RV32GCV 0 -
3997232.0
( 3.8x )
93830
( 1.011 )
23676
( 1.232 )
256 NHWC muRISCV-NN Vector RV32GCV 0 -
3438954.0
( 4.4x )
93830
( 1.011 )
23676
( 1.232 )
512 NHWC muRISCV-NN Vector RV32GCV 0 -
3397966.0
( 4.5x )
93830
( 1.011 )
23676
( 1.232 )
1024 NHWC muRISCV-NN Vector RV32GCV 0 -
3401357.0
( 4.5x )
93830
( 1.011 )
23676
( 1.232 )
2048 NHWC muRISCV-NN Vector RV32GCV 0 -
3404748.0
( 4.5x )
93830
( 1.011 )
23676
( 1.232 )
4096 NHWC muRISCV-NN Vector RV32GCV 0 -
8112710.0
( 1.9x )
94632
( 1.02 )
23748
( 1.236 )
128 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP
6968615.0
( 2.2x )
94660
( 1.02 )
23676
( 1.232 )
256 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP
6547559.0
( 2.3x )
94660
( 1.02 )
23676
( 1.232 )
512 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP
6337031.0
( 2.4x )
94660
( 1.02 )
23676
( 1.232 )
1024 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP
6340422.0
( 2.4x )
94660
( 1.02 )
23676
( 1.232 )
2048 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP
6343813.0
( 2.4x )
94660
( 1.02 )
23676
( 1.232 )
4096 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP

Image Classification (resnet)

Cycles (Speedup) Total ROM (rel.) Total RAM (rel.) VLEN Layout Kernels Mode Arch Unroll Auto-Vectorization
124501936.0
( 0.5x )
221284
( 1.578 )
108356
( 1.952 )
0 NCHW TVM Fallback RV32GC 0 -
101622289.0
( 0.6x )
211726
( 1.509 )
108356
( 1.952 )
0 NHWC TVM Fallback RV32GC 0 -
25170952.0
( 2.2x )
217408
( 1.55 )
108356
( 1.952 )
128 NCHW TVM Fallback RV32GCV 0 Loop+SLP
25155888.0
( 2.2x )
217348
( 1.55 )
108356
( 1.952 )
256 NCHW TVM Fallback RV32GCV 0 Loop+SLP
25019744.0
( 2.3x )
216940
( 1.547 )
108356
( 1.952 )
512 NCHW TVM Fallback RV32GCV 0 Loop+SLP
25022616.0
( 2.3x )
216942
( 1.547 )
108356
( 1.952 )
1024 NCHW TVM Fallback RV32GCV 0 Loop+SLP
25034376.0
( 2.3x )
217018
( 1.547 )
108356
( 1.952 )
2048 NCHW TVM Fallback RV32GCV 0 Loop+SLP
25171042.0
( 2.2x )
217480
( 1.551 )
108356
( 1.952 )
4096 NCHW TVM Fallback RV32GCV 0 Loop+SLP
37232958.0
( 1.5x )
212500
( 1.515 )
108356
( 1.952 )
128 NHWC TVM Fallback RV32GCV 0 Loop+SLP
29613469.0
( 1.9x )
212748
( 1.517 )
108356
( 1.952 )
256 NHWC TVM Fallback RV32GCV 0 Loop+SLP
28405908.0
( 2.0x )
212998
( 1.519 )
108356
( 1.952 )
512 NHWC TVM Fallback RV32GCV 0 Loop+SLP
28177354.0
( 2.0x )
214364
( 1.528 )
108356
( 1.952 )
1024 NHWC TVM Fallback RV32GCV 0 Loop+SLP
28187524.0
( 2.0x )
214352
( 1.528 )
108356
( 1.952 )
2048 NHWC TVM Fallback RV32GCV 0 Loop+SLP
28211384.0
( 2.0x )
214394
( 1.529 )
108356
( 1.952 )
4096 NHWC TVM Fallback RV32GCV 0 Loop+SLP
56369537.0
( Base )
140264
( Base )
55516
( Base )
0 NHWC muRISCV-NN Scalar RV32GC 0 -
72416104.0
( 0.8x )
139562
( 0.995 )
55516
( 1.0 )
0 NHWC muRISCV-NN Vector (Portable) RV32GC 0 -
26154394.0
( 2.2x )
143788
( 1.025 )
55588
( 1.001 )
128 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
18289639.0
( 3.1x )
143986
( 1.027 )
55516
( 1.0 )
256 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
14603103.0
( 3.9x )
143986
( 1.027 )
55516
( 1.0 )
512 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
12923675.0
( 4.4x )
143986
( 1.027 )
55516
( 1.0 )
1024 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
12107866.0
( 4.7x )
143986
( 1.027 )
55516
( 1.0 )
2048 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
11496857.0
( 4.9x )
143986
( 1.027 )
55516
( 1.0 )
4096 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
15258029.0
( 3.7x )
142010
( 1.012 )
55516
( 1.0 )
128 NHWC muRISCV-NN Vector RV32GCV 0 -
9674870.0
( 5.8x )
142138
( 1.013 )
55516
( 1.0 )
256 NHWC muRISCV-NN Vector RV32GCV 0 -
7129032.0
( 7.9x )
142138
( 1.013 )
55516
( 1.0 )
512 NHWC muRISCV-NN Vector RV32GCV 0 -
5886904.0
( 9.6x )
142138
( 1.013 )
55516
( 1.0 )
1024 NHWC muRISCV-NN Vector RV32GCV 0 -
4962031.0
( 11.4x )
142138
( 1.013 )
55516
( 1.0 )
2048 NHWC muRISCV-NN Vector RV32GCV 0 -
4716330.0
( 12.0x )
142138
( 1.013 )
55516
( 1.0 )
4096 NHWC muRISCV-NN Vector RV32GCV 0 -
20453340.0
( 2.8x )
142456
( 1.016 )
55588
( 1.001 )
128 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP
14358057.0
( 3.9x )
142654
( 1.017 )
55516
( 1.0 )
256 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP
11500961.0
( 4.9x )
142654
( 1.017 )
55516
( 1.0 )
512 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP
10199389.0
( 5.5x )
142654
( 1.017 )
55516
( 1.0 )
1024 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP
9567900.0
( 5.9x )
142654
( 1.017 )
55516
( 1.0 )
2048 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP
9095131.0
( 6.2x )
142654
( 1.017 )
55516
( 1.0 )
4096 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP

Anomaly Detection (toycar)

Cycles (Speedup) Total ROM (rel.) Total RAM (rel.) VLEN Layout Kernels Mode Arch Unroll Auto-Vectorization
3199919.0
( 0.5x )
583712
( 1.837 )
5548
( 1.168 )
0 NCHW TVM Fallback RV32GC 0 -
3199919.0
( 0.5x )
583712
( 1.837 )
5548
( 1.168 )
0 NHWC TVM Fallback RV32GC 0 -
1043749.0
( 1.6x )
582890
( 1.834 )
5548
( 1.168 )
128 NCHW TVM Fallback RV32GCV 0 Loop+SLP
1043512.0
( 1.6x )
582730
( 1.834 )
5548
( 1.168 )
256 NCHW TVM Fallback RV32GCV 0 Loop+SLP
1011667.0
( 1.7x )
583598
( 1.837 )
5548
( 1.168 )
512 NCHW TVM Fallback RV32GCV 0 Loop+SLP
1011667.0
( 1.7x )
583598
( 1.837 )
5548
( 1.168 )
1024 NCHW TVM Fallback RV32GCV 0 Loop+SLP
1011667.0
( 1.7x )
583598
( 1.837 )
5548
( 1.168 )
2048 NCHW TVM Fallback RV32GCV 0 Loop+SLP
1011667.0
( 1.7x )
583598
( 1.837 )
5548
( 1.168 )
4096 NCHW TVM Fallback RV32GCV 0 Loop+SLP
1043749.0
( 1.6x )
582890
( 1.834 )
5548
( 1.168 )
128 NHWC TVM Fallback RV32GCV 0 Loop+SLP
1043512.0
( 1.6x )
582730
( 1.834 )
5548
( 1.168 )
256 NHWC TVM Fallback RV32GCV 0 Loop+SLP
1011667.0
( 1.7x )
583598
( 1.837 )
5548
( 1.168 )
512 NHWC TVM Fallback RV32GCV 0 Loop+SLP
1011667.0
( 1.7x )
583598
( 1.837 )
5548
( 1.168 )
1024 NHWC TVM Fallback RV32GCV 0 Loop+SLP
1011667.0
( 1.7x )
583598
( 1.837 )
5548
( 1.168 )
2048 NHWC TVM Fallback RV32GCV 0 Loop+SLP
1011667.0
( 1.7x )
583598
( 1.837 )
5548
( 1.168 )
4096 NHWC TVM Fallback RV32GCV 0 Loop+SLP
1719800.0
( Base )
317776
( Base )
4748
( Base )
0 NHWC muRISCV-NN Scalar RV32GC 0 -
1719799.0
( 1.0x )
317774
( 1.0 )
4748
( 1.0 )
0 NHWC muRISCV-NN Vector (Portable) RV32GC 0 -
629246.0
( 2.7x )
318386
( 1.002 )
4748
( 1.0 )
128 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
515327.0
( 3.3x )
318392
( 1.002 )
4748
( 1.0 )
256 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
458368.0
( 3.8x )
318394
( 1.002 )
4748
( 1.0 )
512 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
429887.0
( 4.0x )
318392
( 1.002 )
4748
( 1.0 )
1024 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
415648.0
( 4.1x )
318394
( 1.002 )
4748
( 1.0 )
2048 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
413863.0
( 4.2x )
318392
( 1.002 )
4748
( 1.0 )
4096 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
618596.0
( 2.8x )
319154
( 1.004 )
4748
( 1.0 )
128 NHWC muRISCV-NN Vector RV32GCV 0 -
501916.0
( 3.4x )
319156
( 1.004 )
4748
( 1.0 )
256 NHWC muRISCV-NN Vector RV32GCV 0 -
443576.0
( 3.9x )
319156
( 1.004 )
4748
( 1.0 )
512 NHWC muRISCV-NN Vector RV32GCV 0 -
414731.0
( 4.1x )
319162
( 1.004 )
4748
( 1.0 )
1024 NHWC muRISCV-NN Vector RV32GCV 0 -
411100.0
( 4.2x )
319166
( 1.004 )
4748
( 1.0 )
2048 NHWC muRISCV-NN Vector RV32GCV 0 -
409242.0
( 4.2x )
319156
( 1.004 )
4748
( 1.0 )
4096 NHWC muRISCV-NN Vector RV32GCV 0 -
629245.0
( 2.7x )
318384
( 1.002 )
4748
( 1.0 )
128 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP
515327.0
( 3.3x )
318392
( 1.002 )
4748
( 1.0 )
256 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP
458367.0
( 3.8x )
318392
( 1.002 )
4748
( 1.0 )
512 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP
429888.0
( 4.0x )
318394
( 1.002 )
4748
( 1.0 )
1024 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP
415648.0
( 4.1x )
318394
( 1.002 )
4748
( 1.0 )
2048 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP
413862.0
( 4.2x )
318388
( 1.002 )
4748
( 1.0 )
4096 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP

Visual Wake Words (vww)

Cycles (Speedup) Total ROM (rel.) Total RAM (rel.) VLEN Layout Kernels Mode Arch Unroll Auto-Vectorization
84665066.0
( 0.5x )
547952
( 1.683 )
180952
( 2.113 )
0 NCHW TVM Fallback RV32GC 0 -
72595771.0
( 0.6x )
525892
( 1.615 )
180952
( 2.113 )
0 NHWC TVM Fallback RV32GC 0 -
18324281.0
( 2.5x )
537750
( 1.652 )
180952
( 2.113 )
128 NCHW TVM Fallback RV32GCV 0 Loop+SLP
18256876.0
( 2.5x )
537196
( 1.65 )
180952
( 2.113 )
256 NCHW TVM Fallback RV32GCV 0 Loop+SLP
18229025.0
( 2.5x )
537036
( 1.649 )
180952
( 2.113 )
512 NCHW TVM Fallback RV32GCV 0 Loop+SLP
17943620.0
( 2.5x )
536246
( 1.647 )
180952
( 2.113 )
1024 NCHW TVM Fallback RV32GCV 0 Loop+SLP
17983142.0
( 2.5x )
536428
( 1.648 )
180952
( 2.113 )
2048 NCHW TVM Fallback RV32GCV 0 Loop+SLP
18345322.0
( 2.5x )
537898
( 1.652 )
180952
( 2.113 )
4096 NCHW TVM Fallback RV32GCV 0 Loop+SLP
30922641.0
( 1.5x )
527206
( 1.619 )
180952
( 2.113 )
128 NHWC TVM Fallback RV32GCV 0 Loop+SLP
26038427.0
( 1.7x )
527038
( 1.619 )
180952
( 2.113 )
256 NHWC TVM Fallback RV32GCV 0 Loop+SLP
24005335.0
( 1.9x )
527548
( 1.62 )
180952
( 2.113 )
512 NHWC TVM Fallback RV32GCV 0 Loop+SLP
23462157.0
( 1.9x )
539900
( 1.658 )
180952
( 2.113 )
1024 NHWC TVM Fallback RV32GCV 0 Loop+SLP
23450001.0
( 1.9x )
558972
( 1.717 )
180952
( 2.113 )
2048 NHWC TVM Fallback RV32GCV 0 Loop+SLP
23566926.0
( 1.9x )
566904
( 1.741 )
180952
( 2.113 )
4096 NHWC TVM Fallback RV32GCV 0 Loop+SLP
45298332.0
( Base )
325598
( Base )
85648
( Base )
0 NHWC muRISCV-NN Scalar RV32GC 0 -
45106025.0
( 1.0x )
324836
( 0.998 )
85648
( 1.0 )
0 NHWC muRISCV-NN Vector (Portable) RV32GC 0 -
19674527.0
( 2.3x )
329680
( 1.013 )
85864
( 1.003 )
128 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
16808164.0
( 2.7x )
329924
( 1.013 )
85648
( 1.0 )
256 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
15434981.0
( 2.9x )
329928
( 1.013 )
85648
( 1.0 )
512 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
14930740.0
( 3.0x )
329924
( 1.013 )
85648
( 1.0 )
1024 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
14727610.0
( 3.1x )
329918
( 1.013 )
85648
( 1.0 )
2048 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
14713567.0
( 3.1x )
329928
( 1.013 )
85648
( 1.0 )
4096 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
13476527.0
( 3.4x )
327760
( 1.007 )
85648
( 1.0 )
128 NHWC muRISCV-NN Vector RV32GCV 0 -
10155571.0
( 4.5x )
327924
( 1.007 )
85648
( 1.0 )
256 NHWC muRISCV-NN Vector RV32GCV 0 -
8864143.0
( 5.1x )
327926
( 1.007 )
85648
( 1.0 )
512 NHWC muRISCV-NN Vector RV32GCV 0 -
8360153.0
( 5.4x )
327924
( 1.007 )
85648
( 1.0 )
1024 NHWC muRISCV-NN Vector RV32GCV 0 -
8315775.0
( 5.4x )
327924
( 1.007 )
85648
( 1.0 )
2048 NHWC muRISCV-NN Vector RV32GCV 0 -
8319166.0
( 5.4x )
327926
( 1.007 )
85648
( 1.0 )
4096 NHWC muRISCV-NN Vector RV32GCV 0 -
22583692.0
( 2.0x )
328586
( 1.009 )
85864
( 1.003 )
128 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP
18959143.0
( 2.4x )
328792
( 1.01 )
85648
( 1.0 )
256 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP
17627431.0
( 2.6x )
328792
( 1.01 )
85648
( 1.0 )
512 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP
17123191.0
( 2.6x )
328790
( 1.01 )
85648
( 1.0 )
1024 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP
16920062.0
( 2.7x )
328792
( 1.01 )
85648
( 1.0 )
2048 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP
16906017.0
( 2.7x )
328792
( 1.01 )
85648
( 1.0 )
4096 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP

Original data

Click here to download the raw files for this benchmark.

2024-11-26
2024-11-21
2024-11-19
2024-11-18
2024-07-12
2024-06-29
2024-03-02
2024-02-26
2024-02-23
2024-02-22
2024-02-20
2024-02-11
2023-12-22
Clone this wiki locally