Skip to content

Benchmarks 2024 11 26 TVM LLVM Os spike_rv64

GitHub Action edited this page Nov 26, 2024 · 1 revision

Setup

Simulator

  • Spike (riscv-isa-sim ) (ISS, CPI=1)
    • Spike : eb0a3e2b0a7c57522928be39de95cd9f8c6dc636
    • Spike PK : fix-gcc14-rvv

Toolchains

Models

Frameworks

  • MLonMCU : develop

  • TVM : Nightly Pre-Build

Miscellaneous

  • Used -Os flag for compilation.
  • Benchmarks generated using MLonMCU deployment tool with minimal efforts.
  • Memory metrics are reported in Bytes

Results (Framework: tvm, Backend: tvmaot, Toolchain: llvm, Flags: -Os, Target: spike_rv64 )

Audio Wake Words (aww)

Cycles (Speedup) Total ROM (rel.) Total RAM (rel.) VLEN Layout Kernels Mode Arch Unroll Auto-Vectorization
28356271.0
( 0.6x )
104722
( 1.208 )
61368
( 2.915 )
0 NCHW TVM Fallback RV64GC 0 -
24004451.0
( 0.7x )
99776
( 1.15 )
61288
( 2.911 )
0 NHWC TVM Fallback RV64GC 0 -
5890709.0
( 2.7x )
103714
( 1.196 )
61296
( 2.911 )
128 NCHW TVM Fallback RV64GCV 0 Loop+SLP
5796380.0
( 2.7x )
103358
( 1.192 )
61296
( 2.911 )
256 NCHW TVM Fallback RV64GCV 0 Loop+SLP
5792380.0
( 2.7x )
103318
( 1.191 )
61296
( 2.911 )
512 NCHW TVM Fallback RV64GCV 0 Loop+SLP
5848725.0
( 2.7x )
103574
( 1.194 )
61296
( 2.911 )
1024 NCHW TVM Fallback RV64GCV 0 Loop+SLP
5847925.0
( 2.7x )
103534
( 1.194 )
61296
( 2.911 )
2048 NCHW TVM Fallback RV64GCV 0 Loop+SLP
5892724.0
( 2.7x )
103818
( 1.197 )
61296
( 2.911 )
4096 NCHW TVM Fallback RV64GCV 0 Loop+SLP
9587040.0
( 1.6x )
100828
( 1.163 )
61288
( 2.911 )
128 NHWC TVM Fallback RV64GCV 0 Loop+SLP
7929144.0
( 2.0x )
100928
( 1.164 )
61288
( 2.911 )
256 NHWC TVM Fallback RV64GCV 0 Loop+SLP
7100055.0
( 2.2x )
100898
( 1.163 )
61288
( 2.911 )
512 NHWC TVM Fallback RV64GCV 0 Loop+SLP
7843373.0
( 2.0x )
119332
( 1.376 )
61288
( 2.911 )
1024 NHWC TVM Fallback RV64GCV 0 Loop+SLP
7854560.0
( 2.0x )
119322
( 1.376 )
61288
( 2.911 )
2048 NHWC TVM Fallback RV64GCV 0 Loop+SLP
7878894.0
( 2.0x )
119360
( 1.376 )
61288
( 2.911 )
4096 NHWC TVM Fallback RV64GCV 0 Loop+SLP
15705418.0
( Base )
86726
( Base )
21056
( Base )
0 NHWC muRISCV-NN Scalar RV64GC 0 -
15650160.0
( 1.0x )
85928
( 0.991 )
25520
( 1.212 )
0 NHWC muRISCV-NN Vector (Portable) RV64GC 0 -
7146391.0
( 2.2x )
89814
( 1.036 )
21056
( 1.0 )
128 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
6265600.0
( 2.5x )
89348
( 1.03 )
21056
( 1.0 )
256 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
5827896.0
( 2.7x )
89348
( 1.03 )
21056
( 1.0 )
512 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
5610909.0
( 2.8x )
89348
( 1.03 )
21056
( 1.0 )
1024 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
5610909.0
( 2.8x )
89348
( 1.03 )
21056
( 1.0 )
2048 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
5614639.0
( 2.8x )
89348
( 1.03 )
21056
( 1.0 )
4096 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
6017576.0
( 2.6x )
88016
( 1.015 )
25520
( 1.212 )
128 NHWC muRISCV-NN Vector RV64GCV 0 -
4883519.0
( 3.2x )
87590
( 1.01 )
25520
( 1.212 )
256 NHWC muRISCV-NN Vector RV64GCV 0 -
4325223.0
( 3.6x )
87590
( 1.01 )
25520
( 1.212 )
512 NHWC muRISCV-NN Vector RV64GCV 0 -
4286092.0
( 3.7x )
87590
( 1.01 )
25520
( 1.212 )
1024 NHWC muRISCV-NN Vector RV64GCV 0 -
4286092.0
( 3.7x )
87590
( 1.01 )
25520
( 1.212 )
2048 NHWC muRISCV-NN Vector RV64GCV 0 -
4287957.0
( 3.7x )
87590
( 1.01 )
25520
( 1.212 )
4096 NHWC muRISCV-NN Vector RV64GCV 0 -
8326965.0
( 1.9x )
88586
( 1.021 )
25520
( 1.212 )
128 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
7451311.0
( 2.1x )
88160
( 1.017 )
25520
( 1.212 )
256 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
7013607.0
( 2.2x )
88160
( 1.017 )
25520
( 1.212 )
512 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
6796620.0
( 2.3x )
88160
( 1.017 )
25520
( 1.212 )
1024 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
6796620.0
( 2.3x )
88160
( 1.017 )
25520
( 1.212 )
2048 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
6800350.0
( 2.3x )
88160
( 1.017 )
25520
( 1.212 )
4096 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP

Image Classification (resnet)

Cycles (Speedup) Total ROM (rel.) Total RAM (rel.) VLEN Layout Kernels Mode Arch Unroll Auto-Vectorization
123435843.0
( 0.5x )
212766
( 1.585 )
110216
( 1.921 )
0 NCHW TVM Fallback RV64GC 0 -
100169412.0
( 0.6x )
205998
( 1.535 )
110216
( 1.921 )
0 NHWC TVM Fallback RV64GC 0 -
24209094.0
( 2.3x )
212044
( 1.58 )
110216
( 1.921 )
128 NCHW TVM Fallback RV64GCV 0 Loop+SLP
24198053.0
( 2.3x )
212020
( 1.58 )
110216
( 1.921 )
256 NCHW TVM Fallback RV64GCV 0 Loop+SLP
24042297.0
( 2.3x )
211574
( 1.576 )
110216
( 1.921 )
512 NCHW TVM Fallback RV64GCV 0 Loop+SLP
24036663.0
( 2.3x )
211584
( 1.576 )
110216
( 1.921 )
1024 NCHW TVM Fallback RV64GCV 0 Loop+SLP
24047583.0
( 2.3x )
211638
( 1.577 )
110216
( 1.921 )
2048 NCHW TVM Fallback RV64GCV 0 Loop+SLP
24216233.0
( 2.3x )
212154
( 1.581 )
110216
( 1.921 )
4096 NCHW TVM Fallback RV64GCV 0 Loop+SLP
35933045.0
( 1.6x )
206850
( 1.541 )
110216
( 1.921 )
128 NHWC TVM Fallback RV64GCV 0 Loop+SLP
34755076.0
( 1.6x )
207750
( 1.548 )
110216
( 1.921 )
256 NHWC TVM Fallback RV64GCV 0 Loop+SLP
34013415.0
( 1.6x )
208056
( 1.55 )
110216
( 1.921 )
512 NHWC TVM Fallback RV64GCV 0 Loop+SLP
34291886.0
( 1.6x )
211168
( 1.573 )
110216
( 1.921 )
1024 NHWC TVM Fallback RV64GCV 0 Loop+SLP
34303075.0
( 1.6x )
211160
( 1.573 )
110216
( 1.921 )
2048 NHWC TVM Fallback RV64GCV 0 Loop+SLP
34332536.0
( 1.6x )
211194
( 1.573 )
110216
( 1.921 )
4096 NHWC TVM Fallback RV64GCV 0 Loop+SLP
55736927.0
( Base )
134226
( Base )
57376
( Base )
0 NHWC muRISCV-NN Scalar RV64GC 0 -
71562670.0
( 0.8x )
133618
( 0.995 )
57376
( 1.0 )
0 NHWC muRISCV-NN Vector (Portable) RV64GC 0 -
25741863.0
( 2.2x )
138106
( 1.029 )
57376
( 1.0 )
128 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
17778716.0
( 3.1x )
137588
( 1.025 )
57376
( 1.0 )
256 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
14046096.0
( 4.0x )
137588
( 1.025 )
57376
( 1.0 )
512 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
12347539.0
( 4.5x )
137588
( 1.025 )
57376
( 1.0 )
1024 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
11519964.0
( 4.8x )
137588
( 1.025 )
57376
( 1.0 )
2048 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
10899749.0
( 5.1x )
137588
( 1.025 )
57376
( 1.0 )
4096 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
15261403.0
( 3.7x )
136156
( 1.014 )
57376
( 1.0 )
128 NHWC muRISCV-NN Vector RV64GCV 0 -
9678205.0
( 5.8x )
135638
( 1.011 )
57376
( 1.0 )
256 NHWC muRISCV-NN Vector RV64GCV 0 -
7132365.0
( 7.8x )
135638
( 1.011 )
57376
( 1.0 )
512 NHWC muRISCV-NN Vector RV64GCV 0 -
5890237.0
( 9.5x )
135638
( 1.011 )
57376
( 1.0 )
1024 NHWC muRISCV-NN Vector RV64GCV 0 -
4963838.0
( 11.2x )
135638
( 1.011 )
57376
( 1.0 )
2048 NHWC muRISCV-NN Vector RV64GCV 0 -
4716611.0
( 11.8x )
135638
( 1.011 )
57376
( 1.0 )
4096 NHWC muRISCV-NN Vector RV64GCV 0 -
19854766.0
( 2.8x )
136814
( 1.019 )
57376
( 1.0 )
128 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
13562787.0
( 4.1x )
136296
( 1.015 )
57376
( 1.0 )
256 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
10613527.0
( 5.3x )
136296
( 1.015 )
57376
( 1.0 )
512 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
9271834.0
( 6.0x )
136296
( 1.015 )
57376
( 1.0 )
1024 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
8616474.0
( 6.5x )
136296
( 1.015 )
57376
( 1.0 )
2048 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
8128684.0
( 6.9x )
136296
( 1.015 )
57376
( 1.0 )
4096 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP

Anomaly Detection (toycar)

Cycles (Speedup) Total ROM (rel.) Total RAM (rel.) VLEN Layout Kernels Mode Arch Unroll Auto-Vectorization
3202427.0
( 0.5x )
578702
( 1.851 )
7408
( 1.121 )
0 NCHW TVM Fallback RV64GC 0 -
3202427.0
( 0.5x )
578702
( 1.851 )
7408
( 1.121 )
0 NHWC TVM Fallback RV64GC 0 -
1041694.0
( 1.6x )
577898
( 1.848 )
7408
( 1.121 )
128 NCHW TVM Fallback RV64GCV 0 Loop+SLP
1041457.0
( 1.6x )
577774
( 1.848 )
7408
( 1.121 )
256 NCHW TVM Fallback RV64GCV 0 Loop+SLP
1013976.0
( 1.6x )
578660
( 1.85 )
7408
( 1.121 )
512 NCHW TVM Fallback RV64GCV 0 Loop+SLP
1013976.0
( 1.6x )
578660
( 1.85 )
7408
( 1.121 )
1024 NCHW TVM Fallback RV64GCV 0 Loop+SLP
1013976.0
( 1.6x )
578660
( 1.85 )
7408
( 1.121 )
2048 NCHW TVM Fallback RV64GCV 0 Loop+SLP
1013976.0
( 1.6x )
578660
( 1.85 )
7408
( 1.121 )
4096 NCHW TVM Fallback RV64GCV 0 Loop+SLP
1041694.0
( 1.6x )
577898
( 1.848 )
7408
( 1.121 )
128 NHWC TVM Fallback RV64GCV 0 Loop+SLP
1041457.0
( 1.6x )
577774
( 1.848 )
7408
( 1.121 )
256 NHWC TVM Fallback RV64GCV 0 Loop+SLP
1013976.0
( 1.6x )
578660
( 1.85 )
7408
( 1.121 )
512 NHWC TVM Fallback RV64GCV 0 Loop+SLP
1013976.0
( 1.6x )
578660
( 1.85 )
7408
( 1.121 )
1024 NHWC TVM Fallback RV64GCV 0 Loop+SLP
1013976.0
( 1.6x )
578660
( 1.85 )
7408
( 1.121 )
2048 NHWC TVM Fallback RV64GCV 0 Loop+SLP
1013976.0
( 1.6x )
578660
( 1.85 )
7408
( 1.121 )
4096 NHWC TVM Fallback RV64GCV 0 Loop+SLP
1662786.0
( Base )
312718
( Base )
6608
( Base )
0 NHWC muRISCV-NN Scalar RV64GC 0 -
1662787.0
( 1.0x )
312720
( 1.0 )
6608
( 1.0 )
0 NHWC muRISCV-NN Vector (Portable) RV64GC 0 -
628443.0
( 2.6x )
313474
( 1.002 )
6608
( 1.0 )
128 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
510980.0
( 3.3x )
313058
( 1.001 )
6608
( 1.0 )
256 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
452260.0
( 3.7x )
313058
( 1.001 )
6608
( 1.0 )
512 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
422901.0
( 3.9x )
313060
( 1.001 )
6608
( 1.0 )
1024 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
408219.0
( 4.1x )
313056
( 1.001 )
6608
( 1.0 )
2048 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
406381.0
( 4.1x )
313066
( 1.001 )
6608
( 1.0 )
4096 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
625472.0
( 2.7x )
314186
( 1.005 )
6608
( 1.0 )
128 NHWC muRISCV-NN Vector RV64GCV 0 -
501954.0
( 3.3x )
313772
( 1.003 )
6608
( 1.0 )
256 NHWC muRISCV-NN Vector RV64GCV 0 -
442769.0
( 3.8x )
313772
( 1.003 )
6608
( 1.0 )
512 NHWC muRISCV-NN Vector RV64GCV 0 -
413514.0
( 4.0x )
313772
( 1.003 )
6608
( 1.0 )
1024 NHWC muRISCV-NN Vector RV64GCV 0 -
409831.0
( 4.1x )
313778
( 1.003 )
6608
( 1.0 )
2048 NHWC muRISCV-NN Vector RV64GCV 0 -
407949.0
( 4.1x )
313780
( 1.003 )
6608
( 1.0 )
4096 NHWC muRISCV-NN Vector RV64GCV 0 -
628443.0
( 2.6x )
313474
( 1.002 )
6608
( 1.0 )
128 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
510980.0
( 3.3x )
313058
( 1.001 )
6608
( 1.0 )
256 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
452259.0
( 3.7x )
313056
( 1.001 )
6608
( 1.0 )
512 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
422900.0
( 3.9x )
313058
( 1.001 )
6608
( 1.0 )
1024 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
408221.0
( 4.1x )
313060
( 1.001 )
6608
( 1.0 )
2048 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
406381.0
( 4.1x )
313060
( 1.001 )
6608
( 1.0 )
4096 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP

Visual Wake Words (vww)

Cycles (Speedup) Total ROM (rel.) Total RAM (rel.) VLEN Layout Kernels Mode Arch Unroll Auto-Vectorization
81866644.0
( 0.5x )
536648
( 1.679 )
182976
( 2.091 )
0 NCHW TVM Fallback RV64GC 0 -
69686529.0
( 0.6x )
519012
( 1.623 )
182816
( 2.089 )
0 NHWC TVM Fallback RV64GC 0 -
18255670.0
( 2.4x )
532050
( 1.664 )
182840
( 2.089 )
128 NCHW TVM Fallback RV64GCV 0 Loop+SLP
18199896.0
( 2.4x )
531504
( 1.662 )
182840
( 2.089 )
256 NCHW TVM Fallback RV64GCV 0 Loop+SLP
18179623.0
( 2.4x )
531288
( 1.662 )
182840
( 2.089 )
512 NCHW TVM Fallback RV64GCV 0 Loop+SLP
17986729.0
( 2.5x )
530490
( 1.659 )
182840
( 2.089 )
1024 NCHW TVM Fallback RV64GCV 0 Loop+SLP
18012143.0
( 2.4x )
530716
( 1.66 )
182840
( 2.089 )
2048 NCHW TVM Fallback RV64GCV 0 Loop+SLP
18268689.0
( 2.4x )
532208
( 1.665 )
182840
( 2.089 )
4096 NCHW TVM Fallback RV64GCV 0 Loop+SLP
28854346.0
( 1.5x )
521786
( 1.632 )
182816
( 2.089 )
128 NHWC TVM Fallback RV64GCV 0 Loop+SLP
24036439.0
( 1.8x )
521668
( 1.632 )
182816
( 2.089 )
256 NHWC TVM Fallback RV64GCV 0 Loop+SLP
22239138.0
( 2.0x )
522704
( 1.635 )
182816
( 2.089 )
512 NHWC TVM Fallback RV64GCV 0 Loop+SLP
22221099.0
( 2.0x )
550762
( 1.723 )
182816
( 2.089 )
1024 NHWC TVM Fallback RV64GCV 0 Loop+SLP
22424692.0
( 2.0x )
578930
( 1.811 )
182816
( 2.089 )
2048 NHWC TVM Fallback RV64GCV 0 Loop+SLP
22522523.0
( 2.0x )
589050
( 1.842 )
182816
( 2.089 )
4096 NHWC TVM Fallback RV64GCV 0 Loop+SLP
44083240.0
( Base )
319714
( Base )
87512
( Base )
0 NHWC muRISCV-NN Scalar RV64GC 0 -
45097701.0
( 1.0x )
319032
( 0.998 )
87512
( 1.0 )
0 NHWC muRISCV-NN Vector (Portable) RV64GC 0 -
18474951.0
( 2.4x )
323724
( 1.013 )
87512
( 1.0 )
128 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
15487671.0
( 2.8x )
322328
( 1.008 )
87512
( 1.0 )
256 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
14061513.0
( 3.1x )
322336
( 1.008 )
87512
( 1.0 )
512 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
13537305.0
( 3.3x )
322332
( 1.008 )
87512
( 1.0 )
1024 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
13323558.0
( 3.3x )
322332
( 1.008 )
87512
( 1.0 )
2048 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
13307090.0
( 3.3x )
322336
( 1.008 )
87512
( 1.0 )
4096 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
13473675.0
( 3.3x )
321952
( 1.007 )
87512
( 1.0 )
128 NHWC muRISCV-NN Vector RV64GCV 0 -
10152615.0
( 4.3x )
320696
( 1.003 )
87512
( 1.0 )
256 NHWC muRISCV-NN Vector RV64GCV 0 -
8861159.0
( 5.0x )
320696
( 1.003 )
87512
( 1.0 )
512 NHWC muRISCV-NN Vector RV64GCV 0 -
8359020.0
( 5.3x )
320696
( 1.003 )
87512
( 1.0 )
1024 NHWC muRISCV-NN Vector RV64GCV 0 -
8311245.0
( 5.3x )
320696
( 1.003 )
87512
( 1.0 )
2048 NHWC muRISCV-NN Vector RV64GCV 0 -
8313110.0
( 5.3x )
320696
( 1.003 )
87512
( 1.0 )
4096 NHWC muRISCV-NN Vector RV64GCV 0 -
22084820.0
( 2.0x )
322522
( 1.009 )
87512
( 1.0 )
128 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
19186160.0
( 2.3x )
321266
( 1.005 )
87512
( 1.0 )
256 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
17799168.0
( 2.5x )
321266
( 1.005 )
87512
( 1.0 )
512 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
17274961.0
( 2.6x )
321266
( 1.005 )
87512
( 1.0 )
1024 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
17059348.0
( 2.6x )
321262
( 1.005 )
87512
( 1.0 )
2048 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
17044745.0
( 2.6x )
321266
( 1.005 )
87512
( 1.0 )
4096 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP

Original data

Click here to download the raw files for this benchmark.

2024-11-26
2024-11-21
2024-11-19
2024-11-18
2024-07-12
2024-06-29
2024-03-02
2024-02-26
2024-02-23
2024-02-22
2024-02-20
2024-02-11
2023-12-22
Clone this wiki locally