Skip to content

Benchmarks 2024 11 26 TVM LLVM O3 spike_rv64

GitHub Action edited this page Nov 26, 2024 · 1 revision

Setup

Simulator

  • Spike (riscv-isa-sim ) (ISS, CPI=1)
    • Spike : eb0a3e2b0a7c57522928be39de95cd9f8c6dc636
    • Spike PK : fix-gcc14-rvv

Toolchains

Models

Frameworks

  • MLonMCU : develop

  • TVM : Nightly Pre-Build

Miscellaneous

  • Used -Os flag for compilation.
  • Benchmarks generated using MLonMCU deployment tool with minimal efforts.
  • Memory metrics are reported in Bytes

Results (Framework: tvm, Backend: tvmaot, Toolchain: llvm, Flags: -O3, Target: spike_rv64 )

Audio Wake Words (aww)

Cycles (Speedup) Total ROM (rel.) Total RAM (rel.) VLEN Layout Kernels Mode Arch Unroll Auto-Vectorization
13151114.0
( 1.2x )
125684
( 1.432 )
61296
( 2.911 )
0 NCHW TVM Fallback RV64GC 0 -
20156761.0
( 0.8x )
109016
( 1.242 )
61288
( 2.911 )
0 NHWC TVM Fallback RV64GC 0 -
25193781.0
( 0.6x )
179788
( 2.049 )
61288
( 2.911 )
128 NCHW TVM Fallback RV64GCV 0 Loop+SLP
25140137.0
( 0.6x )
179446
( 2.045 )
61288
( 2.911 )
256 NCHW TVM Fallback RV64GCV 0 Loop+SLP
25283215.0
( 0.6x )
180044
( 2.051 )
61288
( 2.911 )
512 NCHW TVM Fallback RV64GCV 0 Loop+SLP
26530169.0
( 0.6x )
183682
( 2.093 )
61288
( 2.911 )
1024 NCHW TVM Fallback RV64GCV 0 Loop+SLP
26536661.0
( 0.6x )
183290
( 2.088 )
61288
( 2.911 )
2048 NCHW TVM Fallback RV64GCV 0 Loop+SLP
26391069.0
( 0.6x )
182978
( 2.085 )
61288
( 2.911 )
4096 NCHW TVM Fallback RV64GCV 0 Loop+SLP
3633274.0
( 4.3x )
127816
( 1.456 )
61288
( 2.911 )
128 NHWC TVM Fallback RV64GCV 0 Loop+SLP
2205916.0
( 7.1x )
127868
( 1.457 )
61288
( 2.911 )
256 NHWC TVM Fallback RV64GCV 0 Loop+SLP
1470726.0
( 10.7x )
125856
( 1.434 )
61288
( 2.911 )
512 NHWC TVM Fallback RV64GCV 0 Loop+SLP
725620.0
( 21.6x )
115344
( 1.314 )
61288
( 2.911 )
1024 NHWC TVM Fallback RV64GCV 0 Loop+SLP
736462.0
( 21.3x )
115472
( 1.316 )
61288
( 2.911 )
2048 NHWC TVM Fallback RV64GCV 0 Loop+SLP
736375.0
( 21.3x )
115412
( 1.315 )
61288
( 2.911 )
4096 NHWC TVM Fallback RV64GCV 0 Loop+SLP
15671855.0
( Base )
87764
( Base )
21056
( Base )
0 NHWC muRISCV-NN Scalar RV64GC 0 -
15591560.0
( 1.0x )
87656
( 0.999 )
25520
( 1.212 )
0 NHWC muRISCV-NN Vector (Portable) RV64GC 0 -
7073397.0
( 2.2x )
96374
( 1.098 )
21056
( 1.0 )
128 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
6491057.0
( 2.4x )
95912
( 1.093 )
21056
( 1.0 )
256 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
6200930.0
( 2.5x )
95916
( 1.093 )
21056
( 1.0 )
512 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
6056799.0
( 2.6x )
95916
( 1.093 )
21056
( 1.0 )
1024 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
14529412.0
( 1.1x )
95916
( 1.093 )
21056
( 1.0 )
2048 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
14531277.0
( 1.1x )
95916
( 1.093 )
21056
( 1.0 )
4096 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
6005421.0
( 2.6x )
90350
( 1.029 )
25520
( 1.212 )
128 NHWC muRISCV-NN Vector RV64GCV 0 -
4871412.0
( 3.2x )
89924
( 1.025 )
25520
( 1.212 )
256 NHWC muRISCV-NN Vector RV64GCV 0 -
4313140.0
( 3.6x )
89924
( 1.025 )
25520
( 1.212 )
512 NHWC muRISCV-NN Vector RV64GCV 0 -
4274021.0
( 3.7x )
89924
( 1.025 )
25520
( 1.212 )
1024 NHWC muRISCV-NN Vector RV64GCV 0 -
4274021.0
( 3.7x )
89924
( 1.025 )
25520
( 1.212 )
2048 NHWC muRISCV-NN Vector RV64GCV 0 -
4275886.0
( 3.7x )
89924
( 1.025 )
25520
( 1.212 )
4096 NHWC muRISCV-NN Vector RV64GCV 0 -
8199037.0
( 1.9x )
97030
( 1.106 )
25520
( 1.212 )
128 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
7611578.0
( 2.1x )
96608
( 1.101 )
25520
( 1.212 )
256 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
7321451.0
( 2.1x )
96620
( 1.101 )
25520
( 1.212 )
512 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
7177320.0
( 2.2x )
96620
( 1.101 )
25520
( 1.212 )
1024 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
15648941.0
( 1.0x )
96572
( 1.1 )
25520
( 1.212 )
2048 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
15650806.0
( 1.0x )
96572
( 1.1 )
25520
( 1.212 )
4096 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP

Image Classification (resnet)

Cycles (Speedup) Total ROM (rel.) Total RAM (rel.) VLEN Layout Kernels Mode Arch Unroll Auto-Vectorization
58410743.0
( 1.0x )
226572
( 1.677 )
110216
( 1.921 )
0 NCHW TVM Fallback RV64GC 0 -
67859894.0
( 0.8x )
224162
( 1.659 )
110216
( 1.921 )
0 NHWC TVM Fallback RV64GC 0 -
110154013.0
( 0.5x )
293516
( 2.172 )
110216
( 1.921 )
128 NCHW TVM Fallback RV64GCV 0 Loop+SLP
109051500.0
( 0.5x )
290948
( 2.153 )
110216
( 1.921 )
256 NCHW TVM Fallback RV64GCV 0 Loop+SLP
109583101.0
( 0.5x )
291334
( 2.156 )
110216
( 1.921 )
512 NCHW TVM Fallback RV64GCV 0 Loop+SLP
109685479.0
( 0.5x )
290348
( 2.149 )
110216
( 1.921 )
1024 NCHW TVM Fallback RV64GCV 0 Loop+SLP
109915937.0
( 0.5x )
289688
( 2.144 )
110216
( 1.921 )
2048 NCHW TVM Fallback RV64GCV 0 Loop+SLP
110749310.0
( 0.5x )
289922
( 2.146 )
110216
( 1.921 )
4096 NCHW TVM Fallback RV64GCV 0 Loop+SLP
33941619.0
( 1.6x )
221966
( 1.643 )
110216
( 1.921 )
128 NHWC TVM Fallback RV64GCV 0 Loop+SLP
30800702.0
( 1.8x )
221988
( 1.643 )
110216
( 1.921 )
256 NHWC TVM Fallback RV64GCV 0 Loop+SLP
28432722.0
( 2.0x )
220310
( 1.631 )
110216
( 1.921 )
512 NHWC TVM Fallback RV64GCV 0 Loop+SLP
27154118.0
( 2.0x )
219536
( 1.625 )
110216
( 1.921 )
1024 NHWC TVM Fallback RV64GCV 0 Loop+SLP
26663664.0
( 2.1x )
218750
( 1.619 )
110216
( 1.921 )
2048 NHWC TVM Fallback RV64GCV 0 Loop+SLP
26575287.0
( 2.1x )
218184
( 1.615 )
110216
( 1.921 )
4096 NHWC TVM Fallback RV64GCV 0 Loop+SLP
55600773.0
( Base )
135116
( Base )
57376
( Base )
0 NHWC muRISCV-NN Scalar RV64GC 0 -
71380580.0
( 0.8x )
134784
( 0.998 )
57376
( 1.0 )
0 NHWC muRISCV-NN Vector (Portable) RV64GC 0 -
13524487.0
( 4.1x )
144616
( 1.07 )
57376
( 1.0 )
128 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
11088249.0
( 5.0x )
144102
( 1.067 )
57376
( 1.0 )
256 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
13926693.0
( 4.0x )
144102
( 1.067 )
57376
( 1.0 )
512 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
15406836.0
( 3.6x )
144102
( 1.067 )
57376
( 1.0 )
1024 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
16079408.0
( 3.5x )
144102
( 1.067 )
57376
( 1.0 )
2048 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
35273375.0
( 1.6x )
144102
( 1.067 )
57376
( 1.0 )
4096 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
15151856.0
( 3.7x )
137842
( 1.02 )
57376
( 1.0 )
128 NHWC muRISCV-NN Vector RV64GCV 0 -
9568658.0
( 5.8x )
137324
( 1.016 )
57376
( 1.0 )
256 NHWC muRISCV-NN Vector RV64GCV 0 -
7022818.0
( 7.9x )
137324
( 1.016 )
57376
( 1.0 )
512 NHWC muRISCV-NN Vector RV64GCV 0 -
5780690.0
( 9.6x )
137324
( 1.016 )
57376
( 1.0 )
1024 NHWC muRISCV-NN Vector RV64GCV 0 -
4854291.0
( 11.5x )
137324
( 1.016 )
57376
( 1.0 )
2048 NHWC muRISCV-NN Vector RV64GCV 0 -
4607064.0
( 12.1x )
137324
( 1.016 )
57376
( 1.0 )
4096 NHWC muRISCV-NN Vector RV64GCV 0 -
16929120.0
( 3.3x )
143938
( 1.065 )
57376
( 1.0 )
128 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
15894458.0
( 3.5x )
143424
( 1.061 )
57376
( 1.0 )
256 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
16646967.0
( 3.3x )
143428
( 1.062 )
57376
( 1.0 )
512 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
16355482.0
( 3.4x )
143428
( 1.062 )
57376
( 1.0 )
1024 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
15883222.0
( 3.5x )
143428
( 1.062 )
57376
( 1.0 )
2048 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
42667589.0
( 1.3x )
143428
( 1.062 )
57376
( 1.0 )
4096 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP

Anomaly Detection (toycar)

Cycles (Speedup) Total ROM (rel.) Total RAM (rel.) VLEN Layout Kernels Mode Arch Unroll Auto-Vectorization
1629208.0
( 1.0x )
582990
( 1.863 )
7408
( 1.121 )
0 NCHW TVM Fallback RV64GC 0 -
1629208.0
( 1.0x )
582990
( 1.863 )
7408
( 1.121 )
0 NHWC TVM Fallback RV64GC 0 -
827111.0
( 2.0x )
586314
( 1.874 )
7408
( 1.121 )
128 NCHW TVM Fallback RV64GCV 0 Loop+SLP
767987.0
( 2.2x )
586068
( 1.873 )
7408
( 1.121 )
256 NCHW TVM Fallback RV64GCV 0 Loop+SLP
942838.0
( 1.8x )
586912
( 1.876 )
7408
( 1.121 )
512 NCHW TVM Fallback RV64GCV 0 Loop+SLP
827944.0
( 2.0x )
586822
( 1.876 )
7408
( 1.121 )
1024 NCHW TVM Fallback RV64GCV 0 Loop+SLP
793852.0
( 2.1x )
595760
( 1.904 )
7408
( 1.121 )
2048 NCHW TVM Fallback RV64GCV 0 Loop+SLP
737143.0
( 2.3x )
587926
( 1.879 )
7408
( 1.121 )
4096 NCHW TVM Fallback RV64GCV 0 Loop+SLP
827111.0
( 2.0x )
586314
( 1.874 )
7408
( 1.121 )
128 NHWC TVM Fallback RV64GCV 0 Loop+SLP
767987.0
( 2.2x )
586068
( 1.873 )
7408
( 1.121 )
256 NHWC TVM Fallback RV64GCV 0 Loop+SLP
942838.0
( 1.8x )
586912
( 1.876 )
7408
( 1.121 )
512 NHWC TVM Fallback RV64GCV 0 Loop+SLP
827944.0
( 2.0x )
586822
( 1.876 )
7408
( 1.121 )
1024 NHWC TVM Fallback RV64GCV 0 Loop+SLP
793852.0
( 2.1x )
595760
( 1.904 )
7408
( 1.121 )
2048 NHWC TVM Fallback RV64GCV 0 Loop+SLP
737143.0
( 2.3x )
587926
( 1.879 )
7408
( 1.121 )
4096 NHWC TVM Fallback RV64GCV 0 Loop+SLP
1658945.0
( Base )
312866
( Base )
6608
( Base )
0 NHWC muRISCV-NN Scalar RV64GC 0 -
1658945.0
( 1.0x )
312866
( 1.0 )
6608
( 1.0 )
0 NHWC muRISCV-NN Vector (Portable) RV64GC 0 -
582403.0
( 2.8x )
314130
( 1.004 )
6608
( 1.0 )
128 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
490149.0
( 3.4x )
313680
( 1.003 )
6608
( 1.0 )
256 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
444933.0
( 3.7x )
313680
( 1.003 )
6608
( 1.0 )
512 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
422325.0
( 3.9x )
313680
( 1.003 )
6608
( 1.0 )
1024 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
411021.0
( 4.0x )
313680
( 1.003 )
6608
( 1.0 )
2048 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
1349550.0
( 1.2x )
313680
( 1.003 )
6608
( 1.0 )
4096 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
623306.0
( 2.7x )
314288
( 1.005 )
6608
( 1.0 )
128 NHWC muRISCV-NN Vector RV64GCV 0 -
499787.0
( 3.3x )
313874
( 1.003 )
6608
( 1.0 )
256 NHWC muRISCV-NN Vector RV64GCV 0 -
440603.0
( 3.8x )
313874
( 1.003 )
6608
( 1.0 )
512 NHWC muRISCV-NN Vector RV64GCV 0 -
411347.0
( 4.0x )
313874
( 1.003 )
6608
( 1.0 )
1024 NHWC muRISCV-NN Vector RV64GCV 0 -
407663.0
( 4.1x )
313874
( 1.003 )
6608
( 1.0 )
2048 NHWC muRISCV-NN Vector RV64GCV 0 -
405779.0
( 4.1x )
313874
( 1.003 )
6608
( 1.0 )
4096 NHWC muRISCV-NN Vector RV64GCV 0 -
582403.0
( 2.8x )
314130
( 1.004 )
6608
( 1.0 )
128 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
490149.0
( 3.4x )
313680
( 1.003 )
6608
( 1.0 )
256 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
444933.0
( 3.7x )
313680
( 1.003 )
6608
( 1.0 )
512 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
422325.0
( 3.9x )
313680
( 1.003 )
6608
( 1.0 )
1024 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
411021.0
( 4.0x )
313680
( 1.003 )
6608
( 1.0 )
2048 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
1349550.0
( 1.2x )
313680
( 1.003 )
6608
( 1.0 )
4096 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP

Visual Wake Words (vww)

Cycles (Speedup) Total ROM (rel.) Total RAM (rel.) VLEN Layout Kernels Mode Arch Unroll Auto-Vectorization
37389018.0
( 1.2x )
575098
( 1.793 )
182880
( 2.09 )
0 NCHW TVM Fallback RV64GC 0 -
53415162.0
( 0.8x )
551138
( 1.718 )
182824
( 2.089 )
0 NHWC TVM Fallback RV64GC 0 -
67855139.0
( 0.6x )
712782
( 2.222 )
182832
( 2.089 )
128 NCHW TVM Fallback RV64GCV 0 Loop+SLP
69111644.0
( 0.6x )
717506
( 2.237 )
182824
( 2.089 )
256 NCHW TVM Fallback RV64GCV 0 Loop+SLP
68011322.0
( 0.6x )
718656
( 2.241 )
182824
( 2.089 )
512 NCHW TVM Fallback RV64GCV 0 Loop+SLP
68681188.0
( 0.6x )
721502
( 2.25 )
182824
( 2.089 )
1024 NCHW TVM Fallback RV64GCV 0 Loop+SLP
68745809.0
( 0.6x )
722470
( 2.253 )
182824
( 2.089 )
2048 NCHW TVM Fallback RV64GCV 0 Loop+SLP
68508422.0
( 0.6x )
721732
( 2.25 )
182824
( 2.089 )
4096 NCHW TVM Fallback RV64GCV 0 Loop+SLP
11908833.0
( 3.7x )
622782
( 1.942 )
182824
( 2.089 )
128 NHWC TVM Fallback RV64GCV 0 Loop+SLP
8008466.0
( 5.5x )
621752
( 1.939 )
182824
( 2.089 )
256 NHWC TVM Fallback RV64GCV 0 Loop+SLP
5745353.0
( 7.7x )
615936
( 1.92 )
182824
( 2.089 )
512 NHWC TVM Fallback RV64GCV 0 Loop+SLP
4738832.0
( 9.3x )
603386
( 1.881 )
182824
( 2.089 )
1024 NHWC TVM Fallback RV64GCV 0 Loop+SLP
4231032.0
( 10.4x )
602658
( 1.879 )
182824
( 2.089 )
2048 NHWC TVM Fallback RV64GCV 0 Loop+SLP
4196754.0
( 10.5x )
606450
( 1.891 )
182824
( 2.089 )
4096 NHWC TVM Fallback RV64GCV 0 Loop+SLP
43963094.0
( Base )
320734
( Base )
87512
( Base )
0 NHWC muRISCV-NN Scalar RV64GC 0 -
44869073.0
( 1.0x )
320760
( 1.0 )
87512
( 1.0 )
0 NHWC muRISCV-NN Vector (Portable) RV64GC 0 -
16952261.0
( 2.6x )
330294
( 1.03 )
87512
( 1.0 )
128 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
16531848.0
( 2.7x )
328908
( 1.025 )
87512
( 1.0 )
256 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
17711089.0
( 2.5x )
328910
( 1.025 )
87512
( 1.0 )
512 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
20862898.0
( 2.1x )
328912
( 1.025 )
87512
( 1.0 )
1024 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
24388079.0
( 1.8x )
328910
( 1.025 )
87512
( 1.0 )
2048 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
38352866.0
( 1.1x )
328910
( 1.025 )
87512
( 1.0 )
4096 NHWC muRISCV-NN Scalar RV64GCV 0 Loop+SLP
13374816.0
( 3.3x )
324284
( 1.011 )
87512
( 1.0 )
128 NHWC muRISCV-NN Vector RV64GCV 0 -
10048698.0
( 4.4x )
323024
( 1.007 )
87512
( 1.0 )
256 NHWC muRISCV-NN Vector RV64GCV 0 -
8757279.0
( 5.0x )
323026
( 1.007 )
87512
( 1.0 )
512 NHWC muRISCV-NN Vector RV64GCV 0 -
8255158.0
( 5.3x )
323028
( 1.007 )
87512
( 1.0 )
1024 NHWC muRISCV-NN Vector RV64GCV 0 -
8207392.0
( 5.4x )
323028
( 1.007 )
87512
( 1.0 )
2048 NHWC muRISCV-NN Vector RV64GCV 0 -
8209256.0
( 5.4x )
323022
( 1.007 )
87512
( 1.0 )
4096 NHWC muRISCV-NN Vector RV64GCV 0 -
20591882.0
( 2.1x )
330964
( 1.032 )
87512
( 1.0 )
128 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
20319470.0
( 2.2x )
329710
( 1.028 )
87512
( 1.0 )
256 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
21837399.0
( 2.0x )
329724
( 1.028 )
87512
( 1.0 )
512 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
24989208.0
( 1.8x )
329724
( 1.028 )
87512
( 1.0 )
1024 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
28506085.0
( 1.5x )
329674
( 1.028 )
87512
( 1.0 )
2048 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
42470872.0
( 1.0x )
329674
( 1.028 )
87512
( 1.0 )
4096 NHWC muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP

Original data

Click here to download the raw files for this benchmark.

2024-11-26
2024-11-21
2024-11-19
2024-11-18
2024-07-12
2024-06-29
2024-03-02
2024-02-26
2024-02-23
2024-02-22
2024-02-20
2024-02-11
2023-12-22
Clone this wiki locally