Skip to content

Benchmarks 2024 11 26 TFLM GCC O3 spike_rv64

GitHub Action edited this page Nov 26, 2024 · 1 revision

Setup

Simulator

  • Spike (riscv-isa-sim ) (ISS, CPI=1)
    • Spike : eb0a3e2b0a7c57522928be39de95cd9f8c6dc636
    • Spike PK : fix-gcc14-rvv

Toolchains

Models

Frameworks

  • MLonMCU : develop

  • TFLM : 8eb6b23de4470d6a8da3131650d6a67514dfa130

Miscellaneous

  • Used -Os flag for compilation.
  • Benchmarks generated using MLonMCU deployment tool with minimal efforts.
  • Memory metrics are reported in Bytes

Results (Framework: tflm, Backend: tflmi, Toolchain: gcc, Flags: -O3, Target: spike_rv64 )

Audio Wake Words (aww)

Cycles (Speedup) Total ROM (rel.) Total RAM (rel.) VLEN Kernels Mode Arch Unroll Auto-Vectorization
56875270.0
( 0.3x )
144396
( 0.817 )
38392
( 1.0 )
0 TFLM Reference RV64GC 0 -
36048336.0
( 0.4x )
152228
( 0.861 )
38448
( 1.001 )
128 TFLM Reference RV64GCV 0 Loop+SLP
33166398.0
( 0.5x )
153026
( 0.865 )
38448
( 1.001 )
256 TFLM Reference RV64GCV 0 Loop+SLP
31808365.0
( 0.5x )
153986
( 0.871 )
38448
( 1.001 )
512 TFLM Reference RV64GCV 0 Loop+SLP
31009356.0
( 0.5x )
155118
( 0.877 )
38448
( 1.001 )
1024 TFLM Reference RV64GCV 0 Loop+SLP
31199422.0
( 0.5x )
156382
( 0.884 )
38448
( 1.001 )
2048 TFLM Reference RV64GCV 0 Loop+SLP
31038432.0
( 0.5x )
158336
( 0.895 )
38424
( 1.001 )
4096 TFLM Reference RV64GCV 0 Loop+SLP
15097076.0
( Base )
176846
( Base )
38400
( Base )
0 muRISCV-NN Scalar RV64GC 0 -
15557110.0
( 1.0x )
173798
( 0.983 )
38400
( 1.0 )
0 muRISCV-NN Vector (Portable) RV64GC 0 -
8154451.0
( 1.9x )
195232
( 1.104 )
38448
( 1.001 )
128 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
6751660.0
( 2.2x )
200302
( 1.133 )
38456
( 1.001 )
256 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
6067798.0
( 2.5x )
206948
( 1.17 )
38456
( 1.001 )
512 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
5729747.0
( 2.6x )
219174
( 1.239 )
38456
( 1.001 )
1024 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
5552158.0
( 2.7x )
251376
( 1.421 )
38456
( 1.001 )
2048 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
5548948.0
( 2.7x )
325690
( 1.842 )
38432
( 1.001 )
4096 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
7903715.0
( 1.9x )
178516
( 1.009 )
38400
( 1.0 )
128 muRISCV-NN Vector RV64GCV 0 -
5359705.0
( 2.8x )
178516
( 1.009 )
38400
( 1.0 )
256 muRISCV-NN Vector RV64GCV 0 -
3866590.0
( 3.9x )
178516
( 1.009 )
38400
( 1.0 )
512 muRISCV-NN Vector RV64GCV 0 -
3828538.0
( 3.9x )
178516
( 1.009 )
38400
( 1.0 )
1024 muRISCV-NN Vector RV64GCV 0 -
3830403.0
( 3.9x )
178516
( 1.009 )
38400
( 1.0 )
2048 muRISCV-NN Vector RV64GCV 0 -
3832268.0
( 3.9x )
178516
( 1.009 )
38400
( 1.0 )
4096 muRISCV-NN Vector RV64GCV 0 -
7855898.0
( 1.9x )
193134
( 1.092 )
38448
( 1.001 )
128 muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
5627083.0
( 2.7x )
197528
( 1.117 )
38456
( 1.001 )
256 muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
4612861.0
( 3.3x )
204234
( 1.155 )
38456
( 1.001 )
512 muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
3962426.0
( 3.8x )
216456
( 1.224 )
38456
( 1.001 )
1024 muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
3743430.0
( 4.0x )
248660
( 1.406 )
38456
( 1.001 )
2048 muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
3752783.0
( 4.0x )
322976
( 1.826 )
38432
( 1.001 )
4096 muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP

Image Classification (resnet)

Cycles (Speedup) Total ROM (rel.) Total RAM (rel.) VLEN Kernels Mode Arch Unroll Auto-Vectorization
171735280.0
( 0.3x )
192190
( 0.905 )
71160
( 1.0 )
0 TFLM Reference RV64GC 0 -
70156517.0
( 0.8x )
206156
( 0.97 )
71224
( 1.001 )
128 TFLM Reference RV64GCV 0 Loop+SLP
54001363.0
( 1.0x )
207558
( 0.977 )
71232
( 1.001 )
256 TFLM Reference RV64GCV 0 Loop+SLP
47176487.0
( 1.2x )
210108
( 0.989 )
71232
( 1.001 )
512 TFLM Reference RV64GCV 0 Loop+SLP
44502007.0
( 1.2x )
212014
( 0.998 )
71232
( 1.001 )
1024 TFLM Reference RV64GCV 0 Loop+SLP
45454299.0
( 1.2x )
214294
( 1.009 )
71224
( 1.001 )
2048 TFLM Reference RV64GCV 0 Loop+SLP
44984554.0
( 1.2x )
217924
( 1.026 )
71200
( 1.001 )
4096 TFLM Reference RV64GCV 0 Loop+SLP
54280182.0
( Base )
212432
( Base )
71152
( Base )
0 muRISCV-NN Scalar RV64GC 0 -
71759886.0
( 0.8x )
211752
( 0.997 )
71152
( 1.0 )
0 muRISCV-NN Vector (Portable) RV64GC 0 -
17929483.0
( 3.0x )
236998
( 1.116 )
71216
( 1.001 )
128 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
11420958.0
( 4.8x )
237956
( 1.12 )
71216
( 1.001 )
256 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
8216241.0
( 6.6x )
239982
( 1.13 )
71216
( 1.001 )
512 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
6740087.0
( 8.1x )
241618
( 1.137 )
71216
( 1.001 )
1024 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
6059054.0
( 9.0x )
243182
( 1.145 )
71216
( 1.001 )
2048 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
5727577.0
( 9.5x )
245964
( 1.158 )
71192
( 1.001 )
4096 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
31987203.0
( 1.7x )
219842
( 1.035 )
71152
( 1.0 )
128 muRISCV-NN Vector RV64GCV 0 -
19580875.0
( 2.8x )
219842
( 1.035 )
71152
( 1.0 )
256 muRISCV-NN Vector RV64GCV 0 -
13949944.0
( 3.9x )
219842
( 1.035 )
71152
( 1.0 )
512 muRISCV-NN Vector RV64GCV 0 -
11201800.0
( 4.8x )
219842
( 1.035 )
71152
( 1.0 )
1024 muRISCV-NN Vector RV64GCV 0 -
9046233.0
( 6.0x )
219842
( 1.035 )
71152
( 1.0 )
2048 muRISCV-NN Vector RV64GCV 0 -
8468399.0
( 6.4x )
219842
( 1.035 )
71152
( 1.0 )
4096 muRISCV-NN Vector RV64GCV 0 -
29168736.0
( 1.9x )
236368
( 1.113 )
71216
( 1.001 )
128 muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
17060274.0
( 3.2x )
237326
( 1.117 )
71216
( 1.001 )
256 muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
10954874.0
( 5.0x )
239352
( 1.127 )
71216
( 1.001 )
512 muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
8085078.0
( 6.7x )
240988
( 1.134 )
71216
( 1.001 )
1024 muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
6777768.0
( 8.0x )
242552
( 1.142 )
71216
( 1.001 )
2048 muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
6135548.0
( 8.8x )
245334
( 1.155 )
71192
( 1.001 )
4096 muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP

Anomaly Detection (toycar)

Cycles (Speedup) Total ROM (rel.) Total RAM (rel.) VLEN Kernels Mode Arch Unroll Auto-Vectorization
2773419.0
( 0.6x )
336470
( 0.977 )
21512
( 1.0 )
0 TFLM Reference RV64GC 0 -
1218838.0
( 1.3x )
339158
( 0.984 )
21512
( 1.0 )
128 TFLM Reference RV64GCV 0 Loop+SLP
822780.0
( 2.0x )
339680
( 0.986 )
21512
( 1.0 )
256 TFLM Reference RV64GCV 0 Loop+SLP
625346.0
( 2.6x )
339898
( 0.987 )
21512
( 1.0 )
512 TFLM Reference RV64GCV 0 Loop+SLP
526633.0
( 3.1x )
340338
( 0.988 )
21512
( 1.0 )
1024 TFLM Reference RV64GCV 0 Loop+SLP
477305.0
( 3.4x )
340808
( 0.989 )
21512
( 1.0 )
2048 TFLM Reference RV64GCV 0 Loop+SLP
452624.0
( 3.6x )
341574
( 0.991 )
21512
( 1.0 )
4096 TFLM Reference RV64GCV 0 Loop+SLP
1645338.0
( Base )
344548
( Base )
21512
( Base )
0 muRISCV-NN Scalar RV64GC 0 -
2728482.0
( 0.6x )
344552
( 1.0 )
21512
( 1.0 )
0 muRISCV-NN Vector (Portable) RV64GC 0 -
727662.0
( 2.3x )
344882
( 1.001 )
21512
( 1.0 )
128 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
559451.0
( 2.9x )
345150
( 1.002 )
21512
( 1.0 )
256 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
468976.0
( 3.5x )
345152
( 1.002 )
21512
( 1.0 )
512 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
423130.0
( 3.9x )
345370
( 1.002 )
21512
( 1.0 )
1024 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
387824.0
( 4.2x )
345614
( 1.003 )
21512
( 1.0 )
2048 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
376540.0
( 4.4x )
346016
( 1.004 )
21512
( 1.0 )
4096 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
2018575.0
( 0.8x )
348208
( 1.011 )
21512
( 1.0 )
128 muRISCV-NN Vector RV64GCV 0 -
1758385.0
( 0.9x )
348208
( 1.011 )
21512
( 1.0 )
256 muRISCV-NN Vector RV64GCV 0 -
1628290.0
( 1.0x )
348208
( 1.011 )
21512
( 1.0 )
512 muRISCV-NN Vector RV64GCV 0 -
1564054.0
( 1.1x )
348208
( 1.011 )
21512
( 1.0 )
1024 muRISCV-NN Vector RV64GCV 0 -
1557817.0
( 1.1x )
348208
( 1.011 )
21512
( 1.0 )
2048 muRISCV-NN Vector RV64GCV 0 -
1555530.0
( 1.1x )
348208
( 1.011 )
21512
( 1.0 )
4096 muRISCV-NN Vector RV64GCV 0 -
1224399.0
( 1.3x )
344886
( 1.001 )
21512
( 1.0 )
128 muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
824793.0
( 2.0x )
345154
( 1.002 )
21512
( 1.0 )
256 muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
619091.0
( 2.7x )
345156
( 1.002 )
21512
( 1.0 )
512 muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
515656.0
( 3.2x )
345374
( 1.002 )
21512
( 1.0 )
1024 muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
451575.0
( 3.6x )
345618
( 1.003 )
21512
( 1.0 )
2048 muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
425844.0
( 3.9x )
346020
( 1.004 )
21512
( 1.0 )
4096 muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP

Visual Wake Words (vww)

Cycles (Speedup) Total ROM (rel.) Total RAM (rel.) VLEN Kernels Mode Arch Unroll Auto-Vectorization
146817507.0
( 0.3x )
418038
( 0.928 )
136696
( 1.0 )
0 TFLM Reference RV64GC 0 -
78400305.0
( 0.6x )
425870
( 0.945 )
136752
( 1.0 )
128 TFLM Reference RV64GCV 0 Loop+SLP
70545982.0
( 0.6x )
426668
( 0.947 )
136752
( 1.0 )
256 TFLM Reference RV64GCV 0 Loop+SLP
65840909.0
( 0.7x )
427628
( 0.949 )
136752
( 1.0 )
512 TFLM Reference RV64GCV 0 Loop+SLP
64475973.0
( 0.7x )
428760
( 0.952 )
136752
( 1.0 )
1024 TFLM Reference RV64GCV 0 Loop+SLP
64064837.0
( 0.7x )
430024
( 0.955 )
136752
( 1.0 )
2048 TFLM Reference RV64GCV 0 Loop+SLP
63551760.0
( 0.7x )
431978
( 0.959 )
136728
( 1.0 )
4096 TFLM Reference RV64GCV 0 Loop+SLP
45422167.0
( Base )
450488
( Base )
136704
( Base )
0 muRISCV-NN Scalar RV64GC 0 -
46730216.0
( 1.0x )
447440
( 0.993 )
136704
( 1.0 )
0 muRISCV-NN Vector (Portable) RV64GC 0 -
25318168.0
( 1.8x )
468882
( 1.041 )
136752
( 1.0 )
128 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
21253820.0
( 2.1x )
473952
( 1.052 )
136760
( 1.0 )
256 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
19284071.0
( 2.4x )
480598
( 1.067 )
136760
( 1.0 )
512 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
18340247.0
( 2.5x )
492808
( 1.094 )
136760
( 1.0 )
1024 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
17951765.0
( 2.5x )
525010
( 1.165 )
136760
( 1.0 )
2048 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
17820868.0
( 2.5x )
599340
( 1.33 )
136736
( 1.0 )
4096 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
25292800.0
( 1.8x )
452158
( 1.004 )
136704
( 1.0 )
128 muRISCV-NN Vector RV64GCV 0 -
18424373.0
( 2.5x )
452158
( 1.004 )
136704
( 1.0 )
256 muRISCV-NN Vector RV64GCV 0 -
15802964.0
( 2.9x )
452158
( 1.004 )
136704
( 1.0 )
512 muRISCV-NN Vector RV64GCV 0 -
14710149.0
( 3.1x )
452158
( 1.004 )
136704
( 1.0 )
1024 muRISCV-NN Vector RV64GCV 0 -
14622985.0
( 3.1x )
452158
( 1.004 )
136704
( 1.0 )
2048 muRISCV-NN Vector RV64GCV 0 -
14624850.0
( 3.1x )
452158
( 1.004 )
136704
( 1.0 )
4096 muRISCV-NN Vector RV64GCV 0 -
24252969.0
( 1.9x )
466784
( 1.036 )
136752
( 1.0 )
128 muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
18098949.0
( 2.5x )
471178
( 1.046 )
136760
( 1.0 )
256 muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
15214938.0
( 3.0x )
477884
( 1.061 )
136760
( 1.0 )
512 muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
13418795.0
( 3.4x )
490090
( 1.088 )
136760
( 1.0 )
1024 muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
13046792.0
( 3.5x )
522294
( 1.159 )
136760
( 1.0 )
2048 muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
12907388.0
( 3.5x )
596626
( 1.324 )
136736
( 1.0 )
4096 muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP

Original data

Click here to download the raw files for this benchmark.

2024-11-26
2024-11-21
2024-11-19
2024-11-18
2024-07-12
2024-06-29
2024-03-02
2024-02-26
2024-02-23
2024-02-22
2024-02-20
2024-02-11
2023-12-22
Clone this wiki locally