Skip to content

Benchmarks 2024 11 21 TFLM LLVM O3 spike_rv64

GitHub Action edited this page Nov 21, 2024 · 1 revision

Setup

Simulator

  • Spike (riscv-isa-sim ) (ISS, CPI=1)
    • Spike : 0bc176b3fca43560b9e8586cdbc41cfde073e17a
    • Spike PK : 7e9b671c0415dfd7b562ac934feb9380075d4aa2

Toolchains

Models

Frameworks

  • MLonMCU : develop

  • TFLM : 8eb6b23de4470d6a8da3131650d6a67514dfa130

Miscellaneous

  • Used -Os flag for compilation.
  • Benchmarks generated using MLonMCU deployment tool with minimal efforts.
  • Memory metrics are reported in Bytes

Results (Framework: tflm, Backend: tflmi, Toolchain: llvm, Flags: -O3, Target: spike_rv64 )

Audio Wake Words (aww)

Cycles (Speedup) Total ROM (rel.) Total RAM (rel.) VLEN Kernels Mode Arch Unroll Auto-Vectorization
38220981
( 0.4x )
151210
( 0.884 )
38312
( 1.0 )
0 TFLM Reference RV64GC 0 -
29524020
( 0.5x )
161498
( 0.944 )
38320
( 1.0 )
128 TFLM Reference RV64GCV 0 Loop+SLP
28115428
( 0.5x )
161498
( 0.944 )
38320
( 1.0 )
256 TFLM Reference RV64GCV 0 Loop+SLP
27317724
( 0.5x )
161498
( 0.944 )
38320
( 1.0 )
512 TFLM Reference RV64GCV 0 Loop+SLP
26965592
( 0.5x )
161498
( 0.944 )
38320
( 1.0 )
1024 TFLM Reference RV64GCV 0 Loop+SLP
42750153
( 0.3x )
161498
( 0.944 )
38320
( 1.0 )
2048 TFLM Reference RV64GCV 0 Loop+SLP
42748288
( 0.3x )
161498
( 0.944 )
38320
( 1.0 )
4096 TFLM Reference RV64GCV 0 Loop+SLP
14667305
( Base )
171092
( Base )
38312
( Base )
0 muRISCV-NN Scalar RV64GC 0 -
15003544
( 1.0x )
171044
( 1.0 )
38312
( 1.0 )
0 muRISCV-NN Vector (Portable) RV64GC 0 -
5159524
( 2.8x )
192794
( 1.127 )
38320
( 1.0 )
128 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
4736424
( 3.1x )
192794
( 1.127 )
38320
( 1.0 )
256 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
4412659
( 3.3x )
192794
( 1.127 )
38320
( 1.0 )
512 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
5158864
( 2.8x )
192794
( 1.127 )
38320
( 1.0 )
1024 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
13633342
( 1.1x )
192794
( 1.127 )
38320
( 1.0 )
2048 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
13631477
( 1.1x )
192794
( 1.127 )
38320
( 1.0 )
4096 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
4062587
( 3.6x )
173830
( 1.016 )
38312
( 1.0 )
128 muRISCV-NN Vector RV64GCV 0 -
2807619
( 5.2x )
173830
( 1.016 )
38312
( 1.0 )
256 muRISCV-NN Vector RV64GCV 0 -
2130307
( 6.9x )
173830
( 1.016 )
38312
( 1.0 )
512 muRISCV-NN Vector RV64GCV 0 -
2091188
( 7.0x )
173830
( 1.016 )
38312
( 1.0 )
1024 muRISCV-NN Vector RV64GCV 0 -
2091188
( 7.0x )
173830
( 1.016 )
38312
( 1.0 )
2048 muRISCV-NN Vector RV64GCV 0 -
2093053
( 7.0x )
173830
( 1.016 )
38312
( 1.0 )
4096 muRISCV-NN Vector RV64GCV 0 -
6560220
( 2.2x )
193660
( 1.132 )
38320
( 1.0 )
128 muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
5888832
( 2.5x )
193660
( 1.132 )
38320
( 1.0 )
256 muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
5552289
( 2.6x )
193660
( 1.132 )
38320
( 1.0 )
512 muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
6635414
( 2.2x )
193660
( 1.132 )
38320
( 1.0 )
1024 muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
15113672
( 1.0x )
193660
( 1.132 )
38320
( 1.0 )
2048 muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
15111807
( 1.0x )
193660
( 1.132 )
38320
( 1.0 )
4096 muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP

Image Classification (resnet)

Cycles (Speedup) Total ROM (rel.) Total RAM (rel.) VLEN Kernels Mode Arch Unroll Auto-Vectorization
118582357
( 0.5x )
191040
( 0.935 )
71080
( 1.0 )
0 TFLM Reference RV64GC 0 -
52636165
( 1.1x )
203018
( 0.994 )
71088
( 1.0 )
128 TFLM Reference RV64GCV 0 Loop+SLP
44969645
( 1.2x )
203018
( 0.994 )
71088
( 1.0 )
256 TFLM Reference RV64GCV 0 Loop+SLP
82543537
( 0.7x )
203018
( 0.994 )
71088
( 1.0 )
512 TFLM Reference RV64GCV 0 Loop+SLP
107245059
( 0.5x )
203018
( 0.994 )
71088
( 1.0 )
1024 TFLM Reference RV64GCV 0 Loop+SLP
122521807
( 0.5x )
203018
( 0.994 )
71088
( 1.0 )
2048 TFLM Reference RV64GCV 0 Loop+SLP
122521807
( 0.5x )
203018
( 0.994 )
71088
( 1.0 )
4096 TFLM Reference RV64GCV 0 Loop+SLP
55593701
( Base )
204290
( Base )
71080
( Base )
0 muRISCV-NN Scalar RV64GC 0 -
71341169
( 0.8x )
204028
( 0.999 )
71080
( 1.0 )
0 muRISCV-NN Vector (Portable) RV64GC 0 -
13511242
( 4.1x )
228234
( 1.117 )
71088
( 1.0 )
128 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
11074996
( 5.0x )
228234
( 1.117 )
71088
( 1.0 )
256 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
13881658
( 4.0x )
228234
( 1.117 )
71088
( 1.0 )
512 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
15363666
( 3.6x )
228234
( 1.117 )
71088
( 1.0 )
1024 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
16036238
( 3.5x )
228234
( 1.117 )
71088
( 1.0 )
2048 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
35226475
( 1.6x )
228234
( 1.117 )
71088
( 1.0 )
4096 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
15147423
( 3.7x )
207274
( 1.015 )
71080
( 1.0 )
128 muRISCV-NN Vector RV64GCV 0 -
9564223
( 5.8x )
207274
( 1.015 )
71080
( 1.0 )
256 muRISCV-NN Vector RV64GCV 0 -
7018383
( 7.9x )
207274
( 1.015 )
71080
( 1.0 )
512 muRISCV-NN Vector RV64GCV 0 -
5778120
( 9.6x )
207274
( 1.015 )
71080
( 1.0 )
1024 muRISCV-NN Vector RV64GCV 0 -
4849856
( 11.5x )
207274
( 1.015 )
71080
( 1.0 )
2048 muRISCV-NN Vector RV64GCV 0 -
4602629
( 12.1x )
207274
( 1.015 )
71080
( 1.0 )
4096 muRISCV-NN Vector RV64GCV 0 -
16882088
( 3.3x )
227764
( 1.115 )
71088
( 1.0 )
128 muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
15847171
( 3.5x )
227764
( 1.115 )
71088
( 1.0 )
256 muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
16599560
( 3.3x )
227764
( 1.115 )
71088
( 1.0 )
512 muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
16309880
( 3.4x )
227764
( 1.115 )
71088
( 1.0 )
1024 muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
15840770
( 3.5x )
227764
( 1.115 )
71088
( 1.0 )
2048 muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
42621407
( 1.3x )
227764
( 1.115 )
71088
( 1.0 )
4096 muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP

Anomaly Detection (toycar)

Cycles (Speedup) Total ROM (rel.) Total RAM (rel.) VLEN Kernels Mode Arch Unroll Auto-Vectorization
2781356
( 0.6x )
339582
( 0.984 )
21440
( 1.0 )
0 TFLM Reference RV64GC 0 -
787134
( 2.1x )
343354
( 0.995 )
21440
( 1.0 )
128 TFLM Reference RV64GCV 0 Loop+SLP
613118
( 2.7x )
343354
( 0.995 )
21440
( 1.0 )
256 TFLM Reference RV64GCV 0 Loop+SLP
522654
( 3.1x )
343354
( 0.995 )
21440
( 1.0 )
512 TFLM Reference RV64GCV 0 Loop+SLP
477422
( 3.4x )
343354
( 0.995 )
21440
( 1.0 )
1024 TFLM Reference RV64GCV 0 Loop+SLP
454806
( 3.6x )
343354
( 0.995 )
21440
( 1.0 )
2048 TFLM Reference RV64GCV 0 Loop+SLP
2204550
( 0.7x )
343354
( 0.995 )
21440
( 1.0 )
4096 TFLM Reference RV64GCV 0 Loop+SLP
1630665
( Base )
345136
( Base )
21440
( Base )
0 muRISCV-NN Scalar RV64GC 0 -
2979690
( 0.5x )
345140
( 1.0 )
21440
( 1.0 )
0 muRISCV-NN Vector (Portable) RV64GC 0 -
550028
( 3.0x )
350532
( 1.016 )
21440
( 1.0 )
128 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
462956
( 3.5x )
350532
( 1.016 )
21440
( 1.0 )
256 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
417740
( 3.9x )
350532
( 1.016 )
21440
( 1.0 )
512 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
395132
( 4.1x )
350532
( 1.016 )
21440
( 1.0 )
1024 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
383828
( 4.2x )
350532
( 1.016 )
21440
( 1.0 )
2048 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
1322357
( 1.2x )
350532
( 1.016 )
21440
( 1.0 )
4096 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
1938907
( 0.8x )
346416
( 1.004 )
21440
( 1.0 )
128 muRISCV-NN Vector RV64GCV 0 -
1820539
( 0.9x )
346416
( 1.004 )
21440
( 1.0 )
256 muRISCV-NN Vector RV64GCV 0 -
1761355
( 0.9x )
346416
( 1.004 )
21440
( 1.0 )
512 muRISCV-NN Vector RV64GCV 0 -
1732099
( 0.9x )
346416
( 1.004 )
21440
( 1.0 )
1024 muRISCV-NN Vector RV64GCV 0 -
1728415
( 0.9x )
346416
( 1.004 )
21440
( 1.0 )
2048 muRISCV-NN Vector RV64GCV 0 -
1726531
( 0.9x )
346416
( 1.004 )
21440
( 1.0 )
4096 muRISCV-NN Vector RV64GCV 0 -
782955
( 2.1x )
350536
( 1.016 )
21440
( 1.0 )
128 muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
601675
( 2.7x )
350536
( 1.016 )
21440
( 1.0 )
256 muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
507115
( 3.2x )
350536
( 1.016 )
21440
( 1.0 )
512 muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
459835
( 3.5x )
350536
( 1.016 )
21440
( 1.0 )
1024 muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
436195
( 3.7x )
350536
( 1.016 )
21440
( 1.0 )
2048 muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
2354268
( 0.7x )
350536
( 1.016 )
21440
( 1.0 )
4096 muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP

Visual Wake Words (vww)

Cycles (Speedup) Total ROM (rel.) Total RAM (rel.) VLEN Kernels Mode Arch Unroll Auto-Vectorization
100751269
( 0.4x )
424852
( 0.955 )
136616
( 1.0 )
0 TFLM Reference RV64GC 0 -
71104580
( 0.6x )
435140
( 0.978 )
136624
( 1.0 )
128 TFLM Reference RV64GCV 0 Loop+SLP
68707812
( 0.6x )
435140
( 0.978 )
136624
( 1.0 )
256 TFLM Reference RV64GCV 0 Loop+SLP
68539060
( 0.6x )
435140
( 0.978 )
136624
( 1.0 )
512 TFLM Reference RV64GCV 0 Loop+SLP
74280540
( 0.6x )
435140
( 0.978 )
136624
( 1.0 )
1024 TFLM Reference RV64GCV 0 Loop+SLP
80768121
( 0.5x )
435140
( 0.978 )
136624
( 1.0 )
2048 TFLM Reference RV64GCV 0 Loop+SLP
106211610
( 0.4x )
435140
( 0.978 )
136624
( 1.0 )
4096 TFLM Reference RV64GCV 0 Loop+SLP
43868312
( Base )
444734
( Base )
136616
( Base )
0 muRISCV-NN Scalar RV64GC 0 -
44684389
( 1.0x )
444686
( 1.0 )
136616
( 1.0 )
0 muRISCV-NN Vector (Portable) RV64GC 0 -
16851584
( 2.6x )
466436
( 1.049 )
136624
( 1.0 )
128 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
16431200
( 2.7x )
466436
( 1.049 )
136624
( 1.0 )
256 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
17589992
( 2.5x )
466436
( 1.049 )
136624
( 1.0 )
512 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
20741801
( 2.1x )
466436
( 1.049 )
136624
( 1.0 )
1024 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
24268847
( 1.8x )
466436
( 1.049 )
136624
( 1.0 )
2048 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
38229904
( 1.1x )
466436
( 1.049 )
136624
( 1.0 )
4096 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
13266657
( 3.3x )
447472
( 1.006 )
136616
( 1.0 )
128 muRISCV-NN Vector RV64GCV 0 -
9945673
( 4.4x )
447472
( 1.006 )
136616
( 1.0 )
256 muRISCV-NN Vector RV64GCV 0 -
8654253
( 5.1x )
447472
( 1.006 )
136616
( 1.0 )
512 muRISCV-NN Vector RV64GCV 0 -
8152132
( 5.4x )
447472
( 1.006 )
136616
( 1.0 )
1024 muRISCV-NN Vector RV64GCV 0 -
8104366
( 5.4x )
447472
( 1.006 )
136616
( 1.0 )
2048 muRISCV-NN Vector RV64GCV 0 -
8106231
( 5.4x )
447472
( 1.006 )
136616
( 1.0 )
4096 muRISCV-NN Vector RV64GCV 0 -
20404256
( 2.1x )
467302
( 1.051 )
136624
( 1.0 )
128 muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
20126528
( 2.2x )
467302
( 1.051 )
136624
( 1.0 )
256 muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
21644361
( 2.0x )
467302
( 1.051 )
136624
( 1.0 )
512 muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
24796122
( 1.8x )
467302
( 1.051 )
136624
( 1.0 )
1024 muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
28323144
( 1.5x )
467302
( 1.051 )
136624
( 1.0 )
2048 muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP
42284189
( 1.0x )
467302
( 1.051 )
136624
( 1.0 )
4096 muRISCV-NN Vector (Portable) RV64GCV 0 Loop+SLP

Original data

Click here to download the raw files for this benchmark.

2024-11-26
2024-11-21
2024-11-19
2024-11-18
2024-07-12
2024-06-29
2024-03-02
2024-02-26
2024-02-23
2024-02-22
2024-02-20
2024-02-11
2023-12-22
Clone this wiki locally