Skip to content

Benchmarks CUSTOM TFLM LLVM O3

Philipp van Kempen edited this page Nov 16, 2024 · 1 revision

Setup

Simulator

Toolchains

  • LLVM/Clang:
    • TODO: Version
    • Linker: lld (TODO)
    • RISC-V GCC for Headers, libc,...

Models

Package Versions

  • MLonMCU : main

  • TFLM : main

  • Spike : 0bc176b3fca43560b9e8586cdbc41cfde073e17a

  • Spike PK : 7e9b671c0415dfd7b562ac934feb9380075d4aa2

Miscellaneous

  • Used -Os flag for compilation.
  • Benchmarks generated using MLonMCU deployment tool with minimal efforts.
  • Memory metrics are reported in Bytes

Results (Framework: tflm, Backend: tflmi, Toolchain: llvm, Flags: -O3)

Audio Wake Words (aww)

Cycles (Speedup) Total ROM (rel.) Total RAM (rel.) VLEN Kernels Mode Arch Unroll Auto-Vectorization
43744941
( 0.3x )
184988
( 0.824 )
36148
( 1.0 )
0 TFLM Reference RV32GC 1 -
30274641
( 0.4x )
176782
( 0.788 )
36156
( 1.0 )
128 TFLM Reference RV32GCV 1 Loop+SLP
28775843
( 0.4x )
176782
( 0.788 )
36156
( 1.0 )
256 TFLM Reference RV32GCV 1 Loop+SLP
28071579
( 0.5x )
176782
( 0.788 )
36156
( 1.0 )
512 TFLM Reference RV32GCV 1 Loop+SLP
27722836
( 0.5x )
176782
( 0.788 )
36156
( 1.0 )
1024 TFLM Reference RV32GCV 1 Loop+SLP
45618312
( 0.3x )
176782
( 0.788 )
36156
( 1.0 )
2048 TFLM Reference RV32GCV 1 Loop+SLP
45628479
( 0.3x )
176782
( 0.788 )
36156
( 1.0 )
4096 TFLM Reference RV32GCV 1 Loop+SLP
12762261
( Base )
224482
( Base )
36148
( Base )
0 muRISCV-NN Scalar RV32GC 1 -
5441657
( 2.3x )
207008
( 0.922 )
36156
( 1.0 )
128 muRISCV-NN Scalar RV32GCV 1 Loop+SLP
4987314
( 2.6x )
207008
( 0.922 )
36156
( 1.0 )
256 muRISCV-NN Scalar RV32GCV 1 Loop+SLP
4650042
( 2.7x )
207008
( 0.922 )
36156
( 1.0 )
512 muRISCV-NN Scalar RV32GCV 1 Loop+SLP
5430891
( 2.3x )
207008
( 0.922 )
36156
( 1.0 )
1024 muRISCV-NN Scalar RV32GCV 1 Loop+SLP
14405776
( 0.9x )
207008
( 0.922 )
36156
( 1.0 )
2048 muRISCV-NN Scalar RV32GCV 1 Loop+SLP
14412554
( 0.9x )
207008
( 0.922 )
36156
( 1.0 )
4096 muRISCV-NN Scalar RV32GCV 1 Loop+SLP
4134844
( 3.1x )
217420
( 0.969 )
36148
( 1.0 )
128 muRISCV-NN Vector RV32GCV 1 -
2874138
( 4.4x )
217420
( 0.969 )
36148
( 1.0 )
256 muRISCV-NN Vector RV32GCV 1 -
2193828
( 5.8x )
217420
( 0.969 )
36148
( 1.0 )
512 muRISCV-NN Vector RV32GCV 1 -
2151344
( 5.9x )
217420
( 0.969 )
36148
( 1.0 )
1024 muRISCV-NN Vector RV32GCV 1 -
2154733
( 5.9x )
217420
( 0.969 )
36148
( 1.0 )
2048 muRISCV-NN Vector RV32GCV 1 -
2158122
( 5.9x )
217420
( 0.969 )
36148
( 1.0 )
4096 muRISCV-NN Vector RV32GCV 1 -

Image Classification (resnet)

Cycles (Speedup) Total ROM (rel.) Total RAM (rel.) VLEN Kernels Mode Arch Unroll Auto-Vectorization
103172599
( 0.5x )
230296
( 0.905 )
68916
( 1.0 )
0 TFLM Reference RV32GC 1 -
52726655
( 0.9x )
219378
( 0.862 )
68924
( 1.0 )
128 TFLM Reference RV32GCV 1 Loop+SLP
45065220
( 1.0x )
219378
( 0.862 )
68924
( 1.0 )
256 TFLM Reference RV32GCV 1 Loop+SLP
89148488
( 0.5x )
219378
( 0.862 )
68924
( 1.0 )
512 TFLM Reference RV32GCV 1 Loop+SLP
117445975
( 0.4x )
219378
( 0.862 )
68924
( 1.0 )
1024 TFLM Reference RV32GCV 1 Loop+SLP
134767789
( 0.3x )
219378
( 0.862 )
68924
( 1.0 )
2048 TFLM Reference RV32GCV 1 Loop+SLP
134777956
( 0.3x )
219378
( 0.862 )
68924
( 1.0 )
4096 TFLM Reference RV32GCV 1 Loop+SLP
47066297
( Base )
254392
( Base )
68912
( Base )
0 muRISCV-NN Scalar RV32GC 1 -
11915160
( 4.0x )
239236
( 0.94 )
68920
( 1.0 )
128 muRISCV-NN Scalar RV32GCV 1 Loop+SLP
11575114
( 4.1x )
239236
( 0.94 )
68920
( 1.0 )
256 muRISCV-NN Scalar RV32GCV 1 Loop+SLP
14598011
( 3.2x )
239236
( 0.94 )
68920
( 1.0 )
512 muRISCV-NN Scalar RV32GCV 1 Loop+SLP
16166934
( 2.9x )
239236
( 0.94 )
68920
( 1.0 )
1024 muRISCV-NN Scalar RV32GCV 1 Loop+SLP
16870635
( 2.8x )
239236
( 0.94 )
68920
( 1.0 )
2048 muRISCV-NN Scalar RV32GCV 1 Loop+SLP
36764323
( 1.3x )
239236
( 0.94 )
68920
( 1.0 )
4096 muRISCV-NN Scalar RV32GCV 1 Loop+SLP
15308395
( 3.1x )
248474
( 0.977 )
68912
( 1.0 )
128 muRISCV-NN Vector RV32GCV 1 -
9723407
( 4.8x )
248474
( 0.977 )
68912
( 1.0 )
256 muRISCV-NN Vector RV32GCV 1 -
7176673
( 6.6x )
248474
( 0.977 )
68912
( 1.0 )
512 muRISCV-NN Vector RV32GCV 1 -
5934097
( 7.9x )
248474
( 0.977 )
68912
( 1.0 )
1024 muRISCV-NN Vector RV32GCV 1 -
5008998
( 9.4x )
248474
( 0.977 )
68912
( 1.0 )
2048 muRISCV-NN Vector RV32GCV 1 -
4763183
( 9.9x )
248474
( 0.977 )
68912
( 1.0 )
4096 muRISCV-NN Vector RV32GCV 1 -

Anomaly Detection (toycar)

Cycles (Speedup) Total ROM (rel.) Total RAM (rel.) VLEN Kernels Mode Arch Unroll Auto-Vectorization
2172235
( 0.6x )
354834
( 0.972 )
19384
( 1.0 )
0 TFLM Reference RV32GC 1 -
804320
( 1.7x )
348234
( 0.954 )
19384
( 1.0 )
128 TFLM Reference RV32GCV 1 Loop+SLP
631456
( 2.1x )
348234
( 0.954 )
19384
( 1.0 )
256 TFLM Reference RV32GCV 1 Loop+SLP
540992
( 2.5x )
348234
( 0.954 )
19384
( 1.0 )
512 TFLM Reference RV32GCV 1 Loop+SLP
495760
( 2.7x )
348234
( 0.954 )
19384
( 1.0 )
1024 TFLM Reference RV32GCV 1 Loop+SLP
473144
( 2.9x )
348234
( 0.954 )
19384
( 1.0 )
2048 TFLM Reference RV32GCV 1 Loop+SLP
2421552
( 0.6x )
348234
( 0.954 )
19384
( 1.0 )
4096 TFLM Reference RV32GCV 1 Loop+SLP
1355958
( Base )
365130
( Base )
19384
( Base )
0 muRISCV-NN Scalar RV32GC 1 -
552375
( 2.5x )
356532
( 0.976 )
19384
( 1.0 )
128 muRISCV-NN Scalar RV32GCV 1 Loop+SLP
465699
( 2.9x )
356532
( 0.976 )
19384
( 1.0 )
256 muRISCV-NN Scalar RV32GCV 1 Loop+SLP
420483
( 3.2x )
356532
( 0.976 )
19384
( 1.0 )
512 muRISCV-NN Scalar RV32GCV 1 Loop+SLP
397875
( 3.4x )
356532
( 0.976 )
19384
( 1.0 )
1024 muRISCV-NN Scalar RV32GCV 1 Loop+SLP
386571
( 3.5x )
356532
( 0.976 )
19384
( 1.0 )
2048 muRISCV-NN Scalar RV32GCV 1 Loop+SLP
1368879
( 1.0x )
356532
( 0.976 )
19384
( 1.0 )
4096 muRISCV-NN Scalar RV32GCV 1 Loop+SLP
587274
( 2.3x )
360912
( 0.988 )
19384
( 1.0 )
128 muRISCV-NN Vector RV32GCV 1 -
470594
( 2.9x )
360912
( 0.988 )
19384
( 1.0 )
256 muRISCV-NN Vector RV32GCV 1 -
412254
( 3.3x )
360912
( 0.988 )
19384
( 1.0 )
512 muRISCV-NN Vector RV32GCV 1 -
383408
( 3.5x )
360912
( 0.988 )
19384
( 1.0 )
1024 muRISCV-NN Vector RV32GCV 1 -
379777
( 3.6x )
360912
( 0.988 )
19384
( 1.0 )
2048 muRISCV-NN Vector RV32GCV 1 -
377921
( 3.6x )
360912
( 0.988 )
19384
( 1.0 )
4096 muRISCV-NN Vector RV32GCV 1 -

Visual Wake Words (vww)

Cycles (Speedup) Total ROM (rel.) Total RAM (rel.) VLEN Kernels Mode Arch Unroll Auto-Vectorization
108773568
( 0.4x )
459112
( 0.921 )
134476
( 1.0 )
0 TFLM Reference RV32GC 1 -
72681675
( 0.5x )
450472
( 0.903 )
134484
( 1.0 )
128 TFLM Reference RV32GCV 1 Loop+SLP
70405415
( 0.5x )
450472
( 0.903 )
134484
( 1.0 )
256 TFLM Reference RV32GCV 1 Loop+SLP
70819959
( 0.5x )
450472
( 0.903 )
134484
( 1.0 )
512 TFLM Reference RV32GCV 1 Loop+SLP
77504860
( 0.5x )
450472
( 0.903 )
134484
( 1.0 )
1024 TFLM Reference RV32GCV 1 Loop+SLP
84902960
( 0.4x )
450472
( 0.903 )
134484
( 1.0 )
2048 TFLM Reference RV32GCV 1 Loop+SLP
113653201
( 0.3x )
450472
( 0.903 )
134484
( 1.0 )
4096 TFLM Reference RV32GCV 1 Loop+SLP
38161972
( Base )
498608
( Base )
134476
( Base )
0 muRISCV-NN Scalar RV32GC 1 -
17726201
( 2.2x )
480700
( 0.964 )
134484
( 1.0 )
128 muRISCV-NN Scalar RV32GCV 1 Loop+SLP
17260819
( 2.2x )
480700
( 0.964 )
134484
( 1.0 )
256 muRISCV-NN Scalar RV32GCV 1 Loop+SLP
18508291
( 2.1x )
480700
( 0.964 )
134484
( 1.0 )
512 muRISCV-NN Scalar RV32GCV 1 Loop+SLP
21845416
( 1.7x )
480700
( 0.964 )
134484
( 1.0 )
1024 muRISCV-NN Scalar RV32GCV 1 Loop+SLP
25581497
( 1.5x )
480700
( 0.964 )
134484
( 1.0 )
2048 muRISCV-NN Scalar RV32GCV 1 Loop+SLP
40407453
( 0.9x )
480700
( 0.964 )
134484
( 1.0 )
4096 muRISCV-NN Scalar RV32GCV 1 Loop+SLP
13538800
( 2.8x )
491546
( 0.986 )
134476
( 1.0 )
128 muRISCV-NN Vector RV32GCV 1 -
10206184
( 3.7x )
491546
( 0.986 )
134476
( 1.0 )
256 muRISCV-NN Vector RV32GCV 1 -
8909812
( 4.3x )
491546
( 0.986 )
134476
( 1.0 )
512 muRISCV-NN Vector RV32GCV 1 -
8404302
( 4.5x )
491546
( 0.986 )
134476
( 1.0 )
1024 muRISCV-NN Vector RV32GCV 1 -
8359304
( 4.6x )
491546
( 0.986 )
134476
( 1.0 )
2048 muRISCV-NN Vector RV32GCV 1 -
8362693
( 4.6x )
491546
( 0.986 )
134476
( 1.0 )
4096 muRISCV-NN Vector RV32GCV 1 -

Original data

Click here to download the raw files for this benchmark.

2024-11-26
2024-11-21
2024-11-19
2024-11-18
2024-07-12
2024-06-29
2024-03-02
2024-02-26
2024-02-23
2024-02-22
2024-02-20
2024-02-11
2023-12-22
Clone this wiki locally