Benchmarks 2024 11 21 TFLM LLVM Os spike_rv32

Setup

Simulator

• Spike (riscv-isa-sim ) (ISS, CPI=1)
  • Spike : 0bc176b3fca43560b9e8586cdbc41cfde073e17a
  • Spike PK : 7e9b671c0415dfd7b562ac934feb9380075d4aa2

Toolchains

• RISC-V GCC:

  • Scalar: riscv32-unknown-elf-gcc (gc891d8dc23e) 13.2.0
  • Vector: riscv32-unknown-elf-gcc (gc891d8dc23e) 13.2.0
  • Packed: Self compiled using patches found in https://github.com/riscv-collab/riscv-gcc/pull/258 and https://github.com/riscvarchive/riscv-binutils-gdb/pull/257

• LLVM/Clang:

  • clang version 18.1.8 (https://github.com/llvm/llvm-project.git 3b5b5c1ec4a3095ab096dd780e84d7ab81f3d7ff)
  • Linker: lld (TODO)

Models

• MLPerfTiny Benchmark

• TODO: others!

Frameworks

• MLonMCU : develop

• TFLM : 8eb6b23de4470d6a8da3131650d6a67514dfa130

Miscellaneous

• Used -Os flag for compilation.
• Benchmarks generated using MLonMCU deployment tool with minimal efforts.
• Memory metrics are reported in Bytes

Results (Framework: tflm, Backend: tflmi, Toolchain: llvm, Flags: -Os, Target: spike_rv32 )

Audio Wake Words (aww)

Cycles (Speedup)	Total ROM (rel.)	Total RAM (rel.)	VLEN	Kernels	Mode	Arch	Unroll	Auto-Vectorization
39294217 ( 0.4x )	146784 ( 0.86 )	36060 ( 1.0 )	0	TFLM	Reference	RV32GC	0	-
33486320 ( 0.5x )	152822 ( 0.895 )	36068 ( 1.0 )	128	TFLM	Reference	RV32GCV	0	Loop+SLP
31693648 ( 0.5x )	152822 ( 0.895 )	36068 ( 1.0 )	256	TFLM	Reference	RV32GCV	0	Loop+SLP
30797312 ( 0.5x )	152822 ( 0.895 )	36068 ( 1.0 )	512	TFLM	Reference	RV32GCV	0	Loop+SLP
30349144 ( 0.5x )	152822 ( 0.895 )	36068 ( 1.0 )	1024	TFLM	Reference	RV32GCV	0	Loop+SLP
30352533 ( 0.5x )	152822 ( 0.895 )	36068 ( 1.0 )	2048	TFLM	Reference	RV32GCV	0	Loop+SLP
30352533 ( 0.5x )	152822 ( 0.895 )	36068 ( 1.0 )	4096	TFLM	Reference	RV32GCV	0	Loop+SLP
15086527 ( Base )	170734 ( Base )	36060 ( Base )	0	muRISCV-NN	Scalar	RV32GC	0	-
14946565 ( 1.0x )	169880 ( 0.995 )	36060 ( 1.0 )	0	muRISCV-NN	Vector (Portable)	RV32GC	0	-
6142945 ( 2.5x )	181802 ( 1.065 )	36068 ( 1.0 )	128	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
5140254 ( 2.9x )	181802 ( 1.065 )	36068 ( 1.0 )	256	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
4639006 ( 3.3x )	181802 ( 1.065 )	36068 ( 1.0 )	512	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
4351675 ( 3.5x )	181802 ( 1.065 )	36068 ( 1.0 )	1024	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
4351675 ( 3.5x )	181802 ( 1.065 )	36068 ( 1.0 )	2048	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
4355064 ( 3.5x )	181802 ( 1.065 )	36068 ( 1.0 )	4096	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
4077937 ( 3.7x )	171946 ( 1.007 )	36060 ( 1.0 )	128	muRISCV-NN	Vector	RV32GCV	0	-
2822959 ( 5.3x )	171946 ( 1.007 )	36060 ( 1.0 )	256	muRISCV-NN	Vector	RV32GCV	0	-
2145641 ( 7.0x )	171946 ( 1.007 )	36060 ( 1.0 )	512	muRISCV-NN	Vector	RV32GCV	0	-
2104653 ( 7.2x )	171946 ( 1.007 )	36060 ( 1.0 )	1024	muRISCV-NN	Vector	RV32GCV	0	-
2108042 ( 7.2x )	171946 ( 1.007 )	36060 ( 1.0 )	2048	muRISCV-NN	Vector	RV32GCV	0	-
2111431 ( 7.1x )	171946 ( 1.007 )	36060 ( 1.0 )	4096	muRISCV-NN	Vector	RV32GCV	0	-
6890043 ( 2.2x )	180626 ( 1.058 )	36068 ( 1.0 )	128	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP
5922632 ( 2.5x )	180626 ( 1.058 )	36068 ( 1.0 )	256	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP
5439024 ( 2.8x )	180626 ( 1.058 )	36068 ( 1.0 )	512	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP
5169441 ( 2.9x )	180626 ( 1.058 )	36068 ( 1.0 )	1024	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP
5169441 ( 2.9x )	180626 ( 1.058 )	36068 ( 1.0 )	2048	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP
5172830 ( 2.9x )	180626 ( 1.058 )	36068 ( 1.0 )	4096	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP

Image Classification (resnet)

Cycles (Speedup)	Total ROM (rel.)	Total RAM (rel.)	VLEN	Kernels	Mode	Arch	Unroll	Auto-Vectorization
121052840 ( 0.5x )	188506 ( 0.921 )	68828 ( 1.0 )	0	TFLM	Reference	RV32GC	0	-
56885482 ( 1.0x )	194494 ( 0.951 )	68836 ( 1.0 )	128	TFLM	Reference	RV32GCV	0	Loop+SLP
47128826 ( 1.2x )	194494 ( 0.951 )	68836 ( 1.0 )	256	TFLM	Reference	RV32GCV	0	Loop+SLP
44781922 ( 1.3x )	194494 ( 0.951 )	68836 ( 1.0 )	512	TFLM	Reference	RV32GCV	0	Loop+SLP
44348118 ( 1.3x )	194494 ( 0.951 )	68836 ( 1.0 )	1024	TFLM	Reference	RV32GCV	0	Loop+SLP
44351507 ( 1.3x )	194494 ( 0.951 )	68836 ( 1.0 )	2048	TFLM	Reference	RV32GCV	0	Loop+SLP
44351507 ( 1.3x )	194494 ( 0.951 )	68836 ( 1.0 )	4096	TFLM	Reference	RV32GCV	0	Loop+SLP
56361734 ( Base )	204622 ( Base )	68824 ( Base )	0	muRISCV-NN	Scalar	RV32GC	0	-
72373035 ( 0.8x )	203976 ( 0.997 )	68824 ( 1.0 )	0	muRISCV-NN	Vector (Portable)	RV32GC	0	-
26153796 ( 2.2x )	215920 ( 1.055 )	68832 ( 1.0 )	128	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
18289009 ( 3.1x )	215920 ( 1.055 )	68832 ( 1.0 )	256	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
14602473 ( 3.9x )	215920 ( 1.055 )	68832 ( 1.0 )	512	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
12926434 ( 4.4x )	215920 ( 1.055 )	68832 ( 1.0 )	1024	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
12107234 ( 4.7x )	215920 ( 1.055 )	68832 ( 1.0 )	2048	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
11496223 ( 4.9x )	215920 ( 1.055 )	68832 ( 1.0 )	4096	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
15252556 ( 3.7x )	206548 ( 1.009 )	68824 ( 1.0 )	128	muRISCV-NN	Vector	RV32GCV	0	-
9669360 ( 5.8x )	206548 ( 1.009 )	68824 ( 1.0 )	256	muRISCV-NN	Vector	RV32GCV	0	-
7123522 ( 7.9x )	206548 ( 1.009 )	68824 ( 1.0 )	512	muRISCV-NN	Vector	RV32GCV	0	-
5881394 ( 9.6x )	206548 ( 1.009 )	68824 ( 1.0 )	1024	muRISCV-NN	Vector	RV32GCV	0	-
4956519 ( 11.4x )	206548 ( 1.009 )	68824 ( 1.0 )	2048	muRISCV-NN	Vector	RV32GCV	0	-
4710816 ( 12.0x )	206548 ( 1.009 )	68824 ( 1.0 )	4096	muRISCV-NN	Vector	RV32GCV	0	-
20413187 ( 2.8x )	214644 ( 1.049 )	68832 ( 1.0 )	128	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP
14317512 ( 3.9x )	214644 ( 1.049 )	68832 ( 1.0 )	256	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP
11460236 ( 4.9x )	214644 ( 1.049 )	68832 ( 1.0 )	512	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP
10161963 ( 5.5x )	214644 ( 1.049 )	68832 ( 1.0 )	1024	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP
9527083 ( 5.9x )	214644 ( 1.049 )	68832 ( 1.0 )	2048	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP
9054312 ( 6.2x )	214644 ( 1.049 )	68832 ( 1.0 )	4096	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP

Anomaly Detection (toycar)

Cycles (Speedup)	Total ROM (rel.)	Total RAM (rel.)	VLEN	Kernels	Mode	Arch	Unroll	Auto-Vectorization
2797025 ( 0.6x )	342072 ( 0.983 )	19364 ( 1.0 )	0	TFLM	Reference	RV32GC	0	-
898484 ( 1.9x )	344098 ( 0.989 )	19364 ( 1.0 )	128	TFLM	Reference	RV32GCV	0	Loop+SLP
668212 ( 2.5x )	344098 ( 0.989 )	19364 ( 1.0 )	256	TFLM	Reference	RV32GCV	0	Loop+SLP
553076 ( 3.0x )	344098 ( 0.989 )	19364 ( 1.0 )	512	TFLM	Reference	RV32GCV	0	Loop+SLP
495508 ( 3.4x )	344098 ( 0.989 )	19364 ( 1.0 )	1024	TFLM	Reference	RV32GCV	0	Loop+SLP
466724 ( 3.6x )	344098 ( 0.989 )	19364 ( 1.0 )	2048	TFLM	Reference	RV32GCV	0	Loop+SLP
466529 ( 3.6x )	344098 ( 0.989 )	19364 ( 1.0 )	4096	TFLM	Reference	RV32GCV	0	Loop+SLP
1683764 ( Base )	347942 ( Base )	19364 ( Base )	0	muRISCV-NN	Scalar	RV32GC	0	-
3036470 ( 0.6x )	347944 ( 1.0 )	19364 ( 1.0 )	0	muRISCV-NN	Vector (Portable)	RV32GC	0	-
594867 ( 2.8x )	352042 ( 1.012 )	19364 ( 1.0 )	128	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
480947 ( 3.5x )	352042 ( 1.012 )	19364 ( 1.0 )	256	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
423987 ( 4.0x )	352042 ( 1.012 )	19364 ( 1.0 )	512	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
395507 ( 4.3x )	352042 ( 1.012 )	19364 ( 1.0 )	1024	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
381267 ( 4.4x )	352042 ( 1.012 )	19364 ( 1.0 )	2048	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
379483 ( 4.4x )	352042 ( 1.012 )	19364 ( 1.0 )	4096	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
1940438 ( 0.9x )	349462 ( 1.004 )	19364 ( 1.0 )	128	muRISCV-NN	Vector	RV32GCV	0	-
1823758 ( 0.9x )	349462 ( 1.004 )	19364 ( 1.0 )	256	muRISCV-NN	Vector	RV32GCV	0	-
1765418 ( 1.0x )	349462 ( 1.004 )	19364 ( 1.0 )	512	muRISCV-NN	Vector	RV32GCV	0	-
1736572 ( 1.0x )	349462 ( 1.004 )	19364 ( 1.0 )	1024	muRISCV-NN	Vector	RV32GCV	0	-
1732941 ( 1.0x )	349462 ( 1.004 )	19364 ( 1.0 )	2048	muRISCV-NN	Vector	RV32GCV	0	-
1731085 ( 1.0x )	349462 ( 1.004 )	19364 ( 1.0 )	4096	muRISCV-NN	Vector	RV32GCV	0	-
928878 ( 1.8x )	352044 ( 1.012 )	19364 ( 1.0 )	128	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP
666926 ( 2.5x )	352044 ( 1.012 )	19364 ( 1.0 )	256	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP
535950 ( 3.1x )	352044 ( 1.012 )	19364 ( 1.0 )	512	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP
470462 ( 3.6x )	352044 ( 1.012 )	19364 ( 1.0 )	1024	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP
437718 ( 3.8x )	352044 ( 1.012 )	19364 ( 1.0 )	2048	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP
433630 ( 3.9x )	352044 ( 1.012 )	19364 ( 1.0 )	4096	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP

Visual Wake Words (vww)

Cycles (Speedup)	Total ROM (rel.)	Total RAM (rel.)	VLEN	Kernels	Mode	Arch	Unroll	Auto-Vectorization
103593216 ( 0.4x )	420426 ( 0.946 )	134364 ( 1.0 )	0	TFLM	Reference	RV32GC	0	-
72363231 ( 0.6x )	426464 ( 0.96 )	134372 ( 1.0 )	128	TFLM	Reference	RV32GCV	0	Loop+SLP
67201823 ( 0.7x )	426464 ( 0.96 )	134372 ( 1.0 )	256	TFLM	Reference	RV32GCV	0	Loop+SLP
64750143 ( 0.7x )	426464 ( 0.96 )	134372 ( 1.0 )	512	TFLM	Reference	RV32GCV	0	Loop+SLP
63717839 ( 0.7x )	426464 ( 0.96 )	134372 ( 1.0 )	1024	TFLM	Reference	RV32GCV	0	Loop+SLP
63301844 ( 0.7x )	426464 ( 0.96 )	134372 ( 1.0 )	2048	TFLM	Reference	RV32GCV	0	Loop+SLP
63269560 ( 0.7x )	426464 ( 0.96 )	134372 ( 1.0 )	4096	TFLM	Reference	RV32GCV	0	Loop+SLP
45186114 ( Base )	444376 ( Base )	134364 ( Base )	0	muRISCV-NN	Scalar	RV32GC	0	-
44936981 ( 1.0x )	443522 ( 0.998 )	134364 ( 1.0 )	0	muRISCV-NN	Vector (Portable)	RV32GC	0	-
19572597 ( 2.3x )	455444 ( 1.025 )	134372 ( 1.0 )	128	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
16706197 ( 2.7x )	455444 ( 1.025 )	134372 ( 1.0 )	256	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
15333013 ( 2.9x )	455444 ( 1.025 )	134372 ( 1.0 )	512	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
14832162 ( 3.0x )	455444 ( 1.025 )	134372 ( 1.0 )	1024	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
14625642 ( 3.1x )	455444 ( 1.025 )	134372 ( 1.0 )	2048	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
14611595 ( 3.1x )	455444 ( 1.025 )	134372 ( 1.0 )	4096	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
13372108 ( 3.4x )	445588 ( 1.003 )	134364 ( 1.0 )	128	muRISCV-NN	Vector	RV32GCV	0	-
10051108 ( 4.5x )	445588 ( 1.003 )	134364 ( 1.0 )	256	muRISCV-NN	Vector	RV32GCV	0	-
8759680 ( 5.2x )	445588 ( 1.003 )	134364 ( 1.0 )	512	muRISCV-NN	Vector	RV32GCV	0	-
8255690 ( 5.5x )	445588 ( 1.003 )	134364 ( 1.0 )	1024	muRISCV-NN	Vector	RV32GCV	0	-
8211310 ( 5.5x )	445588 ( 1.003 )	134364 ( 1.0 )	2048	muRISCV-NN	Vector	RV32GCV	0	-
8214699 ( 5.5x )	445588 ( 1.003 )	134364 ( 1.0 )	4096	muRISCV-NN	Vector	RV32GCV	0	-
22412675 ( 2.0x )	454268 ( 1.022 )	134372 ( 1.0 )	128	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP
19628931 ( 2.3x )	454268 ( 1.022 )	134372 ( 1.0 )	256	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP
18297075 ( 2.5x )	454268 ( 1.022 )	134372 ( 1.0 )	512	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP
17796152 ( 2.5x )	454268 ( 1.022 )	134372 ( 1.0 )	1024	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP
17589596 ( 2.6x )	454268 ( 1.022 )	134372 ( 1.0 )	2048	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP
17575531 ( 2.6x )	454268 ( 1.022 )	134372 ( 1.0 )	4096	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP

Original data

Click here to download the raw files for this benchmark.