Benchmarks 2024 11 21 TFLM LLVM Os spike_rv64

Setup

Simulator

• Spike (riscv-isa-sim ) (ISS, CPI=1)
  • Spike : 0bc176b3fca43560b9e8586cdbc41cfde073e17a
  • Spike PK : 7e9b671c0415dfd7b562ac934feb9380075d4aa2

Toolchains

• RISC-V GCC:

  • Scalar: riscv32-unknown-elf-gcc (gc891d8dc23e) 13.2.0
  • Vector: riscv32-unknown-elf-gcc (gc891d8dc23e) 13.2.0
  • Packed: Self compiled using patches found in https://github.com/riscv-collab/riscv-gcc/pull/258 and https://github.com/riscvarchive/riscv-binutils-gdb/pull/257

• LLVM/Clang:

  • clang version 18.1.8 (https://github.com/llvm/llvm-project.git 3b5b5c1ec4a3095ab096dd780e84d7ab81f3d7ff)
  • Linker: lld (TODO)

Models

• MLPerfTiny Benchmark

• TODO: others!

Frameworks

• MLonMCU : develop

• TFLM : 8eb6b23de4470d6a8da3131650d6a67514dfa130

Miscellaneous

• Used -Os flag for compilation.
• Benchmarks generated using MLonMCU deployment tool with minimal efforts.
• Memory metrics are reported in Bytes

Results (Framework: tflm, Backend: tflmi, Toolchain: llvm, Flags: -Os, Target: spike_rv64 )

Audio Wake Words (aww)

Cycles (Speedup)	Total ROM (rel.)	Total RAM (rel.)	VLEN	Kernels	Mode	Arch	Unroll	Auto-Vectorization
39715359 ( 0.4x )	143524 ( 0.865 )	38304 ( 1.0 )	0	TFLM	Reference	RV64GC	0	-
31996087 ( 0.5x )	148650 ( 0.896 )	38312 ( 1.0 )	128	TFLM	Reference	RV64GCV	0	Loop+SLP
30075319 ( 0.5x )	148650 ( 0.896 )	38312 ( 1.0 )	256	TFLM	Reference	RV64GCV	0	Loop+SLP
29114935 ( 0.5x )	148650 ( 0.896 )	38312 ( 1.0 )	512	TFLM	Reference	RV64GCV	0	Loop+SLP
28636608 ( 0.5x )	148650 ( 0.896 )	38312 ( 1.0 )	1024	TFLM	Reference	RV64GCV	0	Loop+SLP
28636608 ( 0.5x )	148650 ( 0.896 )	38312 ( 1.0 )	2048	TFLM	Reference	RV64GCV	0	Loop+SLP
28638473 ( 0.5x )	148650 ( 0.896 )	38312 ( 1.0 )	4096	TFLM	Reference	RV64GCV	0	Loop+SLP
14702893 ( Base )	165840 ( Base )	38304 ( Base )	0	muRISCV-NN	Scalar	RV64GC	0	-
15064322 ( 1.0x )	165092 ( 0.995 )	38304 ( 1.0 )	0	muRISCV-NN	Vector (Portable)	RV64GC	0	-
5820167 ( 2.5x )	173674 ( 1.047 )	38312 ( 1.0 )	128	muRISCV-NN	Scalar	RV64GCV	0	Loop+SLP
4782018 ( 3.1x )	173674 ( 1.047 )	38312 ( 1.0 )	256	muRISCV-NN	Scalar	RV64GCV	0	Loop+SLP
4264931 ( 3.4x )	173674 ( 1.047 )	38312 ( 1.0 )	512	muRISCV-NN	Scalar	RV64GCV	0	Loop+SLP
3964831 ( 3.7x )	173674 ( 1.047 )	38312 ( 1.0 )	1024	muRISCV-NN	Scalar	RV64GCV	0	Loop+SLP
3964831 ( 3.7x )	173674 ( 1.047 )	38312 ( 1.0 )	2048	muRISCV-NN	Scalar	RV64GCV	0	Loop+SLP
3968561 ( 3.7x )	173674 ( 1.047 )	38312 ( 1.0 )	4096	muRISCV-NN	Scalar	RV64GCV	0	Loop+SLP
4074863 ( 3.6x )	167314 ( 1.009 )	38304 ( 1.0 )	128	muRISCV-NN	Vector	RV64GCV	0	-
2819847 ( 5.2x )	167314 ( 1.009 )	38304 ( 1.0 )	256	muRISCV-NN	Vector	RV64GCV	0	-
2142511 ( 6.9x )	167314 ( 1.009 )	38304 ( 1.0 )	512	muRISCV-NN	Vector	RV64GCV	0	-
2103380 ( 7.0x )	167314 ( 1.009 )	38304 ( 1.0 )	1024	muRISCV-NN	Vector	RV64GCV	0	-
2103380 ( 7.0x )	167314 ( 1.009 )	38304 ( 1.0 )	2048	muRISCV-NN	Vector	RV64GCV	0	-
2105245 ( 7.0x )	167314 ( 1.009 )	38304 ( 1.0 )	4096	muRISCV-NN	Vector	RV64GCV	0	-
6891442 ( 2.1x )	172540 ( 1.04 )	38312 ( 1.0 )	128	muRISCV-NN	Vector (Portable)	RV64GCV	0	Loop+SLP
5886365 ( 2.5x )	172540 ( 1.04 )	38312 ( 1.0 )	256	muRISCV-NN	Vector (Portable)	RV64GCV	0	Loop+SLP
5383949 ( 2.7x )	172540 ( 1.04 )	38312 ( 1.0 )	512	muRISCV-NN	Vector (Portable)	RV64GCV	0	Loop+SLP
5102382 ( 2.9x )	172540 ( 1.04 )	38312 ( 1.0 )	1024	muRISCV-NN	Vector (Portable)	RV64GCV	0	Loop+SLP
5102382 ( 2.9x )	172540 ( 1.04 )	38312 ( 1.0 )	2048	muRISCV-NN	Vector (Portable)	RV64GCV	0	Loop+SLP
5106112 ( 2.9x )	172540 ( 1.04 )	38312 ( 1.0 )	4096	muRISCV-NN	Vector (Portable)	RV64GCV	0	Loop+SLP

Image Classification (resnet)

Cycles (Speedup)	Total ROM (rel.)	Total RAM (rel.)	VLEN	Kernels	Mode	Arch	Unroll	Auto-Vectorization
121111112 ( 0.5x )	184936 ( 0.927 )	71072 ( 1.0 )	0	TFLM	Reference	RV64GC	0	-
57755288 ( 1.0x )	190998 ( 0.957 )	71080 ( 1.0 )	128	TFLM	Reference	RV64GCV	0	Loop+SLP
47301688 ( 1.2x )	190998 ( 0.957 )	71080 ( 1.0 )	256	TFLM	Reference	RV64GCV	0	Loop+SLP
44787128 ( 1.2x )	190998 ( 0.957 )	71080 ( 1.0 )	512	TFLM	Reference	RV64GCV	0	Loop+SLP
44324193 ( 1.3x )	190998 ( 0.957 )	71080 ( 1.0 )	1024	TFLM	Reference	RV64GCV	0	Loop+SLP
44324193 ( 1.3x )	190998 ( 0.957 )	71080 ( 1.0 )	2048	TFLM	Reference	RV64GCV	0	Loop+SLP
44326058 ( 1.3x )	190998 ( 0.957 )	71080 ( 1.0 )	4096	TFLM	Reference	RV64GCV	0	Loop+SLP
55734439 ( Base )	199526 ( Base )	71072 ( Base )	0	muRISCV-NN	Scalar	RV64GC	0	-
71528934 ( 0.8x )	198988 ( 0.997 )	71072 ( 1.0 )	0	muRISCV-NN	Vector (Portable)	RV64GC	0	-
25734308 ( 2.2x )	208438 ( 1.045 )	71080 ( 1.0 )	128	muRISCV-NN	Scalar	RV64GCV	0	Loop+SLP
17771159 ( 3.1x )	208438 ( 1.045 )	71080 ( 1.0 )	256	muRISCV-NN	Scalar	RV64GCV	0	Loop+SLP
14040404 ( 4.0x )	208438 ( 1.045 )	71080 ( 1.0 )	512	muRISCV-NN	Scalar	RV64GCV	0	Loop+SLP
12339982 ( 4.5x )	208438 ( 1.045 )	71080 ( 1.0 )	1024	muRISCV-NN	Scalar	RV64GCV	0	Loop+SLP
11510542 ( 4.8x )	208438 ( 1.045 )	71080 ( 1.0 )	2048	muRISCV-NN	Scalar	RV64GCV	0	Loop+SLP
10892192 ( 5.1x )	208438 ( 1.045 )	71080 ( 1.0 )	4096	muRISCV-NN	Scalar	RV64GCV	0	Loop+SLP
15251347 ( 3.7x )	201744 ( 1.011 )	71072 ( 1.0 )	128	muRISCV-NN	Vector	RV64GCV	0	-
9668147 ( 5.8x )	201744 ( 1.011 )	71072 ( 1.0 )	256	muRISCV-NN	Vector	RV64GCV	0	-
7122307 ( 7.8x )	201744 ( 1.011 )	71072 ( 1.0 )	512	muRISCV-NN	Vector	RV64GCV	0	-
5882044 ( 9.5x )	201744 ( 1.011 )	71072 ( 1.0 )	1024	muRISCV-NN	Vector	RV64GCV	0	-
4953780 ( 11.3x )	201744 ( 1.011 )	71072 ( 1.0 )	2048	muRISCV-NN	Vector	RV64GCV	0	-
4706553 ( 11.8x )	201744 ( 1.011 )	71072 ( 1.0 )	4096	muRISCV-NN	Vector	RV64GCV	0	-
19802584 ( 2.8x )	207216 ( 1.039 )	71080 ( 1.0 )	128	muRISCV-NN	Vector (Portable)	RV64GCV	0	Loop+SLP
13510163 ( 4.1x )	207216 ( 1.039 )	71080 ( 1.0 )	256	muRISCV-NN	Vector (Portable)	RV64GCV	0	Loop+SLP
10560683 ( 5.3x )	207216 ( 1.039 )	71080 ( 1.0 )	512	muRISCV-NN	Vector (Portable)	RV64GCV	0	Loop+SLP
9218880 ( 6.0x )	207216 ( 1.039 )	71080 ( 1.0 )	1024	muRISCV-NN	Vector (Portable)	RV64GCV	0	Loop+SLP
8563520 ( 6.5x )	207216 ( 1.039 )	71080 ( 1.0 )	2048	muRISCV-NN	Vector (Portable)	RV64GCV	0	Loop+SLP
8075730 ( 6.9x )	207216 ( 1.039 )	71080 ( 1.0 )	4096	muRISCV-NN	Vector (Portable)	RV64GCV	0	Loop+SLP

Anomaly Detection (toycar)

Cycles (Speedup)	Total ROM (rel.)	Total RAM (rel.)	VLEN	Kernels	Mode	Arch	Unroll	Auto-Vectorization
2782346 ( 0.6x )	338500 ( 0.983 )	21440 ( 1.0 )	0	TFLM	Reference	RV64GC	0	-
952434 ( 1.7x )	340860 ( 0.99 )	21440 ( 1.0 )	128	TFLM	Reference	RV64GCV	0	Loop+SLP
689266 ( 2.4x )	340860 ( 0.99 )	21440 ( 1.0 )	256	TFLM	Reference	RV64GCV	0	Loop+SLP
557682 ( 2.9x )	340860 ( 0.99 )	21440 ( 1.0 )	512	TFLM	Reference	RV64GCV	0	Loop+SLP
491890 ( 3.3x )	340860 ( 0.99 )	21440 ( 1.0 )	1024	TFLM	Reference	RV64GCV	0	Loop+SLP
458994 ( 3.5x )	340860 ( 0.99 )	21440 ( 1.0 )	2048	TFLM	Reference	RV64GCV	0	Loop+SLP
456763 ( 3.6x )	340860 ( 0.99 )	21440 ( 1.0 )	4096	TFLM	Reference	RV64GCV	0	Loop+SLP
1627210 ( Base )	344186 ( Base )	21440 ( Base )	0	muRISCV-NN	Scalar	RV64GC	0	-
2972834 ( 0.5x )	344190 ( 1.0 )	21440 ( 1.0 )	0	muRISCV-NN	Vector (Portable)	RV64GC	0	-
598831 ( 2.7x )	347900 ( 1.011 )	21440 ( 1.0 )	128	muRISCV-NN	Scalar	RV64GCV	0	Loop+SLP
481391 ( 3.4x )	347900 ( 1.011 )	21440 ( 1.0 )	256	muRISCV-NN	Scalar	RV64GCV	0	Loop+SLP
422671 ( 3.8x )	347900 ( 1.011 )	21440 ( 1.0 )	512	muRISCV-NN	Scalar	RV64GCV	0	Loop+SLP
393311 ( 4.1x )	347900 ( 1.011 )	21440 ( 1.0 )	1024	muRISCV-NN	Scalar	RV64GCV	0	Loop+SLP
378631 ( 4.3x )	347900 ( 1.011 )	21440 ( 1.0 )	2048	muRISCV-NN	Scalar	RV64GCV	0	Loop+SLP
376791 ( 4.3x )	347900 ( 1.011 )	21440 ( 1.0 )	4096	muRISCV-NN	Scalar	RV64GCV	0	Loop+SLP
1935500 ( 0.8x )	345820 ( 1.005 )	21440 ( 1.0 )	128	muRISCV-NN	Vector	RV64GCV	0	-
1817132 ( 0.9x )	345820 ( 1.005 )	21440 ( 1.0 )	256	muRISCV-NN	Vector	RV64GCV	0	-
1757948 ( 0.9x )	345820 ( 1.005 )	21440 ( 1.0 )	512	muRISCV-NN	Vector	RV64GCV	0	-
1728692 ( 0.9x )	345820 ( 1.005 )	21440 ( 1.0 )	1024	muRISCV-NN	Vector	RV64GCV	0	-
1725008 ( 0.9x )	345820 ( 1.005 )	21440 ( 1.0 )	2048	muRISCV-NN	Vector	RV64GCV	0	-
1723124 ( 0.9x )	345820 ( 1.005 )	21440 ( 1.0 )	4096	muRISCV-NN	Vector	RV64GCV	0	-
998546 ( 1.6x )	347904 ( 1.011 )	21440 ( 1.0 )	128	muRISCV-NN	Vector (Portable)	RV64GCV	0	Loop+SLP
700178 ( 2.3x )	347904 ( 1.011 )	21440 ( 1.0 )	256	muRISCV-NN	Vector (Portable)	RV64GCV	0	Loop+SLP
550994 ( 3.0x )	347904 ( 1.011 )	21440 ( 1.0 )	512	muRISCV-NN	Vector (Portable)	RV64GCV	0	Loop+SLP
476402 ( 3.4x )	347904 ( 1.011 )	21440 ( 1.0 )	1024	muRISCV-NN	Vector (Portable)	RV64GCV	0	Loop+SLP
439106 ( 3.7x )	347904 ( 1.011 )	21440 ( 1.0 )	2048	muRISCV-NN	Vector (Portable)	RV64GCV	0	Loop+SLP
434450 ( 3.7x )	347904 ( 1.011 )	21440 ( 1.0 )	4096	muRISCV-NN	Vector (Portable)	RV64GCV	0	Loop+SLP

Visual Wake Words (vww)

Cycles (Speedup)	Total ROM (rel.)	Total RAM (rel.)	VLEN	Kernels	Mode	Arch	Unroll	Auto-Vectorization
104295252 ( 0.4x )	417166 ( 0.949 )	136608 ( 1.0 )	0	TFLM	Reference	RV64GC	0	-
67369502 ( 0.7x )	422292 ( 0.961 )	136616 ( 1.0 )	128	TFLM	Reference	RV64GCV	0	Loop+SLP
61839390 ( 0.7x )	422292 ( 0.961 )	136616 ( 1.0 )	256	TFLM	Reference	RV64GCV	0	Loop+SLP
59212574 ( 0.7x )	422292 ( 0.961 )	136616 ( 1.0 )	512	TFLM	Reference	RV64GCV	0	Loop+SLP
58108391 ( 0.8x )	422292 ( 0.961 )	136616 ( 1.0 )	1024	TFLM	Reference	RV64GCV	0	Loop+SLP
57659047 ( 0.8x )	422292 ( 0.961 )	136616 ( 1.0 )	2048	TFLM	Reference	RV64GCV	0	Loop+SLP
57626320 ( 0.8x )	422292 ( 0.961 )	136616 ( 1.0 )	4096	TFLM	Reference	RV64GCV	0	Loop+SLP
43977423 ( Base )	439482 ( Base )	136608 ( Base )	0	muRISCV-NN	Scalar	RV64GC	0	-
44902157 ( 1.0x )	438734 ( 0.998 )	136608 ( 1.0 )	0	muRISCV-NN	Vector (Portable)	RV64GC	0	-
18369155 ( 2.4x )	447316 ( 1.018 )	136616 ( 1.0 )	128	muRISCV-NN	Scalar	RV64GCV	0	Loop+SLP
15392163 ( 2.9x )	447316 ( 1.018 )	136616 ( 1.0 )	256	muRISCV-NN	Scalar	RV64GCV	0	Loop+SLP
13967868 ( 3.1x )	447316 ( 1.018 )	136616 ( 1.0 )	512	muRISCV-NN	Scalar	RV64GCV	0	Loop+SLP
13441796 ( 3.3x )	447316 ( 1.018 )	136616 ( 1.0 )	1024	muRISCV-NN	Scalar	RV64GCV	0	Loop+SLP
13226184 ( 3.3x )	447316 ( 1.018 )	136616 ( 1.0 )	2048	muRISCV-NN	Scalar	RV64GCV	0	Loop+SLP
13211580 ( 3.3x )	447316 ( 1.018 )	136616 ( 1.0 )	4096	muRISCV-NN	Scalar	RV64GCV	0	Loop+SLP
13381911 ( 3.3x )	440956 ( 1.003 )	136608 ( 1.0 )	128	muRISCV-NN	Vector	RV64GCV	0	-
10060855 ( 4.4x )	440956 ( 1.003 )	136608 ( 1.0 )	256	muRISCV-NN	Vector	RV64GCV	0	-
8769399 ( 5.0x )	440956 ( 1.003 )	136608 ( 1.0 )	512	muRISCV-NN	Vector	RV64GCV	0	-
8267260 ( 5.3x )	440956 ( 1.003 )	136608 ( 1.0 )	1024	muRISCV-NN	Vector	RV64GCV	0	-
8219485 ( 5.4x )	440956 ( 1.003 )	136608 ( 1.0 )	2048	muRISCV-NN	Vector	RV64GCV	0	-
8221350 ( 5.3x )	440956 ( 1.003 )	136608 ( 1.0 )	4096	muRISCV-NN	Vector	RV64GCV	0	-
21892523 ( 2.0x )	446182 ( 1.015 )	136616 ( 1.0 )	128	muRISCV-NN	Vector (Portable)	RV64GCV	0	Loop+SLP
18993515 ( 2.3x )	446182 ( 1.015 )	136616 ( 1.0 )	256	muRISCV-NN	Vector (Portable)	RV64GCV	0	Loop+SLP
17606347 ( 2.5x )	446182 ( 1.015 )	136616 ( 1.0 )	512	muRISCV-NN	Vector (Portable)	RV64GCV	0	Loop+SLP
17082052 ( 2.6x )	446182 ( 1.015 )	136616 ( 1.0 )	1024	muRISCV-NN	Vector (Portable)	RV64GCV	0	Loop+SLP
16866396 ( 2.6x )	446182 ( 1.015 )	136616 ( 1.0 )	2048	muRISCV-NN	Vector (Portable)	RV64GCV	0	Loop+SLP
16851770 ( 2.6x )	446182 ( 1.015 )	136616 ( 1.0 )	4096	muRISCV-NN	Vector (Portable)	RV64GCV	0	Loop+SLP

Original data

Click here to download the raw files for this benchmark.