Benchmarks 2024 11 26 TFLM LLVM O3 spike_rv32

Setup

Simulator

• Spike (riscv-isa-sim ) (ISS, CPI=1)
  • Spike : eb0a3e2b0a7c57522928be39de95cd9f8c6dc636
  • Spike PK : fix-gcc14-rvv

Toolchains

• RISC-V GCC:

  • Scalar: riscv32-unknown-elf-gcc (g8b4bb54e6c4) 14.2.1 20241118
  • Vector: riscv32-unknown-elf-gcc (g8b4bb54e6c4) 14.2.1 20241118
  • Packed: Self compiled using patches found in https://github.com/riscv-collab/riscv-gcc/pull/258 and https://github.com/riscvarchive/riscv-binutils-gdb/pull/257

• LLVM/Clang:

  • clang version 18.1.8 (https://github.com/llvm/llvm-project.git 3b5b5c1ec4a3095ab096dd780e84d7ab81f3d7ff)
  • Linker: lld (TODO)

Models

• MLPerfTiny Benchmark

• TODO: others!

Frameworks

• MLonMCU : develop

• TFLM : 8eb6b23de4470d6a8da3131650d6a67514dfa130

Miscellaneous

• Used -Os flag for compilation.
• Benchmarks generated using MLonMCU deployment tool with minimal efforts.
• Memory metrics are reported in Bytes

Results (Framework: tflm, Backend: tflmi, Toolchain: llvm, Flags: -O3, Target: spike_rv32 )

Audio Wake Words (aww)

Cycles (Speedup)	Total ROM (rel.)	Total RAM (rel.)	VLEN	Kernels	Mode	Arch	Unroll	Auto-Vectorization
38583516.0 ( 0.4x )	159832 ( 0.893 )	36084 ( 1.0 )	0	TFLM	Reference	RV32GC	0	-
28766247.0 ( 0.5x )	175122 ( 0.978 )	36092 ( 1.0 )	128	TFLM	Reference	RV32GCV	0	Loop+SLP
27360974.0 ( 0.6x )	175378 ( 0.98 )	36092 ( 1.0 )	256	TFLM	Reference	RV32GCV	0	Loop+SLP
26656775.0 ( 0.6x )	175724 ( 0.981 )	36092 ( 1.0 )	512	TFLM	Reference	RV32GCV	0	Loop+SLP
26302901.0 ( 0.6x )	175982 ( 0.983 )	36092 ( 1.0 )	1024	TFLM	Reference	RV32GCV	0	Loop+SLP
42149645.0 ( 0.4x )	176202 ( 0.984 )	36092 ( 1.0 )	2048	TFLM	Reference	RV32GCV	0	Loop+SLP
42156427.0 ( 0.4x )	177162 ( 0.989 )	36092 ( 1.0 )	4096	TFLM	Reference	RV32GCV	0	Loop+SLP
15103433.0 ( Base )	179046 ( Base )	36084 ( Base )	0	muRISCV-NN	Scalar	RV32GC	0	-
14903933.0 ( 1.0x )	178834 ( 0.999 )	36084 ( 1.0 )	0	muRISCV-NN	Vector (Portable)	RV32GC	0	-
5347285.0 ( 2.8x )	197814 ( 1.105 )	36092 ( 1.0 )	128	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
4930949.0 ( 3.1x )	198138 ( 1.107 )	36092 ( 1.0 )	256	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
4612940.0 ( 3.3x )	198392 ( 1.108 )	36092 ( 1.0 )	512	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
5360735.0 ( 2.8x )	198540 ( 1.109 )	36092 ( 1.0 )	1024	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
13835200.0 ( 1.1x )	198650 ( 1.109 )	36092 ( 1.0 )	2048	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
13828301.0 ( 1.1x )	199218 ( 1.113 )	36092 ( 1.0 )	4096	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
4064607.0 ( 3.7x )	181424 ( 1.013 )	36084 ( 1.0 )	128	muRISCV-NN	Vector	RV32GCV	0	-
2809645.0 ( 5.4x )	181452 ( 1.013 )	36084 ( 1.0 )	256	muRISCV-NN	Vector	RV32GCV	0	-
2132335.0 ( 7.1x )	181416 ( 1.013 )	36084 ( 1.0 )	512	muRISCV-NN	Vector	RV32GCV	0	-
2091351.0 ( 7.2x )	181454 ( 1.013 )	36084 ( 1.0 )	1024	muRISCV-NN	Vector	RV32GCV	0	-
2094742.0 ( 7.2x )	181454 ( 1.013 )	36084 ( 1.0 )	2048	muRISCV-NN	Vector	RV32GCV	0	-
2098133.0 ( 7.2x )	181454 ( 1.013 )	36084 ( 1.0 )	4096	muRISCV-NN	Vector	RV32GCV	0	-
6398459.0 ( 2.4x )	198508 ( 1.109 )	36092 ( 1.0 )	128	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP
5432019.0 ( 2.8x )	198804 ( 1.11 )	36092 ( 1.0 )	256	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP
5097002.0 ( 3.0x )	199066 ( 1.112 )	36092 ( 1.0 )	512	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP
6181653.0 ( 2.4x )	199214 ( 1.113 )	36092 ( 1.0 )	1024	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP
14648560.0 ( 1.0x )	199306 ( 1.113 )	36092 ( 1.0 )	2048	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP
14651951.0 ( 1.0x )	199874 ( 1.116 )	36092 ( 1.0 )	4096	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP

Image Classification (resnet)

Cycles (Speedup)	Total ROM (rel.)	Total RAM (rel.)	VLEN	Kernels	Mode	Arch	Unroll	Auto-Vectorization
120042844.0 ( 0.5x )	200020 ( 0.941 )	68852 ( 1.0 )	0	TFLM	Reference	RV32GC	0	-
52315145.0 ( 1.1x )	217084 ( 1.021 )	68860 ( 1.0 )	128	TFLM	Reference	RV32GCV	0	Loop+SLP
44654160.0 ( 1.3x )	217438 ( 1.023 )	68860 ( 1.0 )	256	TFLM	Reference	RV32GCV	0	Loop+SLP
82961728.0 ( 0.7x )	218012 ( 1.026 )	68860 ( 1.0 )	512	TFLM	Reference	RV32GCV	0	Loop+SLP
107873026.0 ( 0.5x )	218392 ( 1.027 )	68860 ( 1.0 )	1024	TFLM	Reference	RV32GCV	0	Loop+SLP
123206621.0 ( 0.5x )	218732 ( 1.029 )	68860 ( 1.0 )	2048	TFLM	Reference	RV32GCV	0	Loop+SLP
123223693.0 ( 0.5x )	220132 ( 1.036 )	68860 ( 1.0 )	4096	TFLM	Reference	RV32GCV	0	Loop+SLP
56212127.0 ( Base )	212558 ( Base )	68848 ( Base )	0	muRISCV-NN	Scalar	RV32GC	0	-
72277759.0 ( 0.8x )	212186 ( 0.998 )	68848 ( 1.0 )	0	muRISCV-NN	Vector (Portable)	RV32GC	0	-
11808236.0 ( 4.8x )	234276 ( 1.102 )	68856 ( 1.0 )	128	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
11356182.0 ( 4.9x )	234614 ( 1.104 )	68856 ( 1.0 )	256	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
14221400.0 ( 4.0x )	235128 ( 1.106 )	68856 ( 1.0 )	512	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
15712058.0 ( 3.6x )	235418 ( 1.108 )	68856 ( 1.0 )	1024	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
16394497.0 ( 3.4x )	235672 ( 1.109 )	68856 ( 1.0 )	2048	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
35580201.0 ( 1.6x )	236666 ( 1.113 )	68856 ( 1.0 )	4096	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
15153355.0 ( 3.7x )	215228 ( 1.013 )	68848 ( 1.0 )	128	muRISCV-NN	Vector	RV32GCV	0	-
9570159.0 ( 5.9x )	215248 ( 1.013 )	68848 ( 1.0 )	256	muRISCV-NN	Vector	RV32GCV	0	-
7024321.0 ( 8.0x )	215212 ( 1.012 )	68848 ( 1.0 )	512	muRISCV-NN	Vector	RV32GCV	0	-
5782193.0 ( 9.7x )	215250 ( 1.013 )	68848 ( 1.0 )	1024	muRISCV-NN	Vector	RV32GCV	0	-
4857320.0 ( 11.6x )	215250 ( 1.013 )	68848 ( 1.0 )	2048	muRISCV-NN	Vector	RV32GCV	0	-
4611619.0 ( 12.2x )	215250 ( 1.013 )	68848 ( 1.0 )	4096	muRISCV-NN	Vector	RV32GCV	0	-
17594933.0 ( 3.2x )	233742 ( 1.1 )	68856 ( 1.0 )	128	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP
16562252.0 ( 3.4x )	234080 ( 1.101 )	68856 ( 1.0 )	256	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP
17328032.0 ( 3.2x )	234598 ( 1.104 )	68856 ( 1.0 )	512	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP
17041373.0 ( 3.3x )	234888 ( 1.105 )	68856 ( 1.0 )	1024	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP
16582090.0 ( 3.4x )	235142 ( 1.106 )	68856 ( 1.0 )	2048	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP
43363339.0 ( 1.3x )	236136 ( 1.111 )	68856 ( 1.0 )	4096	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP

Anomaly Detection (toycar)

Cycles (Speedup)	Total ROM (rel.)	Total RAM (rel.)	VLEN	Kernels	Mode	Arch	Unroll	Auto-Vectorization
2804439.0 ( 0.6x )	345800 ( 0.983 )	19372 ( 1.0 )	0	TFLM	Reference	RV32GC	0	-
800177.0 ( 2.1x )	349262 ( 0.993 )	19372 ( 1.0 )	128	TFLM	Reference	RV32GCV	0	Loop+SLP
626673.0 ( 2.7x )	349358 ( 0.993 )	19372 ( 1.0 )	256	TFLM	Reference	RV32GCV	0	Loop+SLP
531064.0 ( 3.2x )	349482 ( 0.994 )	19372 ( 1.0 )	512	TFLM	Reference	RV32GCV	0	Loop+SLP
487504.0 ( 3.5x )	349614 ( 0.994 )	19372 ( 1.0 )	1024	TFLM	Reference	RV32GCV	0	Loop+SLP
464888.0 ( 3.6x )	349714 ( 0.994 )	19372 ( 1.0 )	2048	TFLM	Reference	RV32GCV	0	Loop+SLP
2220296.0 ( 0.8x )	349824 ( 0.995 )	19372 ( 1.0 )	4096	TFLM	Reference	RV32GCV	0	Loop+SLP
1689644.0 ( Base )	351696 ( Base )	19372 ( Base )	0	muRISCV-NN	Scalar	RV32GC	0	-
3038990.0 ( 0.6x )	351698 ( 1.0 )	19372 ( 1.0 )	0	muRISCV-NN	Vector (Portable)	RV32GC	0	-
552790.0 ( 3.1x )	356752 ( 1.014 )	19372 ( 1.0 )	128	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
465571.0 ( 3.6x )	356830 ( 1.015 )	19372 ( 1.0 )	256	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
420360.0 ( 4.0x )	356904 ( 1.015 )	19372 ( 1.0 )	512	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
402900.0 ( 4.2x )	356998 ( 1.015 )	19372 ( 1.0 )	1024	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
391588.0 ( 4.3x )	357060 ( 1.015 )	19372 ( 1.0 )	2048	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
1328532.0 ( 1.3x )	357124 ( 1.015 )	19372 ( 1.0 )	4096	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
1937850.0 ( 0.9x )	352802 ( 1.003 )	19372 ( 1.0 )	128	muRISCV-NN	Vector	RV32GCV	0	-
1821170.0 ( 0.9x )	352788 ( 1.003 )	19372 ( 1.0 )	256	muRISCV-NN	Vector	RV32GCV	0	-
1762829.0 ( 1.0x )	352756 ( 1.003 )	19372 ( 1.0 )	512	muRISCV-NN	Vector	RV32GCV	0	-
1733984.0 ( 1.0x )	352794 ( 1.003 )	19372 ( 1.0 )	1024	muRISCV-NN	Vector	RV32GCV	0	-
1730353.0 ( 1.0x )	352794 ( 1.003 )	19372 ( 1.0 )	2048	muRISCV-NN	Vector	RV32GCV	0	-
1728497.0 ( 1.0x )	352794 ( 1.003 )	19372 ( 1.0 )	4096	muRISCV-NN	Vector	RV32GCV	0	-
789422.0 ( 2.1x )	356754 ( 1.014 )	19372 ( 1.0 )	128	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP
607480.0 ( 2.8x )	356832 ( 1.015 )	19372 ( 1.0 )	256	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP
512931.0 ( 3.3x )	356906 ( 1.015 )	19372 ( 1.0 )	512	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP
470794.0 ( 3.6x )	357000 ( 1.015 )	19372 ( 1.0 )	1024	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP
447157.0 ( 3.8x )	357062 ( 1.015 )	19372 ( 1.0 )	2048	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP
2357596.0 ( 0.7x )	357126 ( 1.015 )	19372 ( 1.0 )	4096	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP

Visual Wake Words (vww)

Cycles (Speedup)	Total ROM (rel.)	Total RAM (rel.)	VLEN	Kernels	Mode	Arch	Unroll	Auto-Vectorization
101694057.0 ( 0.4x )	433474 ( 0.958 )	134388 ( 1.0 )	0	TFLM	Reference	RV32GC	0	-
69591099.0 ( 0.6x )	448764 ( 0.991 )	134396 ( 1.0 )	128	TFLM	Reference	RV32GCV	0	Loop+SLP
67271682.0 ( 0.7x )	449020 ( 0.992 )	134396 ( 1.0 )	256	TFLM	Reference	RV32GCV	0	Loop+SLP
67391392.0 ( 0.7x )	449366 ( 0.993 )	134396 ( 1.0 )	512	TFLM	Reference	RV32GCV	0	Loop+SLP
73186430.0 ( 0.6x )	449624 ( 0.993 )	134396 ( 1.0 )	1024	TFLM	Reference	RV32GCV	0	Loop+SLP
79704928.0 ( 0.6x )	449844 ( 0.994 )	134396 ( 1.0 )	2048	TFLM	Reference	RV32GCV	0	Loop+SLP
105202232.0 ( 0.4x )	450572 ( 0.995 )	134396 ( 1.0 )	4096	TFLM	Reference	RV32GCV	0	Loop+SLP
45215610.0 ( Base )	452688 ( Base )	134388 ( Base )	0	muRISCV-NN	Scalar	RV32GC	0	-
44844965.0 ( 1.0x )	452476 ( 1.0 )	134388 ( 1.0 )	0	muRISCV-NN	Vector (Portable)	RV32GC	0	-
17462016.0 ( 2.6x )	471456 ( 1.041 )	134396 ( 1.0 )	128	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
17039215.0 ( 2.7x )	471780 ( 1.042 )	134396 ( 1.0 )	256	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
18215342.0 ( 2.5x )	472034 ( 1.043 )	134396 ( 1.0 )	512	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
21361663.0 ( 2.1x )	472182 ( 1.043 )	134396 ( 1.0 )	1024	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
24888125.0 ( 1.8x )	472292 ( 1.043 )	134396 ( 1.0 )	2048	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
38847252.0 ( 1.2x )	473140 ( 1.045 )	134396 ( 1.0 )	4096	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
13253929.0 ( 3.4x )	455066 ( 1.005 )	134388 ( 1.0 )	128	muRISCV-NN	Vector	RV32GCV	0	-
9932959.0 ( 4.6x )	455094 ( 1.005 )	134388 ( 1.0 )	256	muRISCV-NN	Vector	RV32GCV	0	-
8641538.0 ( 5.2x )	455058 ( 1.005 )	134388 ( 1.0 )	512	muRISCV-NN	Vector	RV32GCV	0	-
8137559.0 ( 5.6x )	455096 ( 1.005 )	134388 ( 1.0 )	1024	muRISCV-NN	Vector	RV32GCV	0	-
8093184.0 ( 5.6x )	455096 ( 1.005 )	134388 ( 1.0 )	2048	muRISCV-NN	Vector	RV32GCV	0	-
8096575.0 ( 5.6x )	455096 ( 1.005 )	134388 ( 1.0 )	4096	muRISCV-NN	Vector	RV32GCV	0	-
20281829.0 ( 2.2x )	472150 ( 1.043 )	134396 ( 1.0 )	128	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP
19155334.0 ( 2.4x )	472446 ( 1.044 )	134396 ( 1.0 )	256	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP
20670053.0 ( 2.2x )	472708 ( 1.044 )	134396 ( 1.0 )	512	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP
23816347.0 ( 1.9x )	472856 ( 1.045 )	134396 ( 1.0 )	1024	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP
27334446.0 ( 1.7x )	472948 ( 1.045 )	134396 ( 1.0 )	2048	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP
41293548.0 ( 1.1x )	473796 ( 1.047 )	134396 ( 1.0 )	4096	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP

Original data

Click here to download the raw files for this benchmark.