Benchmarks 2023 12 22 TFLM GCC

Setup

Simulator

• Spike (riscv-isa-sim ) (ISS, CPI=1)

Toolchains

• Scalar: Self compiled on master branch of riscv-gnu-toolchain
• Packed: Self compiled using patches found in https://github.com/riscv-collab/riscv-gcc/pull/258 and https://github.com/riscvarchive/riscv-binutils-gdb/pull/257
• Vector: Self compiled using this working-branch: riscv-gcc-rvv-next

Models

• MLPerfTiny Benchmark

Package Versions

• MLonMCU : main
• TFLM : main
• TVM : Nightly Pre-Build
• Spike : 0bc176b3fca43560b9e8586cdbc41cfde073e17a
• Spike PK : 7e9b671c0415dfd7b562ac934feb9380075d4aa2

Miscellaneous

• Used -Os flag for compilation.
• Benchmarks generated using MLonMCU deployment tool with minimal efforts.
• Memory metrics are reported in Bytes

Results (Framework: tflm, Toolchain: gcc, Backend: tflmi)

Audio Wake Words (aww)

Cycles	Cycles(Relative)	Total ROM	Total ROM(Relative)	Total RAM	Total RAM(Relative)	ROM read-only	ROM code	VLEN	Kernels	Extensions
174698643	10.496	132369	0.878	36204	1.0	62601	69752	0	Default	-
16644691	1.0	150756	1.0	36212	1.0	62604	88136	0	muRISCV-NN	-
8093904	0.486	163062	1.082	36172	0.999	63556	99490	64	muRISCV-NN	VEXT
1907335	0.115	163062	1.082	36172	0.999	63556	99490	1024	muRISCV-NN	VEXT
15939663	0.958	164734	1.093	36156	0.998	63344	101314	0	muRISCV-NN	PEXT

Image Classification (resnet)

Cycles	Cycles(Relative)	Total ROM	Total ROM(Relative)	Total RAM	Total RAM(Relative)	ROM read-only	ROM code	VLEN	Kernels	Extensions
745795985	9.208	172947	0.934	68968	1.0	101449	71482	0	Default	-
80990027	1.0	185258	1.0	68960	1.0	101452	83790	0	muRISCV-NN	-
34085610	0.421	197854	1.068	68920	0.999	102404	95434	64	muRISCV-NN	VEXT
5856940	0.072	197854	1.068	68920	0.999	102404	95434	1024	muRISCV-NN	VEXT
68431268	0.845	199920	1.079	68904	0.999	102192	97652	0	muRISCV-NN	PEXT

Anomaly Detection (toycar)

Cycles	Cycles(Relative)	Total ROM	Total ROM(Relative)	Total RAM	Total RAM(Relative)	ROM read-only	ROM code	VLEN	Kernels	Extensions
3094956	1.566	333794	0.989	19432	1.0	279932	53846	0	Default	-
1976243	1.0	337442	1.0	19432	1.0	279932	57494	0	muRISCV-NN	-
583051	0.295	349466	1.036	19384	0.998	280884	68566	64	muRISCV-NN	VEXT
392627	0.199	349466	1.036	19384	0.998	280884	68566	1024	muRISCV-NN	VEXT
942627	0.477	351390	1.041	19380	0.997	280672	70642	0	muRISCV-NN	PEXT

Visual Wake Words (vww)

Cycles	Cycles(Relative)	Total ROM	Total ROM(Relative)	Total RAM	Total RAM(Relative)	ROM read-only	ROM code	VLEN	Kernels	Extensions
495266206	9.97	406073	0.957	134520	1.0	336241	69816	0	Default	-
49676817	1.0	424460	1.0	134528	1.0	336244	88200	0	muRISCV-NN	-
24515217	0.493	436776	1.029	134488	1.0	337196	99564	64	muRISCV-NN	VEXT
8009633	0.161	436776	1.029	134488	1.0	337196	99564	1024	muRISCV-NN	VEXT
49178925	0.99	438438	1.033	134472	1.0	336984	101378	0	muRISCV-NN	PEXT

Original data

Click here to download the raw files for this benchmark.