Benchmarks (OLD)

Setup

Simulator

• Spike (riscv-isa-sim ) (ISS, CPI=1)

Toolchains

• Scalar: Self compiled on master branch of riscv-gnu-toolchain
• Packed: Self compiled using patches found in https://github.com/riscv-collab/riscv-gcc/pull/258 and https://github.com/riscvarchive/riscv-binutils-gdb/pull/257
• Vector: Self compiled using this working-branch: riscv-gcc-rvv-next

Models

• MLPerfTiny Benchmark

Miscellaneous

• Used -Os flag for compilation.
• Benchmarks generated using MLonMCU deployment tool with minimal efforts.
• Memory metrics are reported in Bytes

Frameworks

Benchmarks exist for integration with TensorFlow Lite for Microcontrollers (TFLM) and TVM.

TFLite for Microcontrollers

Comparing reference kernel implementations (Default) with cmsis-nn optimized kernels and linkage with libmuriscvnn.a.

Audio Wake Words (aww)

Cycles	Total ROM	Total RAM	ROM read-only	ROM code	VLEN	Kernels	Extensions
153522095	140081	36188	63609	76456		Default
16596256	152628	36196	63368	89244		muRISCV-NN
8866565	153330	36204	63368	89946	64	muRISCV-NN	VEXT
5246477	153330	36204	63368	89946	128	muRISCV-NN	VEXT
3478609	153330	36204	63368	89946	256	muRISCV-NN	VEXT
2597665	153330	36204	63368	89946	512	muRISCV-NN	VEXT
2114537	153330	36204	63368	89946	1024	muRISCV-NN	VEXT
2114537	153330	36204	63368	89946	2048	muRISCV-NN	VEXT
13981704	195616	36376	70492	120128		muRISCV-NN	PEXT

Visual Wake Words (vww)

Cycles	Total ROM	Total RAM	ROM read-only	ROM code	VLEN	Kernels	Extensions
433511902	413807	134504	337249	76542	0	Default
49683114	426354	134512	337008	89330	0	muRISCV-NN
26792825	427056	134520	337008	90032	64	muRISCV-NN	VEXT
16677353	427056	134520	337008	90032	128	muRISCV-NN	VEXT
11855732	427056	134520	337008	90032	256	muRISCV-NN	VEXT
9577078	427056	134520	337008	90032	512	muRISCV-NN	VEXT
8728783	427056	134520	337008	90032	1024	muRISCV-NN	VEXT
8391571	427056	134520	337008	90032	2048	muRISCV-NN	VEXT
41507742	469332	134692	344132	120204	0	muRISCV-NN	PEXT

Image Classification (resnet)

Cycles	Total ROM	Total RAM	ROM read-only	ROM code	VLEN	Kernels	Extensions
687841744	179827	68952	102441	77370	0	Default
62517171	187130	68944	102200	84914	0	muRISCV-NN
37066327	188074	68952	102216	85842	64	muRISCV-NN	VEXT
20268499	188074	68952	102216	85842	128	muRISCV-NN	VEXT
11781521	188074	68952	102216	85842	256	muRISCV-NN	VEXT
7802289	188074	68952	102216	85842	512	muRISCV-NN	VEXT
5988801	188074	68952	102216	85842	1024	muRISCV-NN	VEXT
5104073	188074	68952	102216	85842	2048	muRISCV-NN	VEXT
64766429	232348	69124	109324	118028	0	muRISCV-NN	PEXT

Anomaly Detection (toycar)

Cycles	Total ROM	Total RAM	ROM read-only	ROM code	VLEN	Kernels	Extensions
3428442	342532	19388	280584	61932	0	Default
1919268	345114	19388	280584	64514	0	muRISCV-NN
728156	345452	19392	280584	64852	64	muRISCV-NN	VEXT
621244	345452	19392	280584	64852	128	muRISCV-NN	VEXT
567788	345452	19392	280584	64852	256	muRISCV-NN	VEXT
541060	345452	19392	280584	64852	512	muRISCV-NN	VEXT
537732	345452	19392	280584	64852	1024	muRISCV-NN	VEXT
536068	345452	19392	280584	64852	2048	muRISCV-NN	VEXT
1173827	388802	19568	287708	96098	0	muRISCV-NN	PEXT

TVM

Using CMSIS-NN BYOC feature and linkage with libmuriscvnn.a.

Extension mapping:

• VEXT -> cortex-m55
• PEXT -> cortex-m7

Used data/kernel layouts:

• Untuned: NHWC (TFLite default)
• Tuned: NCHW (Converted using TVM)

Audio Wake Words (aww)

Cycles	Total ROM	Total RAM	ROM read-only	ROM code	VLEN	Kernels	Extensions
30647875	120186	123452	63136	57034		Default
16008614	118014	51280	63136	54862		Autotuned
16449679	99128	19140	36656	62456		muRISCV-NN
9930305	99984	21448	36660	63308	64	muRISCV-NN	VEXT
6737737	99984	21448	36660	63308	128	muRISCV-NN	VEXT
5141453	99984	21448	36660	63308	256	muRISCV-NN	VEXT
4345821	99984	21448	36660	63308	512	muRISCV-NN	VEXT
3948005	99984	21448	36660	63308	1024	muRISCV-NN	VEXT
3948005	99984	21448	36660	63308	2048	muRISCV-NN	VEXT
13703187	100748	20292	36400	64272		muRISCV-NN	PEXT

Visual Wake Words (vww)

Cycles	Total ROM	Total RAM	ROM read-only	ROM code	VLEN	Kernels	Extensions
88112318	569014	294860	495776	73222		Default
48604948	565620	180972	495776	69828		Autotuned
48514662	332820	85588	266336	66468		muRISCV-NN
25611363	333766	85588	266340	67410	64	muRISCV-NN	VEXT
15495891	333782	85588	266340	67426	128	muRISCV-NN	VEXT
10674270	333752	85588	266340	67396	256	muRISCV-NN	VEXT
8395616	333748	85588	266340	67392	512	muRISCV-NN	VEXT
7547321	333754	85588	266340	67398	1024	muRISCV-NN	VEXT
7210109	333750	85588	266340	67394	2048	muRISCV-NN	VEXT
40307469	334704	85588	266080	68548		muRISCV-NN	PEXT

Image Classification (resnet)

Cycles	Total ROM	Total RAM	ROM read-only	ROM code	VLEN	Kernels	Extensions
116326122	221644	108364	167200	54428		Default
57693988	224774	92180	167200	57558		Autotuned
62228697	147016	55444	87344	59656		muRISCV-NN
36779647	148074	55444	87348	60710	64	muRISCV-NN	VEXT
19981819	148074	55444	87348	60710	128	muRISCV-NN	VEXT
11494841	148074	55444	87348	60710	256	muRISCV-NN	VEXT
7515609	148074	55444	87348	60710	512	muRISCV-NN	VEXT
5702121	148074	55444	87348	60710	1024	muRISCV-NN	VEXT
4817393	148074	55444	87348	60710	2048	muRISCV-NN	VEXT
64455063	150898	55444	87088	63734		muRISCV-NN	PEXT

Anomaly Detection (toycar)

Cycles	Total ROM	Total RAM	ROM read-only	ROM code	VLEN	Kernels	Extensions
3134849	589630	5492	538000	51614		Default
2041108	617996	6804	538000	79980		Autotuned
1822351	324996	4692	273936	51044		muRISCV-NN
631118	325494	4692	273940	51538	64	muRISCV-NN	VEXT
524208	325504	4692	273940	51548	128	muRISCV-NN	VEXT
470752	325498	4692	273940	51542	256	muRISCV-NN	VEXT
444024	325506	4692	273940	51550	512	muRISCV-NN	VEXT
440694	325496	4692	273940	51540	1024	muRISCV-NN	VEXT
439032	325504	4692	273940	51548	2048	muRISCV-NN	VEXT
1066416	327302	4692	273680	53546		muRISCV-NN	PEXT

Code size compared to CMSIS-NN

WIP