Benchmarks 2023 06 02

Setup

Simulator

• Spike (riscv-isa-sim ) (ISS, CPI=1)

Toolchains

• Scalar: Self compiled on master branch of riscv-gnu-toolchain
• Packed: Self compiled using patches found in https://github.com/riscv-collab/riscv-gcc/pull/258 and https://github.com/riscvarchive/riscv-binutils-gdb/pull/257
• Vector: Self compiled using this working-branch: riscv-gcc-rvv-next

Models

• MLPerfTiny Benchmark

Package Versions

• MLonMCU : 2ab749d3747c04f3b01973ef955bda5be0964f0f
• TFLM : f050eec7e32a0895f7658db21a4bdbd0975087a5
• TVM : Nightly Pre-Build
• Spike : 0bc176b3fca43560b9e8586cdbc41cfde073e17a
• Spike PK : 7e9b671c0415dfd7b562ac934feb9380075d4aa2

Miscellaneous

• Used -Os flag for compilation.
• Benchmarks generated using MLonMCU deployment tool with minimal efforts.
• Memory metrics are reported in Bytes

Frameworks

Benchmarks exist for integration with TensorFlow Lite for Microcontrollers (TFLM) and TVM.

TFLite for Microcontrollers

Comparing reference kernel implementations (Default) with cmsis-nn optimized kernels and linkage with libmuriscvnn.a.

Audio Wake Words (aww)

Cycles	Cycles(Relative)	Total ROM	Total ROM(Relative)	Total RAM	Total RAM(Relative)	ROM read-only	ROM code	VLEN	Kernels	Extensions
153529267	10.016	144687	0.912	36192	1.0	63653	79826	0	Default
15327748	1.0	158696	1.0	36200	1.0	63656	93832	0	muRISCV-NN
9026170	0.589	157944	0.995	36208	1.0	63652	93088	64	muRISCV-NN	VEXT
2135130	0.139	157944	0.995	36208	1.0	63652	93088	1024	muRISCV-NN	VEXT
6044575	0.394	201872	1.272	36344	1.004	70740	126136	0	muRISCV-NN	PEXT

Visual Wake Words (vww)

Cycles	Cycles(Relative)	Total ROM	Total ROM(Relative)	Total RAM	Total RAM(Relative)	ROM read-only	ROM code	VLEN	Kernels	Extensions
433523791	9.376	418411	0.968	134508	1.0	337293	79910	0	Default
46237716	1.0	432420	1.0	134516	1.0	337296	93916	0	muRISCV-NN
27212603	0.589	431668	0.998	134524	1.0	337292	93172	64	muRISCV-NN	VEXT
8775554	0.19	431668	0.998	134524	1.0	337292	93172	1024	muRISCV-NN	VEXT
19841712	0.429	475588	1.1	134660	1.001	344380	126212	0	muRISCV-NN	PEXT

Image Classification (resnet)

Cycles	Cycles(Relative)	Total ROM	Total ROM(Relative)	Total RAM	Total RAM(Relative)	ROM read-only	ROM code	VLEN	Kernels	Extensions
687851073	12.52	184423	0.956	68956	1.0	102485	80730	0	Default
54941631	1.0	192828	1.0	68948	1.0	102488	89132	0	muRISCV-NN
37856218	0.689	192866	1.0	68956	1.0	102500	89162	64	muRISCV-NN	VEXT
6058820	0.11	192866	1.0	68956	1.0	102500	89162	1024	muRISCV-NN	VEXT
26794473	0.488	238744	1.238	69092	1.002	109572	124176	0	muRISCV-NN	PEXT

Anomaly Detection (toycar)

Cycles	Cycles(Relative)	Total ROM	Total ROM(Relative)	Total RAM	Total RAM(Relative)	ROM read-only	ROM code	VLEN	Kernels	Extensions
3434813	1.788	347340	0.993	19392	1.0	280872	65260	0	Default
1920500	1.0	349928	1.0	19392	1.0	280872	67848	0	muRISCV-NN
729214	0.38	349838	1.0	19396	1.0	280868	67766	64	muRISCV-NN	VEXT
538790	0.281	349838	1.0	19396	1.0	280868	67766	1024	muRISCV-NN	VEXT
788434	0.411	394176	1.126	19536	1.007	287956	101224	0	muRISCV-NN	PEXT

TVM

Using CMSIS-NN BYOC feature and linkage with libmuriscvnn.a.

Extension mapping:

• VEXT -> cortex-m55
• PEXT -> cortex-m7

Used data/kernel layouts:

• Untuned: NHWC (TFLite default)
• Tuned: NCHW (Converted using TVM)

Audio Wake Words (aww)

Cycles	Cycles(Relative)	Total ROM	Total ROM(Relative)	Total RAM	Total RAM(Relative)	ROM read-only	ROM code	VLEN	Layout	Kernels	Extensions
31435272	2.046	117272	1.171	59448	3.106	56212	61044	0	NCHW	Default
29580732	1.926	111594	1.114	59448	3.106	56212	55366	0	NHWC	Default
15699726	1.022	110750	1.106	51276	2.679	56212	54522	0	NCHW	Autotuned
29576111	1.925	111724	1.115	59448	3.106	56212	55496	0	NHWC	Autotuned
15361113	1.0	100174	1.0	19140	1.0	36644	63514	0		muRISCV-NN
10091110	0.657	99576	0.994	23604	1.233	36644	62916	64		muRISCV-NN	VEXT
3987654	0.26	99576	0.994	23604	1.233	36644	62916	1024		muRISCV-NN	VEXT
6755351	0.44	106586	1.064	20292	1.06	36400	70110	0		muRISCV-NN	PEXT

Visual Wake Words (vww)

Cycles	Cycles(Relative)	Total ROM	Total ROM(Relative)	Total RAM	Total RAM(Relative)	ROM read-only	ROM code	VLEN	Layout	Kernels	Extensions
90828165	2.016	550218	1.648	180968	2.114	462932	87270	0	NCHW	Default
86102169	1.911	530776	1.59	180968	2.114	462932	67828	0	NHWC	Default
46356353	1.029	531550	1.593	180968	2.114	462932	68602	0	NCHW	Autotuned
86102172	1.911	530780	1.59	180968	2.114	462932	67832	0	NHWC	Autotuned
45064061	1.0	333770	1.0	85588	1.0	266324	67430	0		muRISCV-NN
26022364	0.577	333460	0.999	85588	1.0	266324	67120	64		muRISCV-NN	VEXT
7585315	0.168	333434	0.999	85588	1.0	266324	67094	1024		muRISCV-NN	VEXT
18622291	0.413	340446	1.02	85588	1.0	266080	74290	0		muRISCV-NN	PEXT

Image Classification (resnet)

Cycles	Cycles(Relative)	Total ROM	Total ROM(Relative)	Total RAM	Total RAM(Relative)	ROM read-only	ROM code	VLEN	Layout	Kernels	Extensions
138976677	2.543	226522	1.536	108360	1.954	163156	63350	0	NCHW	Default
115540068	2.115	217372	1.474	108360	1.954	163156	54200	0	NHWC	Default
56900335	1.041	220452	1.495	92176	1.663	163156	57280	0	NCHW	Autotuned
115536141	2.114	217472	1.475	108360	1.954	163156	54300	0	NHWC	Autotuned
54640014	1.0	147486	1.0	55444	1.0	87332	60138	0		muRISCV-NN
37562250	0.687	147640	1.001	55444	1.0	87332	60292	64		muRISCV-NN	VEXT
5764852	0.106	147640	1.001	55444	1.0	87332	60292	1024		muRISCV-NN	VEXT
26487976	0.485	156704	1.063	55444	1.0	87088	69540	0		muRISCV-NN	PEXT

Anomaly Detection (toycar)

Cycles	Cycles(Relative)	Total ROM	Total ROM(Relative)	Total RAM	Total RAM(Relative)	ROM read-only	ROM code	VLEN	Layout	Kernels	Extensions
3134840	1.72	589618	1.815	5492	1.171	537988	51614	0	NCHW	Default
3134840	1.72	589618	1.815	5492	1.171	537988	51614	0	NHWC	Default
2041108	1.12	617982	1.902	6804	1.45	537988	79978	0	NCHW	Autotuned
2041108	1.12	617982	1.902	6804	1.45	537988	79978	0	NHWC	Autotuned
1822341	1.0	324916	1.0	4692	1.0	273924	50976	0		muRISCV-NN
626114	0.344	324966	1.0	4692	1.0	273924	51026	64		muRISCV-NN	VEXT
435690	0.239	324952	1.0	4692	1.0	273924	51012	1024		muRISCV-NN	VEXT
680159	0.373	327564	1.008	4692	1.0	273680	53808	0		muRISCV-NN	PEXT

Code size compared to CMSIS-NN

WIP