Benchmarks 2024 07 12 TFLM GCC Os spike_rv64

Setup

Simulator

• Spike (riscv-isa-sim ) (ISS, CPI=1)

Toolchains

• RISC-V GCC:
  • Scalar: TODO: version & url
  • Vector: TODO: version & url
  • Packed: Self compiled using patches found in https://github.com/riscv-collab/riscv-gcc/pull/258 and https://github.com/riscvarchive/riscv-binutils-gdb/pull/257

Models

• MLPerfTiny Benchmark

• TODO: others!

Package Versions

• MLonMCU : main

• TFLM : main

• Spike : 0bc176b3fca43560b9e8586cdbc41cfde073e17a

• Spike PK : 7e9b671c0415dfd7b562ac934feb9380075d4aa2

Miscellaneous

• Used -Os flag for compilation.
• Benchmarks generated using MLonMCU deployment tool with minimal efforts.
• Memory metrics are reported in Bytes

Results (Framework: tflm, Backend: tflmi, Toolchain: gcc, Flags: -Os)

Audio Wake Words (aww)

Cycles (Speedup)	Total ROM (rel.)	Total RAM (rel.)	VLEN	Kernels	Mode	Arch	Unroll	Auto-Vectorization
165492824 ( 0.1x )	129999 ( 0.873 )	38440 ( 1.0 )	128	TFLM	Reference	RV64GC	False	-
165492824 ( 0.1x )	130001 ( 0.873 )	38440 ( 1.0 )	128	TFLM	Reference	RV64GCV	False	Loop+SLP
18671271 ( Base )	148942 ( Base )	38448 ( Base )	128	muRISCV-NN	Scalar	RV64GC	False	-
18660337 ( 1.0x )	149766 ( 1.006 )	38448 ( 1.0 )	128	muRISCV-NN	Scalar	RV64GCV	False	Loop+SLP
4180366 ( 4.5x )	149752 ( 1.005 )	38448 ( 1.0 )	128	muRISCV-NN	Vector	RV64GCV	False	-

Image Classification (resnet)

Cycles (Speedup)	Total ROM (rel.)	Total RAM (rel.)	VLEN	Kernels	Mode	Arch	Unroll	Auto-Vectorization
696993700 ( 0.1x )	170559 ( 0.935 )	71208 ( 1.0 )	128	TFLM	Reference	RV64GC	False	-
696993700 ( 0.1x )	170553 ( 0.935 )	71208 ( 1.0 )	128	TFLM	Reference	RV64GCV	False	Loop+SLP
88730172 ( Base )	182496 ( Base )	71192 ( Base )	128	muRISCV-NN	Scalar	RV64GC	False	-
88730480 ( 1.0x )	183048 ( 1.003 )	71192 ( 1.0 )	128	muRISCV-NN	Scalar	RV64GCV	False	Loop+SLP
15625959 ( 5.7x )	184270 ( 1.01 )	71192 ( 1.0 )	128	muRISCV-NN	Vector	RV64GCV	False	-

Anomaly Detection (toycar)

Cycles (Speedup)	Total ROM (rel.)	Total RAM (rel.)	VLEN	Kernels	Mode	Arch	Unroll	Auto-Vectorization
3877122 ( 0.5x )	330846 ( 0.99 )	21528 ( 1.0 )	128	TFLM	Reference	RV64GC	False	-
3877122 ( 0.5x )	330838 ( 0.99 )	21528 ( 1.0 )	128	TFLM	Reference	RV64GCV	False	Loop+SLP
1948153 ( Base )	334342 ( Base )	21528 ( Base )	128	muRISCV-NN	Scalar	RV64GC	False	-
1948206 ( 1.0x )	334360 ( 1.0 )	21528 ( 1.0 )	128	muRISCV-NN	Scalar	RV64GCV	False	Loop+SLP
589615 ( 3.3x )	335070 ( 1.002 )	21528 ( 1.0 )	128	muRISCV-NN	Vector	RV64GCV	False	-

Visual Wake Words (vww)

Cycles (Speedup)	Total ROM (rel.)	Total RAM (rel.)	VLEN	Kernels	Mode	Arch	Unroll	Auto-Vectorization
466676303 ( 0.1x )	403713 ( 0.955 )	136760 ( 1.0 )	128	TFLM	Reference	RV64GC	False	-
466676303 ( 0.1x )	403715 ( 0.955 )	136760 ( 1.0 )	128	TFLM	Reference	RV64GCV	False	Loop+SLP
55403045 ( Base )	422656 ( Base )	136768 ( Base )	128	muRISCV-NN	Scalar	RV64GC	False	-
55404653 ( 1.0x )	423480 ( 1.002 )	136768 ( 1.0 )	128	muRISCV-NN	Scalar	RV64GCV	False	Loop+SLP
13885455 ( 4.0x )	423466 ( 1.002 )	136768 ( 1.0 )	128	muRISCV-NN	Vector	RV64GCV	False	-

Original data

Click here to download the raw files for this benchmark.