Benchmarks CUSTOM TVM GCC Os

Setup

Simulator

Spike (riscv-isa-sim ) (ISS, CPI=1)

Toolchains

RISC-V GCC:
- Scalar: TODO: version & url
- Vector: TODO: version & url
- Packed: Self compiled using patches found in https://github.com/riscv-collab/riscv-gcc/pull/258 and https://github.com/riscvarchive/riscv-binutils-gdb/pull/257

Models

MLPerfTiny Benchmark
TODO: others!

Package Versions

MLonMCU : main
TVM : Nightly Pre-Build
Spike : 0bc176b3fca43560b9e8586cdbc41cfde073e17a
Spike PK : 7e9b671c0415dfd7b562ac934feb9380075d4aa2

Miscellaneous

Used -Os flag for compilation.
Benchmarks generated using MLonMCU deployment tool with minimal efforts.
Memory metrics are reported in Bytes

Results (Framework: tvm, Backend: tvmaot, Toolchain: gcc, Flags: -Os)

Audio Wake Words (`aww`)

Cycles (Speedup)	Total ROM (rel.)	Total RAM (rel.)	VLEN	Layout	Kernels	Mode	Arch	Unroll	Auto-Vectorization
`31536718` ( 0.5x )	`107754` ( 1.266 )	`59536` ( 3.097 )	128	NCHW	TVM	Fallback	RV32GC	False	-
`29578962` ( 0.6x )	`101832` ( 1.197 )	`59536` ( 3.097 )	128	NHWC	TVM	Fallback	RV32GC	False	-
`15873108` ( 1.1x )	`101286` ( 1.19 )	`51364` ( 2.672 )	128	NCHW	TVM	Autotuned	RV32GC	False	-
`29574349` ( 0.6x )	`101974` ( 1.198 )	`59536` ( 3.097 )	128	NHWC	TVM	Autotuned	RV32GC	False	-
`31529514` ( 0.5x )	`107730` ( 1.266 )	`59536` ( 3.097 )	128	NCHW	TVM	Fallback	RV32GCV	False	Loop+SLP
`31529514` ( 0.5x )	`107730` ( 1.266 )	`59536` ( 3.097 )	256	NCHW	TVM	Fallback	RV32GCV	False	Loop+SLP
`31529514` ( 0.5x )	`107730` ( 1.266 )	`59536` ( 3.097 )	512	NCHW	TVM	Fallback	RV32GCV	False	Loop+SLP
`31529514` ( 0.5x )	`107730` ( 1.266 )	`59536` ( 3.097 )	1024	NCHW	TVM	Fallback	RV32GCV	False	Loop+SLP
`31529514` ( 0.5x )	`107730` ( 1.266 )	`59536` ( 3.097 )	2048	NCHW	TVM	Fallback	RV32GCV	False	Loop+SLP
`31529514` ( 0.5x )	`107730` ( 1.266 )	`59536` ( 3.097 )	4096	NCHW	TVM	Fallback	RV32GCV	False	Loop+SLP
`29578958` ( 0.6x )	`101832` ( 1.197 )	`59536` ( 3.097 )	128	NHWC	TVM	Fallback	RV32GCV	False	Loop+SLP
`29578958` ( 0.6x )	`101832` ( 1.197 )	`59536` ( 3.097 )	256	NHWC	TVM	Fallback	RV32GCV	False	Loop+SLP
`29578958` ( 0.6x )	`101832` ( 1.197 )	`59536` ( 3.097 )	512	NHWC	TVM	Fallback	RV32GCV	False	Loop+SLP
`29578958` ( 0.6x )	`101832` ( 1.197 )	`59536` ( 3.097 )	1024	NHWC	TVM	Fallback	RV32GCV	False	Loop+SLP
`29578958` ( 0.6x )	`101832` ( 1.197 )	`59536` ( 3.097 )	2048	NHWC	TVM	Fallback	RV32GCV	False	Loop+SLP
`29578958` ( 0.6x )	`101832` ( 1.197 )	`59536` ( 3.097 )	4096	NHWC	TVM	Fallback	RV32GCV	False	Loop+SLP
`15873104` ( 1.1x )	`101288` ( 1.19 )	`51364` ( 2.672 )	128	NCHW	TVM	Autotuned	RV32GCV	False	Loop+SLP
`15873104` ( 1.1x )	`101288` ( 1.19 )	`51364` ( 2.672 )	256	NCHW	TVM	Autotuned	RV32GCV	False	Loop+SLP
`15873104` ( 1.1x )	`101286` ( 1.19 )	`51364` ( 2.672 )	512	NCHW	TVM	Autotuned	RV32GCV	False	Loop+SLP
`15873104` ( 1.1x )	`101288` ( 1.19 )	`51364` ( 2.672 )	1024	NCHW	TVM	Autotuned	RV32GCV	False	Loop+SLP
`15873104` ( 1.1x )	`101286` ( 1.19 )	`51364` ( 2.672 )	2048	NCHW	TVM	Autotuned	RV32GCV	False	Loop+SLP
`15873104` ( 1.1x )	`101286` ( 1.19 )	`51364` ( 2.672 )	4096	NCHW	TVM	Autotuned	RV32GCV	False	Loop+SLP
`29574345` ( 0.6x )	`101974` ( 1.198 )	`59536` ( 3.097 )	128	NHWC	TVM	Autotuned	RV32GCV	False	Loop+SLP
`29574345` ( 0.6x )	`101974` ( 1.198 )	`59536` ( 3.097 )	256	NHWC	TVM	Autotuned	RV32GCV	False	Loop+SLP
`29574345` ( 0.6x )	`101974` ( 1.198 )	`59536` ( 3.097 )	512	NHWC	TVM	Autotuned	RV32GCV	False	Loop+SLP
`29574345` ( 0.6x )	`101974` ( 1.198 )	`59536` ( 3.097 )	1024	NHWC	TVM	Autotuned	RV32GCV	False	Loop+SLP
`29574345` ( 0.6x )	`101974` ( 1.198 )	`59536` ( 3.097 )	2048	NHWC	TVM	Autotuned	RV32GCV	False	Loop+SLP
`29574345` ( 0.6x )	`101974` ( 1.198 )	`59536` ( 3.097 )	4096	NHWC	TVM	Autotuned	RV32GCV	False	Loop+SLP
`28034895` ( 0.6x )	`119608` ( 1.406 )	`59484` ( 3.094 )	128	NCHW	TVM	Fallback	RV32GCP	False	-
`26289773` ( 0.6x )	`113910` ( 1.339 )	`59484` ( 3.094 )	128	NHWC	TVM	Fallback	RV32GCP	False	-
`12897999` ( 1.3x )	`113210` ( 1.331 )	`51312` ( 2.669 )	128	NCHW	TVM	Autotuned	RV32GCP	False	-
`26285154` ( 0.6x )	`114032` ( 1.34 )	`59484` ( 3.094 )	128	NHWC	TVM	Autotuned	RV32GCP	False	-
`16859465` ( Base )	`85088` ( Base )	`19224` ( Base )	128	NHWC	muRISCV-NN	Scalar	RV32GC	False	-
`16859465` ( 1.0x )	`85088` ( 1.0 )	`19224` ( 1.0 )	128	NHWC	muRISCV-NN	Scalar	RV32GCV	False	Loop+SLP
`16859465` ( 1.0x )	`85088` ( 1.0 )	`19224` ( 1.0 )	256	NHWC	muRISCV-NN	Scalar	RV32GCV	False	Loop+SLP
`16859465` ( 1.0x )	`85088` ( 1.0 )	`19224` ( 1.0 )	512	NHWC	muRISCV-NN	Scalar	RV32GCV	False	Loop+SLP
`16859465` ( 1.0x )	`85088` ( 1.0 )	`19224` ( 1.0 )	1024	NHWC	muRISCV-NN	Scalar	RV32GCV	False	Loop+SLP
`16859465` ( 1.0x )	`85088` ( 1.0 )	`19224` ( 1.0 )	2048	NHWC	muRISCV-NN	Scalar	RV32GCV	False	Loop+SLP
`16859465` ( 1.0x )	`85088` ( 1.0 )	`19224` ( 1.0 )	4096	NHWC	muRISCV-NN	Scalar	RV32GCV	False	Loop+SLP
`5986504` ( 2.8x )	`86224` ( 1.013 )	`23688` ( 1.232 )	128	NHWC	muRISCV-NN	Vector	RV32GCV	False	-
`4841392` ( 3.5x )	`86224` ( 1.013 )	`23688` ( 1.232 )	256	NHWC	muRISCV-NN	Vector	RV32GCV	False	-
`4273728` ( 3.9x )	`86224` ( 1.013 )	`23688` ( 1.232 )	512	NHWC	muRISCV-NN	Vector	RV32GCV	False	-
`4232016` ( 4.0x )	`86224` ( 1.013 )	`23688` ( 1.232 )	1024	NHWC	muRISCV-NN	Vector	RV32GCV	False	-
`4232016` ( 4.0x )	`86224` ( 1.013 )	`23688` ( 1.232 )	2048	NHWC	muRISCV-NN	Vector	RV32GCV	False	-
`4235405` ( 4.0x )	`86224` ( 1.013 )	`23688` ( 1.232 )	4096	NHWC	muRISCV-NN	Vector	RV32GCV	False	-
`13443818` ( 1.3x )	`97022` ( 1.14 )	`19172` ( 0.997 )	128	NHWC	muRISCV-NN	Scalar	RV32GCP	False	-
`15707735` ( 1.1x )	`99210` ( 1.166 )	`20324` ( 1.057 )	128	NHWC	muRISCV-NN	Packed	RV32GCP	False	-

Image Classification (`resnet`)

Cycles (Speedup)	Total ROM (rel.)	Total RAM (rel.)	VLEN	Layout	Kernels	Mode	Arch	Unroll	Auto-Vectorization
`132014570` ( 0.6x )	`217002` ( 1.615 )	`108448` ( 1.953 )	128	NCHW	TVM	Fallback	RV32GC	False	-
`115757507` ( 0.7x )	`207908` ( 1.548 )	`108448` ( 1.953 )	128	NHWC	TVM	Fallback	RV32GC	False	-
`57193829` ( 1.4x )	`211090` ( 1.571 )	`92264` ( 1.662 )	128	NCHW	TVM	Autotuned	RV32GC	False	-
`115753589` ( 0.7x )	`208014` ( 1.548 )	`108448` ( 1.953 )	128	NHWC	TVM	Autotuned	RV32GC	False	-
`132147940` ( 0.6x )	`216992` ( 1.615 )	`108448` ( 1.953 )	128	NCHW	TVM	Fallback	RV32GCV	False	Loop+SLP
`132147940` ( 0.6x )	`216994` ( 1.615 )	`108448` ( 1.953 )	256	NCHW	TVM	Fallback	RV32GCV	False	Loop+SLP
`132147940` ( 0.6x )	`216992` ( 1.615 )	`108448` ( 1.953 )	512	NCHW	TVM	Fallback	RV32GCV	False	Loop+SLP
`132147940` ( 0.6x )	`216994` ( 1.615 )	`108448` ( 1.953 )	1024	NCHW	TVM	Fallback	RV32GCV	False	Loop+SLP
`132147940` ( 0.6x )	`216992` ( 1.615 )	`108448` ( 1.953 )	2048	NCHW	TVM	Fallback	RV32GCV	False	Loop+SLP
`132147940` ( 0.6x )	`216992` ( 1.615 )	`108448` ( 1.953 )	4096	NCHW	TVM	Fallback	RV32GCV	False	Loop+SLP
`115757501` ( 0.7x )	`207908` ( 1.548 )	`108448` ( 1.953 )	128	NHWC	TVM	Fallback	RV32GCV	False	Loop+SLP
`115757501` ( 0.7x )	`207908` ( 1.548 )	`108448` ( 1.953 )	256	NHWC	TVM	Fallback	RV32GCV	False	Loop+SLP
`115757501` ( 0.7x )	`207908` ( 1.548 )	`108448` ( 1.953 )	512	NHWC	TVM	Fallback	RV32GCV	False	Loop+SLP
`115757501` ( 0.7x )	`207908` ( 1.548 )	`108448` ( 1.953 )	1024	NHWC	TVM	Fallback	RV32GCV	False	Loop+SLP
`115757501` ( 0.7x )	`207908` ( 1.548 )	`108448` ( 1.953 )	2048	NHWC	TVM	Fallback	RV32GCV	False	Loop+SLP
`115757501` ( 0.7x )	`207906` ( 1.548 )	`108448` ( 1.953 )	4096	NHWC	TVM	Fallback	RV32GCV	False	Loop+SLP
`57193823` ( 1.4x )	`211096` ( 1.571 )	`92264` ( 1.662 )	128	NCHW	TVM	Autotuned	RV32GCV	False	Loop+SLP
`57193823` ( 1.4x )	`211098` ( 1.571 )	`92264` ( 1.662 )	256	NCHW	TVM	Autotuned	RV32GCV	False	Loop+SLP
`57193823` ( 1.4x )	`211098` ( 1.571 )	`92264` ( 1.662 )	512	NCHW	TVM	Autotuned	RV32GCV	False	Loop+SLP
`57193823` ( 1.4x )	`211096` ( 1.571 )	`92264` ( 1.662 )	1024	NCHW	TVM	Autotuned	RV32GCV	False	Loop+SLP
`57193823` ( 1.4x )	`211096` ( 1.571 )	`92264` ( 1.662 )	2048	NCHW	TVM	Autotuned	RV32GCV	False	Loop+SLP
`57193823` ( 1.4x )	`211096` ( 1.571 )	`92264` ( 1.662 )	4096	NCHW	TVM	Autotuned	RV32GCV	False	Loop+SLP
`115753583` ( 0.7x )	`208014` ( 1.548 )	`108448` ( 1.953 )	128	NHWC	TVM	Autotuned	RV32GCV	False	Loop+SLP
`115753583` ( 0.7x )	`208012` ( 1.548 )	`108448` ( 1.953 )	256	NHWC	TVM	Autotuned	RV32GCV	False	Loop+SLP
`115753583` ( 0.7x )	`208012` ( 1.548 )	`108448` ( 1.953 )	512	NHWC	TVM	Autotuned	RV32GCV	False	Loop+SLP
`115753583` ( 0.7x )	`208012` ( 1.548 )	`108448` ( 1.953 )	1024	NHWC	TVM	Autotuned	RV32GCV	False	Loop+SLP
`115753583` ( 0.7x )	`208014` ( 1.548 )	`108448` ( 1.953 )	2048	NHWC	TVM	Autotuned	RV32GCV	False	Loop+SLP
`115753583` ( 0.7x )	`208014` ( 1.548 )	`108448` ( 1.953 )	4096	NHWC	TVM	Autotuned	RV32GCV	False	Loop+SLP
`134393924` ( 0.6x )	`229126` ( 1.706 )	`108396` ( 1.952 )	128	NCHW	TVM	Fallback	RV32GCP	False	-
`102349260` ( 0.8x )	`219946` ( 1.637 )	`108396` ( 1.952 )	128	NHWC	TVM	Fallback	RV32GCP	False	-
`44503696` ( 1.8x )	`222790` ( 1.658 )	`92212` ( 1.661 )	128	NCHW	TVM	Autotuned	RV32GCP	False	-
`102345331` ( 0.8x )	`220022` ( 1.638 )	`108396` ( 1.952 )	128	NHWC	TVM	Autotuned	RV32GCP	False	-
`81013966` ( Base )	`134334` ( Base )	`55528` ( Base )	128	NHWC	muRISCV-NN	Scalar	RV32GC	False	-
`81013966` ( 1.0x )	`134334` ( 1.0 )	`55528` ( 1.0 )	128	NHWC	muRISCV-NN	Scalar	RV32GCV	False	Loop+SLP
`81013966` ( 1.0x )	`134334` ( 1.0 )	`55528` ( 1.0 )	256	NHWC	muRISCV-NN	Scalar	RV32GCV	False	Loop+SLP
`81013966` ( 1.0x )	`134334` ( 1.0 )	`55528` ( 1.0 )	512	NHWC	muRISCV-NN	Scalar	RV32GCV	False	Loop+SLP
`81013966` ( 1.0x )	`134334` ( 1.0 )	`55528` ( 1.0 )	1024	NHWC	muRISCV-NN	Scalar	RV32GCV	False	Loop+SLP
`81013966` ( 1.0x )	`134334` ( 1.0 )	`55528` ( 1.0 )	2048	NHWC	muRISCV-NN	Scalar	RV32GCV	False	Loop+SLP
`81013966` ( 1.0x )	`134334` ( 1.0 )	`55528` ( 1.0 )	4096	NHWC	muRISCV-NN	Scalar	RV32GCV	False	Loop+SLP
`15492115` ( 5.2x )	`135824` ( 1.011 )	`55528` ( 1.0 )	128	NHWC	muRISCV-NN	Vector	RV32GCV	False	-
`9805107` ( 8.3x )	`135824` ( 1.011 )	`55528` ( 1.0 )	256	NHWC	muRISCV-NN	Vector	RV32GCV	False	-
`7211459` ( 11.2x )	`135824` ( 1.011 )	`55528` ( 1.0 )	512	NHWC	muRISCV-NN	Vector	RV32GCV	False	-
`5945939` ( 13.6x )	`135824` ( 1.011 )	`55528` ( 1.0 )	1024	NHWC	muRISCV-NN	Vector	RV32GCV	False	-
`5004248` ( 16.2x )	`135824` ( 1.011 )	`55528` ( 1.0 )	2048	NHWC	muRISCV-NN	Vector	RV32GCV	False	-
`4753721` ( 17.0x )	`135824` ( 1.011 )	`55528` ( 1.0 )	4096	NHWC	muRISCV-NN	Vector	RV32GCV	False	-
`62991356` ( 1.3x )	`146268` ( 1.089 )	`55476` ( 0.999 )	128	NHWC	muRISCV-NN	Scalar	RV32GCP	False	-
`68441625` ( 1.2x )	`149086` ( 1.11 )	`55476` ( 0.999 )	128	NHWC	muRISCV-NN	Packed	RV32GCP	False	-

Anomaly Detection (`toycar`)

Cycles (Speedup)	Total ROM (rel.)	Total RAM (rel.)	VLEN	Layout	Kernels	Mode	Arch	Unroll	Auto-Vectorization
`3230152` ( 0.6x )	`580142` ( 1.839 )	`5580` ( 1.167 )	128	NCHW	TVM	Fallback	RV32GC	False	-
`3230152` ( 0.6x )	`580142` ( 1.839 )	`5580` ( 1.167 )	128	NHWC	TVM	Fallback	RV32GC	False	-
`2147464` ( 0.8x )	`609314` ( 1.932 )	`6892` ( 1.442 )	128	NCHW	TVM	Autotuned	RV32GC	False	-
`2147464` ( 0.8x )	`609314` ( 1.932 )	`6892` ( 1.442 )	128	NHWC	TVM	Autotuned	RV32GC	False	-
`3230152` ( 0.6x )	`580130` ( 1.839 )	`5580` ( 1.167 )	128	NCHW	TVM	Fallback	RV32GCV	False	Loop+SLP
`3230152` ( 0.6x )	`580130` ( 1.839 )	`5580` ( 1.167 )	256	NCHW	TVM	Fallback	RV32GCV	False	Loop+SLP
`3230152` ( 0.6x )	`580130` ( 1.839 )	`5580` ( 1.167 )	512	NCHW	TVM	Fallback	RV32GCV	False	Loop+SLP
`3230152` ( 0.6x )	`580130` ( 1.839 )	`5580` ( 1.167 )	1024	NCHW	TVM	Fallback	RV32GCV	False	Loop+SLP
`3230152` ( 0.6x )	`580130` ( 1.839 )	`5580` ( 1.167 )	2048	NCHW	TVM	Fallback	RV32GCV	False	Loop+SLP
`3230152` ( 0.6x )	`580130` ( 1.839 )	`5580` ( 1.167 )	4096	NCHW	TVM	Fallback	RV32GCV	False	Loop+SLP
`3230152` ( 0.6x )	`580130` ( 1.839 )	`5580` ( 1.167 )	128	NHWC	TVM	Fallback	RV32GCV	False	Loop+SLP
`3230152` ( 0.6x )	`580130` ( 1.839 )	`5580` ( 1.167 )	256	NHWC	TVM	Fallback	RV32GCV	False	Loop+SLP
`3230152` ( 0.6x )	`580130` ( 1.839 )	`5580` ( 1.167 )	512	NHWC	TVM	Fallback	RV32GCV	False	Loop+SLP
`3230152` ( 0.6x )	`580130` ( 1.839 )	`5580` ( 1.167 )	1024	NHWC	TVM	Fallback	RV32GCV	False	Loop+SLP
`3230152` ( 0.6x )	`580130` ( 1.839 )	`5580` ( 1.167 )	2048	NHWC	TVM	Fallback	RV32GCV	False	Loop+SLP
`3230152` ( 0.6x )	`580130` ( 1.839 )	`5580` ( 1.167 )	4096	NHWC	TVM	Fallback	RV32GCV	False	Loop+SLP
`2147464` ( 0.8x )	`609314` ( 1.932 )	`6892` ( 1.442 )	128	NCHW	TVM	Autotuned	RV32GCV	False	Loop+SLP
`2147464` ( 0.8x )	`609314` ( 1.932 )	`6892` ( 1.442 )	256	NCHW	TVM	Autotuned	RV32GCV	False	Loop+SLP
`2147464` ( 0.8x )	`609314` ( 1.932 )	`6892` ( 1.442 )	512	NCHW	TVM	Autotuned	RV32GCV	False	Loop+SLP
`2147464` ( 0.8x )	`609314` ( 1.932 )	`6892` ( 1.442 )	1024	NCHW	TVM	Autotuned	RV32GCV	False	Loop+SLP
`2147464` ( 0.8x )	`609314` ( 1.932 )	`6892` ( 1.442 )	2048	NCHW	TVM	Autotuned	RV32GCV	False	Loop+SLP
`2147464` ( 0.8x )	`609314` ( 1.932 )	`6892` ( 1.442 )	4096	NCHW	TVM	Autotuned	RV32GCV	False	Loop+SLP
`2147464` ( 0.8x )	`609314` ( 1.932 )	`6892` ( 1.442 )	128	NHWC	TVM	Autotuned	RV32GCV	False	Loop+SLP
`2147464` ( 0.8x )	`609314` ( 1.932 )	`6892` ( 1.442 )	256	NHWC	TVM	Autotuned	RV32GCV	False	Loop+SLP
`2147464` ( 0.8x )	`609314` ( 1.932 )	`6892` ( 1.442 )	512	NHWC	TVM	Autotuned	RV32GCV	False	Loop+SLP
`2147464` ( 0.8x )	`609314` ( 1.932 )	`6892` ( 1.442 )	1024	NHWC	TVM	Autotuned	RV32GCV	False	Loop+SLP
`2147464` ( 0.8x )	`609314` ( 1.932 )	`6892` ( 1.442 )	2048	NHWC	TVM	Autotuned	RV32GCV	False	Loop+SLP
`2147464` ( 0.8x )	`609314` ( 1.932 )	`6892` ( 1.442 )	4096	NHWC	TVM	Autotuned	RV32GCV	False	Loop+SLP
`2867860` ( 0.6x )	`592550` ( 1.879 )	`5528` ( 1.156 )	128	NCHW	TVM	Fallback	RV32GCP	False	-
`2867860` ( 0.6x )	`592550` ( 1.879 )	`5528` ( 1.156 )	128	NHWC	TVM	Fallback	RV32GCP	False	-
`1769706` ( 1.0x )	`618794` ( 1.962 )	`6840` ( 1.431 )	128	NCHW	TVM	Autotuned	RV32GCP	False	-
`1769706` ( 1.0x )	`618794` ( 1.962 )	`6840` ( 1.431 )	128	NHWC	TVM	Autotuned	RV32GCP	False	-
`1820395` ( Base )	`315404` ( Base )	`4780` ( Base )	128	NHWC	muRISCV-NN	Scalar	RV32GC	False	-
`1820395` ( 1.0x )	`315404` ( 1.0 )	`4780` ( 1.0 )	128	NHWC	muRISCV-NN	Scalar	RV32GCV	False	Loop+SLP
`1820395` ( 1.0x )	`315404` ( 1.0 )	`4780` ( 1.0 )	256	NHWC	muRISCV-NN	Scalar	RV32GCV	False	Loop+SLP
`1820393` ( 1.0x )	`315398` ( 1.0 )	`4780` ( 1.0 )	512	NHWC	muRISCV-NN	Scalar	RV32GCV	False	Loop+SLP
`1820393` ( 1.0x )	`315398` ( 1.0 )	`4780` ( 1.0 )	1024	NHWC	muRISCV-NN	Scalar	RV32GCV	False	Loop+SLP
`1820395` ( 1.0x )	`315404` ( 1.0 )	`4780` ( 1.0 )	2048	NHWC	muRISCV-NN	Scalar	RV32GCV	False	Loop+SLP
`1820395` ( 1.0x )	`315404` ( 1.0 )	`4780` ( 1.0 )	4096	NHWC	muRISCV-NN	Scalar	RV32GCV	False	Loop+SLP
`627197` ( 2.9x )	`315804` ( 1.001 )	`4780` ( 1.0 )	128	NHWC	muRISCV-NN	Vector	RV32GCV	False	-
`508827` ( 3.6x )	`315798` ( 1.001 )	`4780` ( 1.0 )	256	NHWC	muRISCV-NN	Vector	RV32GCV	False	-
`449645` ( 4.0x )	`315804` ( 1.001 )	`4780` ( 1.0 )	512	NHWC	muRISCV-NN	Vector	RV32GCV	False	-
`420389` ( 4.3x )	`315804` ( 1.001 )	`4780` ( 1.0 )	1024	NHWC	muRISCV-NN	Vector	RV32GCV	False	-
`416705` ( 4.4x )	`315804` ( 1.001 )	`4780` ( 1.0 )	2048	NHWC	muRISCV-NN	Vector	RV32GCV	False	-
`414819` ( 4.4x )	`315798` ( 1.001 )	`4780` ( 1.0 )	4096	NHWC	muRISCV-NN	Vector	RV32GCV	False	-
`1656914` ( 1.1x )	`327622` ( 1.039 )	`4728` ( 0.989 )	128	NHWC	muRISCV-NN	Scalar	RV32GCP	False	-
`996966` ( 1.8x )	`327840` ( 1.039 )	`4728` ( 0.989 )	128	NHWC	muRISCV-NN	Packed	RV32GCP	False	-

Visual Wake Words (`vww`)

Cycles (Speedup)	Total ROM (rel.)	Total RAM (rel.)	VLEN	Layout	Kernels	Mode	Arch	Unroll	Auto-Vectorization
`91779188` ( 0.5x )	`540914` ( 1.696 )	`181056` ( 2.113 )	128	NCHW	TVM	Fallback	RV32GC	False	-
`86076684` ( 0.6x )	`520542` ( 1.632 )	`181056` ( 2.113 )	128	NHWC	TVM	Fallback	RV32GC	False	-
`46542367` ( 1.1x )	`521918` ( 1.637 )	`181056` ( 2.113 )	128	NCHW	TVM	Autotuned	RV32GC	False	-
`86076687` ( 0.6x )	`520544` ( 1.632 )	`181056` ( 2.113 )	128	NHWC	TVM	Autotuned	RV32GC	False	-
`91775924` ( 0.5x )	`540822` ( 1.696 )	`181056` ( 2.113 )	128	NCHW	TVM	Fallback	RV32GCV	False	Loop+SLP
`91775924` ( 0.5x )	`540820` ( 1.696 )	`181056` ( 2.113 )	256	NCHW	TVM	Fallback	RV32GCV	False	Loop+SLP
`91775924` ( 0.5x )	`540820` ( 1.696 )	`181056` ( 2.113 )	512	NCHW	TVM	Fallback	RV32GCV	False	Loop+SLP
`91775924` ( 0.5x )	`540820` ( 1.696 )	`181056` ( 2.113 )	1024	NCHW	TVM	Fallback	RV32GCV	False	Loop+SLP
`91775924` ( 0.5x )	`540820` ( 1.696 )	`181056` ( 2.113 )	2048	NCHW	TVM	Fallback	RV32GCV	False	Loop+SLP
`91775924` ( 0.5x )	`540822` ( 1.696 )	`181056` ( 2.113 )	4096	NCHW	TVM	Fallback	RV32GCV	False	Loop+SLP
`86076684` ( 0.6x )	`520542` ( 1.632 )	`181056` ( 2.113 )	128	NHWC	TVM	Fallback	RV32GCV	False	Loop+SLP
`86076684` ( 0.6x )	`520542` ( 1.632 )	`181056` ( 2.113 )	256	NHWC	TVM	Fallback	RV32GCV	False	Loop+SLP
`86076684` ( 0.6x )	`520542` ( 1.632 )	`181056` ( 2.113 )	512	NHWC	TVM	Fallback	RV32GCV	False	Loop+SLP
`86076684` ( 0.6x )	`520542` ( 1.632 )	`181056` ( 2.113 )	1024	NHWC	TVM	Fallback	RV32GCV	False	Loop+SLP
`86076684` ( 0.6x )	`520542` ( 1.632 )	`181056` ( 2.113 )	2048	NHWC	TVM	Fallback	RV32GCV	False	Loop+SLP
`86076684` ( 0.6x )	`520542` ( 1.632 )	`181056` ( 2.113 )	4096	NHWC	TVM	Fallback	RV32GCV	False	Loop+SLP
`46557547` ( 1.1x )	`521910` ( 1.637 )	`181056` ( 2.113 )	128	NCHW	TVM	Autotuned	RV32GCV	False	Loop+SLP
`46557547` ( 1.1x )	`521912` ( 1.637 )	`181056` ( 2.113 )	256	NCHW	TVM	Autotuned	RV32GCV	False	Loop+SLP
`46557547` ( 1.1x )	`521910` ( 1.637 )	`181056` ( 2.113 )	512	NCHW	TVM	Autotuned	RV32GCV	False	Loop+SLP
`46557547` ( 1.1x )	`521912` ( 1.637 )	`181056` ( 2.113 )	1024	NCHW	TVM	Autotuned	RV32GCV	False	Loop+SLP
`46557547` ( 1.1x )	`521910` ( 1.637 )	`181056` ( 2.113 )	2048	NCHW	TVM	Autotuned	RV32GCV	False	Loop+SLP
`46557547` ( 1.1x )	`521912` ( 1.637 )	`181056` ( 2.113 )	4096	NCHW	TVM	Autotuned	RV32GCV	False	Loop+SLP
`86076687` ( 0.6x )	`520544` ( 1.632 )	`181056` ( 2.113 )	128	NHWC	TVM	Autotuned	RV32GCV	False	Loop+SLP
`86076687` ( 0.6x )	`520544` ( 1.632 )	`181056` ( 2.113 )	256	NHWC	TVM	Autotuned	RV32GCV	False	Loop+SLP
`86076687` ( 0.6x )	`520544` ( 1.632 )	`181056` ( 2.113 )	512	NHWC	TVM	Autotuned	RV32GCV	False	Loop+SLP
`86076687` ( 0.6x )	`520544` ( 1.632 )	`181056` ( 2.113 )	1024	NHWC	TVM	Autotuned	RV32GCV	False	Loop+SLP
`86076687` ( 0.6x )	`520544` ( 1.632 )	`181056` ( 2.113 )	2048	NHWC	TVM	Autotuned	RV32GCV	False	Loop+SLP
`86076687` ( 0.6x )	`520544` ( 1.632 )	`181056` ( 2.113 )	4096	NHWC	TVM	Autotuned	RV32GCV	False	Loop+SLP
`81680879` ( 0.6x )	`551532` ( 1.729 )	`181004` ( 2.113 )	128	NCHW	TVM	Fallback	RV32GCP	False	-
`76410709` ( 0.7x )	`531796` ( 1.668 )	`181004` ( 2.113 )	128	NHWC	TVM	Fallback	RV32GCP	False	-
`37728221` ( 1.3x )	`533010` ( 1.671 )	`181004` ( 2.113 )	128	NCHW	TVM	Autotuned	RV32GCP	False	-
`76410712` ( 0.7x )	`531800` ( 1.668 )	`181004` ( 2.113 )	128	NHWC	TVM	Autotuned	RV32GCP	False	-
`49801853` ( Base )	`318912` ( Base )	`85672` ( Base )	128	NHWC	muRISCV-NN	Scalar	RV32GC	False	-
`49801853` ( 1.0x )	`318912` ( 1.0 )	`85672` ( 1.0 )	128	NHWC	muRISCV-NN	Scalar	RV32GCV	False	Loop+SLP
`49801853` ( 1.0x )	`318912` ( 1.0 )	`85672` ( 1.0 )	256	NHWC	muRISCV-NN	Scalar	RV32GCV	False	Loop+SLP
`49801853` ( 1.0x )	`318912` ( 1.0 )	`85672` ( 1.0 )	512	NHWC	muRISCV-NN	Scalar	RV32GCV	False	Loop+SLP
`49801853` ( 1.0x )	`318912` ( 1.0 )	`85672` ( 1.0 )	1024	NHWC	muRISCV-NN	Scalar	RV32GCV	False	Loop+SLP
`49801853` ( 1.0x )	`318912` ( 1.0 )	`85672` ( 1.0 )	2048	NHWC	muRISCV-NN	Scalar	RV32GCV	False	Loop+SLP
`49801853` ( 1.0x )	`318912` ( 1.0 )	`85672` ( 1.0 )	4096	NHWC	muRISCV-NN	Scalar	RV32GCV	False	Loop+SLP
`13603234` ( 3.7x )	`320344` ( 1.004 )	`85672` ( 1.0 )	128	NHWC	muRISCV-NN	Vector	RV32GCV	False	-
`10274606` ( 4.8x )	`320344` ( 1.004 )	`85672` ( 1.0 )	256	NHWC	muRISCV-NN	Vector	RV32GCV	False	-
`8985484` ( 5.5x )	`320344` ( 1.004 )	`85672` ( 1.0 )	512	NHWC	muRISCV-NN	Vector	RV32GCV	False	-
`8480985` ( 5.9x )	`320344` ( 1.004 )	`85672` ( 1.0 )	1024	NHWC	muRISCV-NN	Vector	RV32GCV	False	-
`8432371` ( 5.9x )	`320344` ( 1.004 )	`85672` ( 1.0 )	2048	NHWC	muRISCV-NN	Vector	RV32GCV	False	-
`8435760` ( 5.9x )	`320344` ( 1.004 )	`85672` ( 1.0 )	4096	NHWC	muRISCV-NN	Vector	RV32GCV	False	-
`40880384` ( 1.2x )	`330632` ( 1.037 )	`85620` ( 0.999 )	128	NHWC	muRISCV-NN	Scalar	RV32GCP	False	-
`49304041` ( 1.0x )	`333076` ( 1.044 )	`85620` ( 0.999 )	128	NHWC	muRISCV-NN	Packed	RV32GCP	False	-

Original data

Click here to download the raw files for this benchmark.

2024-11-26

TFLM

GCC

O3

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

Os

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

LLVM

O3

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

Os

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

TVM

GCC

O3

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

Os

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

LLVM

O3

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

Os

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

2024-11-21

TFLM

GCC

O3

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

Os

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

LLVM

O3

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

Os

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

TVM

GCC

O3

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

Os

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

LLVM

O3

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

Os

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

2024-11-19

TFLM

GCC

Os

Spike (RV32) (spike_rv32)

Spike (RV64) (spike_rv64)

LLVM

Os

Spike (RV32) (spike_rv32)

Spike (RV64) (spike_rv64)

TVM

GCC

Os

Spike (RV32) (spike_rv32)

Spike (RV64) (spike_rv64)

LLVM

Os

Spike (RV32) (spike_rv32)

Spike (RV64) (spike_rv64)

2024-11-18

TFLM

GCC

Os

Spike (RV32) (spike_rv32)

Spike (RV64) (spike_rv64)

LLVM

Os

Spike (RV32) (spike_rv32)

Spike (RV64) (spike_rv64)

TVM

GCC

Os

Spike (RV32) (spike_rv32)

Spike (RV64) (spike_rv64)

LLVM

Os

Spike (RV32) (spike_rv32)

Spike (RV64) (spike_rv64)

2024-07-12

TFLM

GCC

Os

Spike (RV64) (spike_rv64)

LLVM

Os

Spike (RV64) (spike_rv64)

TVM

GCC

Os

Spike (RV64) (spike_rv64)

LLVM

Os

Spike (RV64) (spike_rv64)

2024-06-29

TFLM

GCC

Os

SpikeRV32

LLVM

Os

SpikeRV32

TVM

GCC

Os

SpikeRV32

LLVM

Os

SpikeRV32

2024-03-02

TFLM

GCC

O3

SpikeRV32

Os

SpikeRV32

LLVM

O3

SpikeRV32

Os

SpikeRV32

TVM

GCC

O3

SpikeRV32

Os

SpikeRV32

LLVM

O3

SpikeRV32

Os

SpikeRV32

2024-02-26

TFLM

GCC

s

SpikeRV32

LLVM

s

SpikeRV32

TVM

GCC

s

SpikeRV32

LLVM

s

SpikeRV32

2024-02-23

TFLM

GCC

s

SpikeRV32

LLVM

s

SpikeRV32

TVM

GCC

s

SpikeRV32

LLVM

s

SpikeRV32

2024-02-22

TFLM

GCC

s

SpikeRV32

LLVM

s

SpikeRV32

TVM

GCC

s

SpikeRV32

LLVM

s

SpikeRV32

2024-02-20

TFLM

GCC

s

SpikeRV32

LLVM

s

SpikeRV32

TVM

GCC

s

SpikeRV32

LLVM

s

SpikeRV32

2024-02-11

TFLM

GCC

s

SpikeRV32

LLVM

s

SpikeRV32

TVM

GCC

s

SpikeRV32

LLVM

s

SpikeRV32

2023-12-22

TFLM

GCC

s

SpikeRV32

TVM

GCC

s

SpikeRV32

2024-11-26

TFLM

GCC

O3

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

Os

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

LLVM

O3

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

Os

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

TVM

GCC

O3

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

Os

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

LLVM

O3

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

Os

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

2024-11-21

TFLM

GCC

O3

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

Os

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

LLVM

O3

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

Os

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

TVM

GCC

O3

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

Os

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

LLVM

O3

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

Os

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

2024-11-19

TFLM

GCC

Os

Spike (RV32) (spike_rv32)

Spike (RV64) (spike_rv64)

LLVM

Os

Spike (RV32) (spike_rv32)

Spike (RV64) (spike_rv64)

TVM

GCC

Os

Spike (RV32) (spike_rv32)

Spike (RV64) (spike_rv64)

LLVM

Os

Spike (RV32) (spike_rv32)

Spike (RV64) (spike_rv64)

2024-11-18

TFLM

GCC

Os

Spike (RV32) (spike_rv32)

Spike (RV64) (spike_rv64)

LLVM

Os

Spike (RV32) (spike_rv32)

Spike (RV64) (spike_rv64)

TVM

GCC

Os

Spike (RV32) (spike_rv32)

Spike (RV64) (spike_rv64)

LLVM

Os

Spike (RV32) (spike_rv32)

Spike (RV64) (spike_rv64)

2024-07-12

TFLM

GCC

Os

Spike (RV64) (spike_rv64)

LLVM

Os

Spike (RV64) (spike_rv64)

TVM

GCC

Os

Spike (RV64) (spike_rv64)

LLVM

Os

Spike (RV64) (spike_rv64)

2024-06-29

TFLM

GCC

Os

SpikeRV32

LLVM

Os

SpikeRV32

TVM

GCC

Os

SpikeRV32

LLVM

Os

SpikeRV32

2024-03-02

TFLM

GCC

O3

SpikeRV32

Os

SpikeRV32

LLVM

O3

SpikeRV32

Os

SpikeRV32

TVM

GCC

O3

SpikeRV32

Os

SpikeRV32

LLVM

O3

SpikeRV32

Os

SpikeRV32

2024-02-26

TFLM

GCC

s

SpikeRV32

LLVM

s

SpikeRV32

TVM

GCC

s

SpikeRV32

LLVM

s

SpikeRV32

2024-02-23

TFLM

GCC

s

SpikeRV32

LLVM

s

SpikeRV32

TVM

GCC

s

SpikeRV32

LLVM

s

SpikeRV32

2024-02-22

TFLM

GCC

s

SpikeRV32

LLVM

s

SpikeRV32

TVM

GCC

s

SpikeRV32

LLVM

s

SpikeRV32

2024-02-20

TFLM

GCC

s

SpikeRV32

LLVM

s

SpikeRV32

TVM

GCC

s

SpikeRV32

LLVM

s

SpikeRV32

2024-02-11

TFLM

GCC

s

SpikeRV32

LLVM

s

SpikeRV32

TVM

GCC

s

SpikeRV32

LLVM

s

SpikeRV32

2023-12-22

TFLM

GCC

s

SpikeRV32

TVM

GCC

s

SpikeRV32

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Benchmarks CUSTOM TVM GCC Os

Setup

Simulator

Toolchains

Models

Package Versions

Miscellaneous

Results (Framework: tvm, Backend: tvmaot, Toolchain: gcc, Flags: -Os)

Audio Wake Words (`aww`)

Image Classification (`resnet`)

Anomaly Detection (`toycar`)

Visual Wake Words (`vww`)

Original data

Clone this wiki locally

Benchmarks CUSTOM TVM GCC Os

Setup

Simulator

Toolchains

Models

Package Versions

Miscellaneous

Results (Framework: tvm, Backend: tvmaot, Toolchain: gcc, Flags: -Os)

Audio Wake Words (aww)

Image Classification (resnet)

Anomaly Detection (toycar)

Visual Wake Words (vww)

Original data

Clone this wiki locally

Audio Wake Words (`aww`)

Image Classification (`resnet`)

Anomaly Detection (`toycar`)

Visual Wake Words (`vww`)