Benchmarks 2024 03 02 TVM LLVM O3

Setup

Simulator

Spike (riscv-isa-sim ) (ISS, CPI=1)

Toolchains

LLVM/Clang:
- TODO: Version
- Linker: lld (TODO)
- RISC-V GCC for Headers, libc,...

Models

MLPerfTiny Benchmark
TODO: others!

Package Versions

MLonMCU : main
TVM : Nightly Pre-Build
Spike : 0bc176b3fca43560b9e8586cdbc41cfde073e17a
Spike PK : 7e9b671c0415dfd7b562ac934feb9380075d4aa2

Miscellaneous

Used -Os flag for compilation.
Benchmarks generated using MLonMCU deployment tool with minimal efforts.
Memory metrics are reported in Bytes

Results (Framework: tvm, Backend: tvmaot, Toolchain: llvm, Flags: -O3)

Audio Wake Words (`aww`)

Cycles (Speedup)	Total ROM (rel.)	Total RAM (rel.)	VLEN	Layout	Kernels	Mode	Arch	Auto-Vectorization
`13664442` ( 1.1x )	`128044` ( 1.392 )	`59508` ( 3.097 )	0	NCHW	TVM	Fallback	RV32GC	-
`23395474` ( 0.7x )	`112466` ( 1.222 )	`59508` ( 3.097 )	0	NHWC	TVM	Fallback	RV32GC	-
`12283411` ( 1.3x )	`109692` ( 1.192 )	`51336` ( 2.672 )	0	NCHW	TVM	Autotuned	RV32GC	-
`23395187` ( 0.7x )	`112510` ( 1.223 )	`59508` ( 3.097 )	0	NHWC	TVM	Autotuned	RV32GC	-
`7139459` ( 2.2x )	`128154` ( 1.393 )	`59508` ( 3.097 )	128	NCHW	TVM	Fallback	RV32GCV	Loop+SLP
`7139616` ( 2.2x )	`128154` ( 1.393 )	`59508` ( 3.097 )	256	NCHW	TVM	Fallback	RV32GCV	Loop+SLP
`7143883` ( 2.2x )	`128154` ( 1.393 )	`59508` ( 3.097 )	512	NCHW	TVM	Fallback	RV32GCV	Loop+SLP
`7147202` ( 2.2x )	`128154` ( 1.393 )	`59508` ( 3.097 )	1024	NCHW	TVM	Fallback	RV32GCV	Loop+SLP
`7155530` ( 2.2x )	`128154` ( 1.393 )	`59508` ( 3.097 )	2048	NCHW	TVM	Fallback	RV32GCV	Loop+SLP
`7172475` ( 2.2x )	`128154` ( 1.393 )	`59508` ( 3.097 )	4096	NCHW	TVM	Fallback	RV32GCV	Loop+SLP
`3675716` ( 4.3x )	`134828` ( 1.465 )	`59508` ( 3.097 )	128	NHWC	TVM	Fallback	RV32GCV	Loop+SLP
`2239792` ( 7.0x )	`134828` ( 1.465 )	`59508` ( 3.097 )	256	NHWC	TVM	Fallback	RV32GCV	Loop+SLP
`1522890` ( 10.3x )	`134828` ( 1.465 )	`59508` ( 3.097 )	512	NHWC	TVM	Fallback	RV32GCV	Loop+SLP
`1758669` ( 8.9x )	`134828` ( 1.465 )	`59508` ( 3.097 )	1024	NHWC	TVM	Fallback	RV32GCV	Loop+SLP
`22748654` ( 0.7x )	`134828` ( 1.465 )	`59508` ( 3.097 )	2048	NHWC	TVM	Fallback	RV32GCV	Loop+SLP
`22755432` ( 0.7x )	`134828` ( 1.465 )	`59508` ( 3.097 )	4096	NHWC	TVM	Fallback	RV32GCV	Loop+SLP
`4770885` ( 3.3x )	`113932` ( 1.238 )	`51336` ( 2.672 )	128	NCHW	TVM	Autotuned	RV32GCV	Loop+SLP
`3818945` ( 4.1x )	`113932` ( 1.238 )	`51336` ( 2.672 )	256	NCHW	TVM	Autotuned	RV32GCV	Loop+SLP
`3471225` ( 4.5x )	`113932` ( 1.238 )	`51336` ( 2.672 )	512	NCHW	TVM	Autotuned	RV32GCV	Loop+SLP
`3484756` ( 4.5x )	`113932` ( 1.238 )	`51336` ( 2.672 )	1024	NCHW	TVM	Autotuned	RV32GCV	Loop+SLP
`11401697` ( 1.4x )	`113932` ( 1.238 )	`51336` ( 2.672 )	2048	NCHW	TVM	Autotuned	RV32GCV	Loop+SLP
`11405086` ( 1.4x )	`113932` ( 1.238 )	`51336` ( 2.672 )	4096	NCHW	TVM	Autotuned	RV32GCV	Loop+SLP
`3675556` ( 4.3x )	`135082` ( 1.468 )	`59508` ( 3.097 )	128	NHWC	TVM	Autotuned	RV32GCV	Loop+SLP
`2239696` ( 7.0x )	`135082` ( 1.468 )	`59508` ( 3.097 )	256	NHWC	TVM	Autotuned	RV32GCV	Loop+SLP
`1522826` ( 10.3x )	`135082` ( 1.468 )	`59508` ( 3.097 )	512	NHWC	TVM	Autotuned	RV32GCV	Loop+SLP
`1758621` ( 8.9x )	`135082` ( 1.468 )	`59508` ( 3.097 )	1024	NHWC	TVM	Autotuned	RV32GCV	Loop+SLP
`22749585` ( 0.7x )	`135082` ( 1.468 )	`59508` ( 3.097 )	2048	NHWC	TVM	Autotuned	RV32GCV	Loop+SLP
`22756363` ( 0.7x )	`135082` ( 1.468 )	`59508` ( 3.097 )	4096	NHWC	TVM	Autotuned	RV32GCV	Loop+SLP
`15686016` ( Base )	`92018` ( Base )	`19212` ( Base )	0	NHWC	muRISCV-NN	Scalar	RV32GC	-
`5932120` ( 2.6x )	`99776` ( 1.084 )	`19212` ( 1.0 )	128	NHWC	muRISCV-NN	Scalar	RV32GCV	Loop+SLP
`5317611` ( 2.9x )	`99776` ( 1.084 )	`19212` ( 1.0 )	256	NHWC	muRISCV-NN	Scalar	RV32GCV	Loop+SLP
`5013136` ( 3.1x )	`99776` ( 1.084 )	`19212` ( 1.0 )	512	NHWC	muRISCV-NN	Scalar	RV32GCV	Loop+SLP
`4859204` ( 3.2x )	`99776` ( 1.084 )	`19212` ( 1.0 )	1024	NHWC	muRISCV-NN	Scalar	RV32GCV	Loop+SLP
`13842025` ( 1.1x )	`99776` ( 1.084 )	`19212` ( 1.0 )	2048	NHWC	muRISCV-NN	Scalar	RV32GCV	Loop+SLP
`13845414` ( 1.1x )	`99776` ( 1.084 )	`19212` ( 1.0 )	4096	NHWC	muRISCV-NN	Scalar	RV32GCV	Loop+SLP
`5171345` ( 3.0x )	`93796` ( 1.019 )	`23676` ( 1.232 )	128	NHWC	muRISCV-NN	Vector	RV32GCV	-
`4031599` ( 3.9x )	`93796` ( 1.019 )	`23676` ( 1.232 )	256	NHWC	muRISCV-NN	Vector	RV32GCV	-
`3470329` ( 4.5x )	`93796` ( 1.019 )	`23676` ( 1.232 )	512	NHWC	muRISCV-NN	Vector	RV32GCV	-
`3427845` ( 4.6x )	`93796` ( 1.019 )	`23676` ( 1.232 )	1024	NHWC	muRISCV-NN	Vector	RV32GCV	-
`3431234` ( 4.6x )	`93796` ( 1.019 )	`23676` ( 1.232 )	2048	NHWC	muRISCV-NN	Vector	RV32GCV	-
`3434623` ( 4.6x )	`93796` ( 1.019 )	`23676` ( 1.232 )	4096	NHWC	muRISCV-NN	Vector	RV32GCV	-

Image Classification (`resnet`)

Cycles (Speedup)	Total ROM (rel.)	Total RAM (rel.)	VLEN	Layout	Kernels	Mode	Arch	Auto-Vectorization
`61246489` ( 1.0x )	`233308` ( 1.675 )	`108420` ( 1.953 )	0	NCHW	TVM	Fallback	RV32GC	-
`82672615` ( 0.7x )	`215966` ( 1.55 )	`108420` ( 1.953 )	0	NHWC	TVM	Fallback	RV32GC	-
`52034836` ( 1.1x )	`224974` ( 1.615 )	`92236` ( 1.661 )	0	NCHW	TVM	Autotuned	RV32GC	-
`82672333` ( 0.7x )	`215980` ( 1.551 )	`108420` ( 1.953 )	0	NHWC	TVM	Autotuned	RV32GC	-
`67927841` ( 0.9x )	`277116` ( 1.989 )	`108420` ( 1.953 )	128	NCHW	TVM	Fallback	RV32GCV	Loop+SLP
`67890196` ( 0.9x )	`277116` ( 1.989 )	`108420` ( 1.953 )	256	NCHW	TVM	Fallback	RV32GCV	Loop+SLP
`68445549` ( 0.9x )	`277116` ( 1.989 )	`108420` ( 1.953 )	512	NCHW	TVM	Fallback	RV32GCV	Loop+SLP
`68452277` ( 0.9x )	`277116` ( 1.989 )	`108420` ( 1.953 )	1024	NCHW	TVM	Fallback	RV32GCV	Loop+SLP
`68470539` ( 0.9x )	`277120` ( 1.989 )	`108420` ( 1.953 )	2048	NCHW	TVM	Fallback	RV32GCV	Loop+SLP
`68501040` ( 0.9x )	`277120` ( 1.989 )	`108420` ( 1.953 )	4096	NCHW	TVM	Fallback	RV32GCV	Loop+SLP
`28923316` ( 2.0x )	`225878` ( 1.622 )	`108420` ( 1.953 )	128	NHWC	TVM	Fallback	RV32GCV	Loop+SLP
`25786661` ( 2.3x )	`225878` ( 1.622 )	`108420` ( 1.953 )	256	NHWC	TVM	Fallback	RV32GCV	Loop+SLP
`24262821` ( 2.4x )	`225878` ( 1.622 )	`108420` ( 1.953 )	512	NHWC	TVM	Fallback	RV32GCV	Loop+SLP
`53047914` ( 1.1x )	`225878` ( 1.622 )	`108420` ( 1.953 )	1024	NHWC	TVM	Fallback	RV32GCV	Loop+SLP
`72079578` ( 0.8x )	`225878` ( 1.622 )	`108420` ( 1.953 )	2048	NHWC	TVM	Fallback	RV32GCV	Loop+SLP
`72079578` ( 0.8x )	`225878` ( 1.622 )	`108420` ( 1.953 )	4096	NHWC	TVM	Fallback	RV32GCV	Loop+SLP
`16656626` ( 3.5x )	`253100` ( 1.817 )	`92236` ( 1.661 )	128	NCHW	TVM	Autotuned	RV32GCV	Loop+SLP
`12991895` ( 4.5x )	`253100` ( 1.817 )	`92236` ( 1.661 )	256	NCHW	TVM	Autotuned	RV32GCV	Loop+SLP
`29398137` ( 2.0x )	`253100` ( 1.817 )	`92236` ( 1.661 )	512	NCHW	TVM	Autotuned	RV32GCV	Loop+SLP
`42815949` ( 1.4x )	`253108` ( 1.817 )	`92236` ( 1.661 )	1024	NCHW	TVM	Autotuned	RV32GCV	Loop+SLP
`55172122` ( 1.1x )	`253108` ( 1.817 )	`92236` ( 1.661 )	2048	NCHW	TVM	Autotuned	RV32GCV	Loop+SLP
`55212790` ( 1.1x )	`253108` ( 1.817 )	`92236` ( 1.661 )	4096	NCHW	TVM	Autotuned	RV32GCV	Loop+SLP
`28923320` ( 2.0x )	`225824` ( 1.621 )	`108420` ( 1.953 )	128	NHWC	TVM	Autotuned	RV32GCV	Loop+SLP
`25786681` ( 2.3x )	`225828` ( 1.621 )	`108420` ( 1.953 )	256	NHWC	TVM	Autotuned	RV32GCV	Loop+SLP
`24262849` ( 2.4x )	`225828` ( 1.621 )	`108420` ( 1.953 )	512	NHWC	TVM	Autotuned	RV32GCV	Loop+SLP
`53047946` ( 1.1x )	`225828` ( 1.621 )	`108420` ( 1.953 )	1024	NHWC	TVM	Autotuned	RV32GCV	Loop+SLP
`72080454` ( 0.8x )	`225828` ( 1.621 )	`108420` ( 1.953 )	2048	NHWC	TVM	Autotuned	RV32GCV	Loop+SLP
`72080454` ( 0.8x )	`225828` ( 1.621 )	`108420` ( 1.953 )	4096	NHWC	TVM	Autotuned	RV32GCV	Loop+SLP
`58326869` ( Base )	`139294` ( Base )	`55516` ( Base )	0	NHWC	muRISCV-NN	Scalar	RV32GC	-
`11900749` ( 4.9x )	`148314` ( 1.065 )	`55516` ( 1.0 )	128	NHWC	muRISCV-NN	Scalar	RV32GCV	Loop+SLP
`11560823` ( 5.0x )	`148314` ( 1.065 )	`55516` ( 1.0 )	256	NHWC	muRISCV-NN	Scalar	RV32GCV	Loop+SLP
`14587109` ( 4.0x )	`148314` ( 1.065 )	`55516` ( 1.0 )	512	NHWC	muRISCV-NN	Scalar	RV32GCV	Loop+SLP
`16152643` ( 3.6x )	`148314` ( 1.065 )	`55516` ( 1.0 )	1024	NHWC	muRISCV-NN	Scalar	RV32GCV	Loop+SLP
`16856344` ( 3.5x )	`148314` ( 1.065 )	`55516` ( 1.0 )	2048	NHWC	muRISCV-NN	Scalar	RV32GCV	Loop+SLP
`36746643` ( 1.6x )	`148314` ( 1.065 )	`55516` ( 1.0 )	4096	NHWC	muRISCV-NN	Scalar	RV32GCV	Loop+SLP
`15306726` ( 3.8x )	`141274` ( 1.014 )	`55516` ( 1.0 )	128	NHWC	muRISCV-NN	Vector	RV32GCV	-
`9721738` ( 6.0x )	`141274` ( 1.014 )	`55516` ( 1.0 )	256	NHWC	muRISCV-NN	Vector	RV32GCV	-
`7175004` ( 8.1x )	`141274` ( 1.014 )	`55516` ( 1.0 )	512	NHWC	muRISCV-NN	Vector	RV32GCV	-
`5932428` ( 9.8x )	`141274` ( 1.014 )	`55516` ( 1.0 )	1024	NHWC	muRISCV-NN	Vector	RV32GCV	-
`5007329` ( 11.6x )	`141274` ( 1.014 )	`55516` ( 1.0 )	2048	NHWC	muRISCV-NN	Vector	RV32GCV	-
`4761514` ( 12.2x )	`141274` ( 1.014 )	`55516` ( 1.0 )	4096	NHWC	muRISCV-NN	Vector	RV32GCV	-

Anomaly Detection (`toycar`)

Cycles (Speedup)	Total ROM (rel.)	Total RAM (rel.)	VLEN	Layout	Kernels	Mode	Arch	Auto-Vectorization
`1669213` ( 1.0x )	`586676` ( 1.856 )	`5572` ( 1.168 )	0	NCHW	TVM	Fallback	RV32GC	-
`1669213` ( 1.0x )	`586676` ( 1.856 )	`5572` ( 1.168 )	0	NHWC	TVM	Fallback	RV32GC	-
`2246254` ( 0.8x )	`609586` ( 1.929 )	`6884` ( 1.443 )	0	NCHW	TVM	Autotuned	RV32GC	-
`2246254` ( 0.8x )	`609586` ( 1.929 )	`6884` ( 1.443 )	0	NHWC	TVM	Autotuned	RV32GC	-
`758695` ( 2.3x )	`588236` ( 1.861 )	`5572` ( 1.168 )	128	NCHW	TVM	Fallback	RV32GCV	Loop+SLP
`757749` ( 2.3x )	`588236` ( 1.861 )	`5572` ( 1.168 )	256	NCHW	TVM	Fallback	RV32GCV	Loop+SLP
`767299` ( 2.2x )	`588236` ( 1.861 )	`5572` ( 1.168 )	512	NCHW	TVM	Fallback	RV32GCV	Loop+SLP
`767299` ( 2.2x )	`588236` ( 1.861 )	`5572` ( 1.168 )	1024	NCHW	TVM	Fallback	RV32GCV	Loop+SLP
`767299` ( 2.2x )	`588236` ( 1.861 )	`5572` ( 1.168 )	2048	NCHW	TVM	Fallback	RV32GCV	Loop+SLP
`767299` ( 2.2x )	`588236` ( 1.861 )	`5572` ( 1.168 )	4096	NCHW	TVM	Fallback	RV32GCV	Loop+SLP
`758695` ( 2.3x )	`588236` ( 1.861 )	`5572` ( 1.168 )	128	NHWC	TVM	Fallback	RV32GCV	Loop+SLP
`757749` ( 2.3x )	`588236` ( 1.861 )	`5572` ( 1.168 )	256	NHWC	TVM	Fallback	RV32GCV	Loop+SLP
`767299` ( 2.2x )	`588236` ( 1.861 )	`5572` ( 1.168 )	512	NHWC	TVM	Fallback	RV32GCV	Loop+SLP
`767299` ( 2.2x )	`588236` ( 1.861 )	`5572` ( 1.168 )	1024	NHWC	TVM	Fallback	RV32GCV	Loop+SLP
`767299` ( 2.2x )	`588236` ( 1.861 )	`5572` ( 1.168 )	2048	NHWC	TVM	Fallback	RV32GCV	Loop+SLP
`767299` ( 2.2x )	`588236` ( 1.861 )	`5572` ( 1.168 )	4096	NHWC	TVM	Fallback	RV32GCV	Loop+SLP
`1181651` ( 1.5x )	`603170` ( 1.909 )	`6884` ( 1.443 )	128	NCHW	TVM	Autotuned	RV32GCV	Loop+SLP
`1141790` ( 1.5x )	`603170` ( 1.909 )	`6884` ( 1.443 )	256	NCHW	TVM	Autotuned	RV32GCV	Loop+SLP
`1123466` ( 1.5x )	`603170` ( 1.909 )	`6884` ( 1.443 )	512	NCHW	TVM	Autotuned	RV32GCV	Loop+SLP
`1116913` ( 1.5x )	`603170` ( 1.909 )	`6884` ( 1.443 )	1024	NCHW	TVM	Autotuned	RV32GCV	Loop+SLP
`1115331` ( 1.5x )	`603170` ( 1.909 )	`6884` ( 1.443 )	2048	NCHW	TVM	Autotuned	RV32GCV	Loop+SLP
`1428115` ( 1.2x )	`603170` ( 1.909 )	`6884` ( 1.443 )	4096	NCHW	TVM	Autotuned	RV32GCV	Loop+SLP
`1181651` ( 1.5x )	`603170` ( 1.909 )	`6884` ( 1.443 )	128	NHWC	TVM	Autotuned	RV32GCV	Loop+SLP
`1141790` ( 1.5x )	`603170` ( 1.909 )	`6884` ( 1.443 )	256	NHWC	TVM	Autotuned	RV32GCV	Loop+SLP
`1123466` ( 1.5x )	`603170` ( 1.909 )	`6884` ( 1.443 )	512	NHWC	TVM	Autotuned	RV32GCV	Loop+SLP
`1116913` ( 1.5x )	`603170` ( 1.909 )	`6884` ( 1.443 )	1024	NHWC	TVM	Autotuned	RV32GCV	Loop+SLP
`1115331` ( 1.5x )	`603170` ( 1.909 )	`6884` ( 1.443 )	2048	NHWC	TVM	Autotuned	RV32GCV	Loop+SLP
`1428115` ( 1.2x )	`603170` ( 1.909 )	`6884` ( 1.443 )	4096	NHWC	TVM	Autotuned	RV32GCV	Loop+SLP
`1721506` ( Base )	`316028` ( Base )	`4772` ( Base )	0	NHWC	muRISCV-NN	Scalar	RV32GC	-
`574417` ( 3.0x )	`317392` ( 1.004 )	`4772` ( 1.0 )	128	NHWC	muRISCV-NN	Scalar	RV32GCV	Loop+SLP
`488812` ( 3.5x )	`317392` ( 1.004 )	`4772` ( 1.0 )	256	NHWC	muRISCV-NN	Scalar	RV32GCV	Loop+SLP
`443596` ( 3.9x )	`317392` ( 1.004 )	`4772` ( 1.0 )	512	NHWC	muRISCV-NN	Scalar	RV32GCV	Loop+SLP
`420988` ( 4.1x )	`317392` ( 1.004 )	`4772` ( 1.0 )	1024	NHWC	muRISCV-NN	Scalar	RV32GCV	Loop+SLP
`409684` ( 4.2x )	`317392` ( 1.004 )	`4772` ( 1.0 )	2048	NHWC	muRISCV-NN	Scalar	RV32GCV	Loop+SLP
`1391992` ( 1.2x )	`317392` ( 1.004 )	`4772` ( 1.0 )	4096	NHWC	muRISCV-NN	Scalar	RV32GCV	Loop+SLP
`615192` ( 2.8x )	`316630` ( 1.002 )	`4772` ( 1.0 )	128	NHWC	muRISCV-NN	Vector	RV32GCV	-
`498512` ( 3.5x )	`316630` ( 1.002 )	`4772` ( 1.0 )	256	NHWC	muRISCV-NN	Vector	RV32GCV	-
`440172` ( 3.9x )	`316630` ( 1.002 )	`4772` ( 1.0 )	512	NHWC	muRISCV-NN	Vector	RV32GCV	-
`411326` ( 4.2x )	`316630` ( 1.002 )	`4772` ( 1.0 )	1024	NHWC	muRISCV-NN	Vector	RV32GCV	-
`407695` ( 4.2x )	`316630` ( 1.002 )	`4772` ( 1.0 )	2048	NHWC	muRISCV-NN	Vector	RV32GCV	-
`405839` ( 4.2x )	`316630` ( 1.002 )	`4772` ( 1.0 )	4096	NHWC	muRISCV-NN	Vector	RV32GCV	-

Visual Wake Words (`vww`)

Cycles (Speedup)	Total ROM (rel.)	Total RAM (rel.)	VLEN	Layout	Kernels	Mode	Arch	Auto-Vectorization
`41879031` ( 1.1x )	`597140` ( 1.839 )	`181040` ( 2.113 )	0	NCHW	TVM	Fallback	RV32GC	-
`65386577` ( 0.7x )	`555064` ( 1.71 )	`181040` ( 2.113 )	0	NHWC	TVM	Fallback	RV32GC	-
`37578989` ( 1.3x )	`548590` ( 1.69 )	`181040` ( 2.113 )	0	NCHW	TVM	Autotuned	RV32GC	-
`65386579` ( 0.7x )	`555066` ( 1.71 )	`181040` ( 2.113 )	0	NHWC	TVM	Autotuned	RV32GC	-
`52933558` ( 0.9x )	`707428` ( 2.179 )	`181040` ( 2.113 )	128	NCHW	TVM	Fallback	RV32GCV	Loop+SLP
`52870804` ( 0.9x )	`707430` ( 2.179 )	`181040` ( 2.113 )	256	NCHW	TVM	Fallback	RV32GCV	Loop+SLP
`52874193` ( 0.9x )	`707428` ( 2.179 )	`181040` ( 2.113 )	512	NCHW	TVM	Fallback	RV32GCV	Loop+SLP
`52884358` ( 0.9x )	`707424` ( 2.179 )	`181040` ( 2.113 )	1024	NCHW	TVM	Fallback	RV32GCV	Loop+SLP
`52908080` ( 0.9x )	`707424` ( 2.179 )	`181040` ( 2.113 )	2048	NCHW	TVM	Fallback	RV32GCV	Loop+SLP
`52952138` ( 0.9x )	`707426` ( 2.179 )	`181040` ( 2.113 )	4096	NCHW	TVM	Fallback	RV32GCV	Loop+SLP
`12994108` ( 3.6x )	`643494` ( 1.982 )	`181040` ( 2.113 )	128	NHWC	TVM	Fallback	RV32GCV	Loop+SLP
`9328731` ( 5.0x )	`643494` ( 1.982 )	`181040` ( 2.113 )	256	NHWC	TVM	Fallback	RV32GCV	Loop+SLP
`10058014` ( 4.7x )	`643490` ( 1.982 )	`181040` ( 2.113 )	512	NHWC	TVM	Fallback	RV32GCV	Loop+SLP
`19247151` ( 2.4x )	`643490` ( 1.982 )	`181040` ( 2.113 )	1024	NHWC	TVM	Fallback	RV32GCV	Loop+SLP
`29273035` ( 1.6x )	`643498` ( 1.982 )	`181040` ( 2.113 )	2048	NHWC	TVM	Fallback	RV32GCV	Loop+SLP
`57503731` ( 0.8x )	`643496` ( 1.982 )	`181040` ( 2.113 )	4096	NHWC	TVM	Fallback	RV32GCV	Loop+SLP
`16977624` ( 2.8x )	`577218` ( 1.778 )	`181040` ( 2.113 )	128	NCHW	TVM	Autotuned	RV32GCV	Loop+SLP
`14937578` ( 3.1x )	`577224` ( 1.778 )	`181040` ( 2.113 )	256	NCHW	TVM	Autotuned	RV32GCV	Loop+SLP
`15798670` ( 3.0x )	`577214` ( 1.778 )	`181040` ( 2.113 )	512	NCHW	TVM	Autotuned	RV32GCV	Loop+SLP
`18682103` ( 2.5x )	`577220` ( 1.778 )	`181040` ( 2.113 )	1024	NCHW	TVM	Autotuned	RV32GCV	Loop+SLP
`22882263` ( 2.1x )	`577218` ( 1.778 )	`181040` ( 2.113 )	2048	NCHW	TVM	Autotuned	RV32GCV	Loop+SLP
`35188739` ( 1.3x )	`577220` ( 1.778 )	`181040` ( 2.113 )	4096	NCHW	TVM	Autotuned	RV32GCV	Loop+SLP
`12994286` ( 3.6x )	`643544` ( 1.982 )	`181040` ( 2.113 )	128	NHWC	TVM	Autotuned	RV32GCV	Loop+SLP
`9328705` ( 5.0x )	`643550` ( 1.982 )	`181040` ( 2.113 )	256	NHWC	TVM	Autotuned	RV32GCV	Loop+SLP
`10057883` ( 4.7x )	`643548` ( 1.982 )	`181040` ( 2.113 )	512	NHWC	TVM	Autotuned	RV32GCV	Loop+SLP
`19246967` ( 2.4x )	`643548` ( 1.982 )	`181040` ( 2.113 )	1024	NHWC	TVM	Autotuned	RV32GCV	Loop+SLP
`29272826` ( 1.6x )	`643548` ( 1.982 )	`181040` ( 2.113 )	2048	NHWC	TVM	Autotuned	RV32GCV	Loop+SLP
`57503507` ( 0.8x )	`643548` ( 1.982 )	`181040` ( 2.113 )	4096	NHWC	TVM	Autotuned	RV32GCV	Loop+SLP
`47005386` ( Base )	`324662` ( Base )	`85664` ( Base )	0	NHWC	muRISCV-NN	Scalar	RV32GC	-
`17779199` ( 2.6x )	`333732` ( 1.028 )	`85664` ( 1.0 )	128	NHWC	muRISCV-NN	Scalar	RV32GCV	Loop+SLP
`17323034` ( 2.7x )	`333734` ( 1.028 )	`85664` ( 1.0 )	256	NHWC	muRISCV-NN	Scalar	RV32GCV	Loop+SLP
`18578502` ( 2.5x )	`333732` ( 1.028 )	`85664` ( 1.0 )	512	NHWC	muRISCV-NN	Scalar	RV32GCV	Loop+SLP
`21919150` ( 2.1x )	`333730` ( 1.028 )	`85664` ( 1.0 )	1024	NHWC	muRISCV-NN	Scalar	RV32GCV	Loop+SLP
`25658687` ( 1.8x )	`333730` ( 1.028 )	`85664` ( 1.0 )	2048	NHWC	muRISCV-NN	Scalar	RV32GCV	Loop+SLP
`40487526` ( 1.2x )	`333732` ( 1.028 )	`85664` ( 1.0 )	4096	NHWC	muRISCV-NN	Scalar	RV32GCV	Loop+SLP
`13642079` ( 3.4x )	`327544` ( 1.009 )	`85664` ( 1.0 )	128	NHWC	muRISCV-NN	Vector	RV32GCV	-
`10309462` ( 4.6x )	`327542` ( 1.009 )	`85664` ( 1.0 )	256	NHWC	muRISCV-NN	Vector	RV32GCV	-
`9013091` ( 5.2x )	`327546` ( 1.009 )	`85664` ( 1.0 )	512	NHWC	muRISCV-NN	Vector	RV32GCV	-
`8507581` ( 5.5x )	`327544` ( 1.009 )	`85664` ( 1.0 )	1024	NHWC	muRISCV-NN	Vector	RV32GCV	-
`8462583` ( 5.6x )	`327544` ( 1.009 )	`85664` ( 1.0 )	2048	NHWC	muRISCV-NN	Vector	RV32GCV	-
`8465972` ( 5.6x )	`327546` ( 1.009 )	`85664` ( 1.0 )	4096	NHWC	muRISCV-NN	Vector	RV32GCV	-

Original data

Click here to download the raw files for this benchmark.

2024-11-26

TFLM

GCC

O3

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

Os

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

LLVM

O3

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

Os

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

TVM

GCC

O3

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

Os

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

LLVM

O3

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

Os

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

2024-11-21

TFLM

GCC

O3

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

Os

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

LLVM

O3

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

Os

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

TVM

GCC

O3

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

Os

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

LLVM

O3

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

Os

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

2024-11-19

TFLM

GCC

Os

Spike (RV32) (spike_rv32)

Spike (RV64) (spike_rv64)

LLVM

Os

Spike (RV32) (spike_rv32)

Spike (RV64) (spike_rv64)

TVM

GCC

Os

Spike (RV32) (spike_rv32)

Spike (RV64) (spike_rv64)

LLVM

Os

Spike (RV32) (spike_rv32)

Spike (RV64) (spike_rv64)

2024-11-18

TFLM

GCC

Os

Spike (RV32) (spike_rv32)

Spike (RV64) (spike_rv64)

LLVM

Os

Spike (RV32) (spike_rv32)

Spike (RV64) (spike_rv64)

TVM

GCC

Os

Spike (RV32) (spike_rv32)

Spike (RV64) (spike_rv64)

LLVM

Os

Spike (RV32) (spike_rv32)

Spike (RV64) (spike_rv64)

2024-07-12

TFLM

GCC

Os

Spike (RV64) (spike_rv64)

LLVM

Os

Spike (RV64) (spike_rv64)

TVM

GCC

Os

Spike (RV64) (spike_rv64)

LLVM

Os

Spike (RV64) (spike_rv64)

2024-06-29

TFLM

GCC

Os

SpikeRV32

LLVM

Os

SpikeRV32

TVM

GCC

Os

SpikeRV32

LLVM

Os

SpikeRV32

2024-03-02

TFLM

GCC

O3

SpikeRV32

Os

SpikeRV32

LLVM

O3

SpikeRV32

Os

SpikeRV32

TVM

GCC

O3

SpikeRV32

Os

SpikeRV32

LLVM

O3

SpikeRV32

Os

SpikeRV32

2024-02-26

TFLM

GCC

s

SpikeRV32

LLVM

s

SpikeRV32

TVM

GCC

s

SpikeRV32

LLVM

s

SpikeRV32

2024-02-23

TFLM

GCC

s

SpikeRV32

LLVM

s

SpikeRV32

TVM

GCC

s

SpikeRV32

LLVM

s

SpikeRV32

2024-02-22

TFLM

GCC

s

SpikeRV32

LLVM

s

SpikeRV32

TVM

GCC

s

SpikeRV32

LLVM

s

SpikeRV32

2024-02-20

TFLM

GCC

s

SpikeRV32

LLVM

s

SpikeRV32

TVM

GCC

s

SpikeRV32

LLVM

s

SpikeRV32

2024-02-11

TFLM

GCC

s

SpikeRV32

LLVM

s

SpikeRV32

TVM

GCC

s

SpikeRV32

LLVM

s

SpikeRV32

2023-12-22

TFLM

GCC

s

SpikeRV32

TVM

GCC

s

SpikeRV32

2024-11-26

TFLM

GCC

O3

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

Os

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

LLVM

O3

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

Os

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

TVM

GCC

O3

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

Os

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

LLVM

O3

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

Os

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

2024-11-21

TFLM

GCC

O3

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

Os

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

LLVM

O3

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

Os

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

TVM

GCC

O3

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

Os

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

LLVM

O3

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

Os

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

2024-11-19

TFLM

GCC

Os

Spike (RV32) (spike_rv32)

Spike (RV64) (spike_rv64)

LLVM

Os

Spike (RV32) (spike_rv32)

Spike (RV64) (spike_rv64)

TVM

GCC

Os

Spike (RV32) (spike_rv32)

Spike (RV64) (spike_rv64)

LLVM

Os

Spike (RV32) (spike_rv32)

Spike (RV64) (spike_rv64)

2024-11-18

TFLM

GCC

Os

Spike (RV32) (spike_rv32)

Spike (RV64) (spike_rv64)

LLVM

Os

Spike (RV32) (spike_rv32)

Spike (RV64) (spike_rv64)

TVM

GCC

Os

Spike (RV32) (spike_rv32)

Spike (RV64) (spike_rv64)

LLVM

Os

Spike (RV32) (spike_rv32)

Spike (RV64) (spike_rv64)

2024-07-12

TFLM

GCC

Os

Spike (RV64) (spike_rv64)

LLVM

Os

Spike (RV64) (spike_rv64)

TVM

GCC

Os

Spike (RV64) (spike_rv64)

LLVM

Os

Spike (RV64) (spike_rv64)

2024-06-29

TFLM

GCC

Os

SpikeRV32

LLVM

Os

SpikeRV32

TVM

GCC

Os

SpikeRV32

LLVM

Os

SpikeRV32

2024-03-02

TFLM

GCC

O3

SpikeRV32

Os

SpikeRV32

LLVM

O3

SpikeRV32

Os

SpikeRV32

TVM

GCC

O3

SpikeRV32

Os

SpikeRV32

LLVM

O3

SpikeRV32

Os

SpikeRV32

2024-02-26

TFLM

GCC

s

SpikeRV32

LLVM

s

SpikeRV32

TVM

GCC

s

SpikeRV32

LLVM

s

SpikeRV32

2024-02-23

TFLM

GCC

s

SpikeRV32

LLVM

s

SpikeRV32

TVM

GCC

s

SpikeRV32

LLVM

s

SpikeRV32

2024-02-22

TFLM

GCC

s

SpikeRV32

LLVM

s

SpikeRV32

TVM

GCC

s

SpikeRV32

LLVM

s

SpikeRV32

2024-02-20

TFLM

GCC

s

SpikeRV32

LLVM

s

SpikeRV32

TVM

GCC

s

SpikeRV32

LLVM

s

SpikeRV32

2024-02-11

TFLM

GCC

s

SpikeRV32

LLVM

s

SpikeRV32

TVM

GCC

s

SpikeRV32

LLVM

s

SpikeRV32

2023-12-22

TFLM

GCC

s

SpikeRV32

TVM

GCC

s

SpikeRV32

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Benchmarks 2024 03 02 TVM LLVM O3

Setup

Simulator

Toolchains

Models

Package Versions

Miscellaneous

Results (Framework: tvm, Backend: tvmaot, Toolchain: llvm, Flags: -O3)

Audio Wake Words (`aww`)

Image Classification (`resnet`)

Anomaly Detection (`toycar`)

Visual Wake Words (`vww`)

Original data

Clone this wiki locally

Benchmarks 2024 03 02 TVM LLVM O3

Setup

Simulator

Toolchains

Models

Package Versions

Miscellaneous

Results (Framework: tvm, Backend: tvmaot, Toolchain: llvm, Flags: -O3)

Audio Wake Words (aww)

Image Classification (resnet)

Anomaly Detection (toycar)

Visual Wake Words (vww)

Original data

Clone this wiki locally

Audio Wake Words (`aww`)

Image Classification (`resnet`)

Anomaly Detection (`toycar`)

Visual Wake Words (`vww`)