Call for testers: benchmark qwisp on your Apple Silicon Mac (8/16GB + slow-NAND MacBooks especially wanted)

[penta2himajin]

qwisp runs Qwen3.6-35B-A3B (MoE) locally on Apple Silicon, streaming expert weights from flash so the model fits far beyond your RAM — with a bit-exact lossless mode as the reference. It works today, but almost all published numbers come from one dev machine (M1 Max / 64GB) plus SSD-throttle approximations. We need real hardware.

What we need most

hardware	why
8 GB / 16 GB Macs (any M-series)	the streaming tiers (C=64 / C=128) — only throttle-approximated today
MacBook Air / base MacBooks with 256 GB SSD	slow-NAND floor (~1.5 GB/s reads) — the tier we can least reproduce
M2 / M3 / M4 of any config	GPU-generation scaling
anything else	every row helps — resident 32GB+ included

Unstable output is wanted data too. If a row says **LOOPY** (0.31), please post it — run-to-run stability variance on streaming tiers is exactly what we're trying to map. There is no wrong result.

How (~15 min, mostly download)

brew install penta2himajin/qwisp/qwisp
qwisp pull        # downloads the model (~20 GB) + writes config
qwisp benchtest   # ~1-2 min; prints a markdown report

At the end, benchtest prints a one-click URL — cmd+click it in the terminal and the issue form opens pre-filled. Just press Submit. (Or paste the report into a new benchtest report manually.)

Please use v0.3.3 or later (qwisp version) — it reports the numeric stability ratio, so all rows share one format.

Notes:

• Plug into AC power if you're on a laptop (battery throttles the GPU; benchtest records which).
• The benchmark is greedy/deterministic and needs no account or telemetry — you see exactly what you post.

Results so far

chip	RAM	disk read	tier / mode	code-256	nl-256	long-600	provenance	report
M1 Max (32c GPU)	64 GB	3.5 GB/s	resident · strict	87.1 tok/s	91.5	85.2	dev machine	#36
your Mac here							community

Community rows are aggregated automatically into bench/RESULTS.md (median decode/TTFT + LOOPY rate per hardware group) — your submitted issue is ingested by a bot PR. Dev-measured and throttle-approximated figures live in the README Performance table.

Questions / problems

If qwisp pull or benchtest fails outright, open a regular issue with the error output — install failures are as valuable as benchmark rows right now.

Thank you! 🙏

---

Discussion thread on r/LocalLLM: https://www.reddit.com/r/LocalLLM/comments/1uwa85m/i_built_a_singlemodel_engine_that_streams_moe/

[penta2himajin]

Investigation update: the strict-streaming slowness you may have seen on 16GB machines is now a tracked investigation with a working hypothesis (Metal working-set overflow at C=128) — see #69. 16GB owners: one more 5-minute run would settle it: QWISP_DEVICE_RAM=8 qwisp benchtest and post via the one-click URL. Your first-wave reports are what localized this — thank you.

[penta2himajin]

Correction to the ask in my comment above. I asked 16GB owners to run QWISP_DEVICE_RAM=8 qwisp benchtest to settle #69. Two things have changed:

1. #69 is settled and now closed. #79 and #93 did exactly that run and reported strict code-256 at 8.4 and 11.2 tok/s, against a pre-fix field range of 1.2–3.4 — that is what closed it. Thank you.

2. The strict row in qwisp benchtest is not a clean measurement of strict, and I would rather flag that than keep collecting it as if it were. The benchmark runs its three bolt rows and its strict row in one process, and the bolt expert arena is never released — so the strict row is measured with bolt's arena still resident. On a streaming tier that is not the memory state a plain qwisp --lossless run has. Tracked as #165.

What this means for reports already submitted: the bolt rows are unaffected — those are the default mode and were measured as the default mode. Only the single strict row per streaming-tier report carries this caveat, and I will annotate those in bench/RESULTS.md once #165's measurement says how large the effect is.

The call itself stands and reports are still very much wanted — 8GB and slow-NAND MacBooks especially. Just run qwisp benchtest plainly; the QWISP_DEVICE_RAM override is no longer needed.

[penta2himajin]

Parked (2026-09-12): standing call-for-testers, not an active engineering workstream. Remains open as the community intake for 8/16GB + slow-NAND numbers; no scheduled work on the maintainer side until reports arrive (see also parked #86 / #164).