Measure attention projections instead of serving them, and record the amortisation trap

[AndreSlavescu]

The q, k, v and o projections are ~25% of a decode token with no meTile path, so they look like the
obvious next thing to serve. At matched bf16 they aren't worth serving, for the reason the standing goal
already records: MLX's decode GEMVs run at the hardware streaming limit.

shape	rows	MLX GB/s	gap	meTile	bit-exact
Qwen2.5-1.5B q/o	1	146	0.8x	0.993x	yes
Qwen2.5-1.5B k/v	1	254	0.5x	0.721x	yes
Qwen3-8B q/o	1	121	1.0x	1.007x	yes
Qwen3-8B k/v	1	149	0.8x	0.996x	yes
Qwen2.5-1.5B q/o	32	108	1.1x	1.366x	yes
Qwen2.5-1.5B k/v	32	50	2.4x	1.559x	NO
Qwen3-8B k/v	32	111	1.1x	1.411x	NO

A gap below 1.0x means the weights are cache-resident and DRAM isn't even the constraint. Eight rows
is parity. A win appears only at 32 rows — batch or prefill, not decode — and two of four shapes stop
being bit-exact there, so routing it would trade the logit-equality contract for a win outside the case
the task was about.

So this closes as measured rather than built. Adding the path would deliver parity at decode plus a
code path, a tuning surface, and a bit-exactness risk.

The measurement is the deliverable, and I got it wrong first

A first version sized the inner dispatch count for 64 MB of traffic per eval. For a 4.7 MB projection
that's 13 dispatches — about 390 µs of work against a ~200 µs mx.eval round trip, so nearly a third of
every sample was overhead. Because the two sides pay it differently, it reported MLX at 52 GB/s with a
2.3x gap and meTile at 1.828x. Both were fiction. At a gigabyte per eval the round trip is ~2% and the
same shape reads 146 GB/s at 0.993x.

That's the third time this session insufficient amortisation manufactured a win:

1. Module reloads inside a timing loop reported the scheduling pass at 0.83–0.92x — it was measuring the
   Metal compiler.
2. A per-call API reported the int4 multi-row QMV at 0.05x, where the published path uses a prepared
   executor (real answer: 1.45–1.73x).
3. This one, at 1.828x.

The rule that would have caught all three: if a measurement of a small kernel shows a large difference,
suspect the harness before believing it.

benchmarks/projection_headroom.py takes the traffic target as a flag, so lowering it reproduces the
error rather than leaving the claim to be trusted.

680 pass, 7 skipped.

🤖 Generated with Claude Code