perf(seedless): fold the cross-layer resid_add + input rmsnorm into the existing gdn_resid_postnorm_rows kernel (−39 dispatch/token)

[penta2himajin]

Observation

encodeLayer ends with the MoE residual fold, and the next layer's encodePreMoE starts with the input norm:

// layer i, tail
SeedlessFusedVerify.encodeResidAdd(enc, h: hBuf, r: moeOut, total: M * H)
// layer i+1, head
SeedlessFusedVerify.encodeRmsNormRows(enc, x: hBuf, w: L.inputLN, out: normed, rows: M, D: H, eps: eps)

That pair is exactly what gdn_resid_postnorm_rows already computes — h += r; out = rmsnorm(h, w). The kernel exists, ships, and is already used one step earlier in the same function for the mixer residual + post-norm (F5, notes/07 §3 Wave 2). It is being applied to one of the two structurally identical sites and not the other.

Change

In encodeLayer / encodeLayerBolt, hand the next layer's inputLN to encodeGdnResidPostNormRows instead of emitting resid_add then rmsnorm:

• layer 0: unchanged (no preceding MoE residual).
• layers 1..n-1: one fused dispatch replaces two.
• final layer: keep the plain encodeResidAdd (its consumer is the final norm, not a layer input).

Requires threading layers[i+1].inputLN into the layer-i encode, or restructuring the loop to encode the fold at the head of layer i+1 with the previous layer's moeOut — the latter mirrors the pendingResid idiom already used in SeedlessMetalForward.encodeMixerHalf, so it is the closer precedent.

Effect

• −1 dispatch/layer ⇒ −39/token (39 inter-layer boundaries; ≈5.6% of the 694 stock-default baseline corrected in perf: Step 0 — decode dispatch inventory + serial-vs-concurrent encoder bubble probe #143 — the original "6% of 654" used the QWISP_FUSE_MOE2=1 shape, which is not the default).
• One fewer full hBuf write→read round trip per layer (H×2 bytes each way at M=1; M×H at verify widths).
• Removes 39 serial-encoder barriers, which is the same currency perf: Step 0 — decode dispatch inventory + serial-vs-concurrent encoder bubble probe #143 is trying to price.

Small in isolation. Worth doing anyway because it is nearly free, it is independent of #143's outcome, and it gives P1 a known-magnitude delta to calibrate the dispatch counter against.

Why this is not a rewrite (Prohibition 3)

No new kernel, no kernel edit, no change to the op sequence's arithmetic — only which encode helper emits it. Same class of change as the F5 fusion that introduced the kernel.

Bit-exactness

Constructive, and verified by reading the kernel (2026-07-25) rather than assumed:

gdn_resid_postnorm_rows does not carry an unrounded float register into the norm. It rounds the residual sum
into half hn, stores it, and the normalization reads back from that stored half
(SeedlessFusedVerify.swift:907, :936). The separate residual kernel also writes half
(SeedlessMetalForward.swift:1019). So "store-then-reload" vs "register reuse" is a distinction without a
difference here — there is no f16 rounding divergence to worry about, which was the one real risk in this change.

The equivalence is moreover already locked for the mixer-side use by the F5 test at M ∈ {1, 8}
(SeedlessVerifyTests.swift:2177). This issue still gets its own locked test (G-A) because the cross-layer
wiring is new, but the arithmetic question is settled.

Gates

• G-A (locked, model-free): new RAWTESTS case — fold ON vs fold OFF produce bit-identical hBuf and normed across a multi-layer synthetic stack, M ∈ {1, 2, 8}, both layer types. Bumps the RAWTESTS total = N counter.
• G-B (lossless, model): spec-vs-greedy 512/512 on the standard 4 regimes; flag-off byte-unchanged (QWISP_FUSE_XLAYER=0).
• G-D (speed, report-only): tok/s before/after on resident + streaming, plus the P1 dispatch count from perf: Step 0 — decode dispatch inventory + serial-vs-concurrent encoder bubble probe #143 (expect exactly −39; the earlier "or −40 counting the head" was wrong — layer 0 is unchanged by this change and the head is not a layer boundary. A mismatch means the inventory in perf: Step 0 — decode dispatch inventory + serial-vs-concurrent encoder bubble probe #143 is wrong, which is itself useful).

Refs

#143 (Step 0 / dispatch inventory — this issue does not depend on its outcome) · notes/10:12 (the canonical "
"2026-07-05 recon already classified this exact change "(d) MoE-resid + next-input-norm fold = GO, stage 2, "
"別 devloop" — this issue is picking up a scheduled item, not proposing a new one) · notes/07 §3 Wave 2, §6 "
"(F5 and the fusion doctrine; §35 explicitly deferred this fold as cross-layer) · SeedlessFusedVerify.encodeGdnResidPostNormRows, encodePreMoE, encodeLayer · SeedlessMetalForward.encodeMixerHalf (pendingResid idiom)

[penta2himajin]

Review (Claude + Codex cross-check, 2026-07-25) — PROCEED, with corrected expectations

Reviewed alongside #143/#145/#146/#147. This is the one issue in that batch that survives review intact, and the body has been corrected accordingly.

Confirmed

• Prior art supports it. notes/10:12 (the canonical 2026-07-05 recon) already classified this exact change as "(d) MoE-resid + next-input-norm fold = GO, stage 2, 別 devloop", and notes/07:35 had deferred it as cross-layer. This is a scheduled item being picked up, not a new proposal — the issue now says so.
• It satisfies the governing doctrine. notes/10's recon states the rule as "S2 教訓 = 最大並列度 grid を満たす": a fold is GO only when the destination kernel's grid already matches the parallelism of the work folded in. gdn_resid_postnorm_rows already does a per-row reduction, and the incoming cross-layer residual is elementwise — no grid collapse. This is precisely why (d) was GO while (c) (perf(seedless): fuse gate qmm8 + route_top8 into one dispatch (−40/token; removes a 1-threadgroup serialization point per layer) #145) was NO-GO.
• The bit-exactness risk is not real. Read the kernel: it rounds the sum into half hn, stores it, and normalizes from the stored half (SeedlessFusedVerify.swift:907, :936); the separate residual kernel also writes half (SeedlessMetalForward.swift:1019). There is no unrounded-register path, so store-then-reload vs register-reuse cannot diverge. F5's locked test already covers the equivalence at M ∈ {1,8} (SeedlessVerifyTests.swift:2177). G-A still worth having for the new cross-layer wiring.
• −39 is right (39 inter-layer boundaries). "−40 counting the head" was wrong and has been removed.

Corrected downward

• The baseline is 694 dispatches/token at stock defaults, not 654 — fuseMOE2 is opt-in (SeedlessFusedVerify.swift:3106), not default-on. So this is ≈5.6% of dispatches, not 6% of 654.
• Expect a sub-1% wall-clock win, not 3–6%. notes/09 §1 already decomposes the roofline gap as weight floor ~3.6 ms · dispatch intervals ~1.5 ms · inefficient M=1 GEMV ~8 ms. The engine is occupancy-limited, not dispatch-limited; 39 dispatches is a slice of the ~1.5 ms term. The buffer round-trip saving (one full hBuf write→read per layer) may actually be the larger half of the benefit.

That is not an argument against doing it — it is cheap, low-risk, already sanctioned, and it calibrates #143's P1 counter. It is an argument against quoting a % target in advance.

Refs

notes/10:12 · notes/07:35 · notes/09 §1 · #143 (baseline corrected there too)

[penta2himajin]

追記: 兄弟 fold の実測値が存在する(前コメントの根拠を強化・一部訂正)

前のコメントで「sub-1% を期待値とすべき」と書いたが、根拠を notes/09 の間接的な分解に置いていた。直接の実測プライアが commit 6383eba にあるので訂正・補強する。

notes/10 §0 の兄弟候補 (b) は実装され、測られている

6383eba = 「II-C stage 1: (b) combine_rows → S2 fold」。判定文(そのまま引用):

the speed gain is SUB-NOISE — paired A/B M=1 +68µs (+0.5%), unmeasurable at wall (run-to-run variance ±5-6% swamps it) … The stage-1 recon's +3-4% estimate was a fuseSHEXP-proxy over-extrapolation: that proxy folded compute, whereas combine→S2 only removes dispatch overhead.

つまり notes/10 §0 が (b) と (d) の両方に付けた「+3-4%」という見積り自体が、compute も畳む proxy からの過大外挿だったと実測で判明している。(b) の実測は +0.5%。

この issue への含意

• (d)=本 issue は (b) と同じ「dispatch のみを削る fold」の系統。ただし完全に同一ではない: hBuf の write→read 往復を層ごとに1本消すので、純粋な dispatch-only よりは上振れ余地がある。
• それでも +3-6% を期待値として掲げるのは、既に反証された外挿を繰り返すことになる。本文の効果欄は「≈5.6% of 694 dispatches」= dispatch 数の割合であって wall の期待値ではない、と読むべき。
• 併せて注意: (b) は M=8 で −0.6〜1.3ms の退行が出て M==1 gate で回避された。本 issue の G-A が M ∈ {1,2,8} を要求しているのは正しいが、M>1 の速度退行は identity テストでは捕まらない(当時それを見逃し、driver の 4-regime wall 実測が覆した)。G-D に M>1 の paired A/B を明示的に入れることを勧める。

記録側の問題(この issue が「新規」に見えた理由)

notes/10:12 は (d) を GO(stage 2)のまま残しているが、(b) が sub-noise だった後に (d) を同 magnitude 予想でスキップした判断は、notes/ にも commit message にも書き戻されていない(私の作業メモにのみ存在した)。repo だけを読めば (d) は未着手の GO に見える — 本 issue はそう読まれて起票された。起票者の落ち度ではなく記録の欠落なので、実施するにせよ見送るにせよ、結論は notes/10 に書き戻すべき。

判断は変えない

やる価値の根拠は「%」ではなく、低リスク・低コスト・bit-exact が構成的に保証・#143 の P1 カウンタの校正になるという点。期待値だけ +0.5-1% 級に置き換えて進めるのが妥当。もし wall で有意差が出なければ、(b) と同じく opt-in default-OFF の検証済み資産として温存する着地が前例に沿う。

Refs: 6383eba(判定文) · notes/10:12 · notes/09 §1