fix(dspark): re-apply verifier final RMSNorm in L1 distillation target

[minziyu]

Problem

DSpark drafter training distills the drafter towards the verifier's token distribution with an L1 loss. The target probabilities were computed as:

target_logits = F.linear(target_hidden, lm_head_weight)

But the collected target_hidden states come from vLLM's ExampleHiddenStatesConnector, which exports the verifier's final hidden state before its final RMSNorm. The verifier's real sampling distribution is softmax(lm_head(rmsnorm(h, w))), so the L1 target was a systematically biased distribution (TV ≈ 0.145 measured on Qwen3-30B-A3B).

The failure mode is insidious: the loss is self-consistent, so training still converges and outputs stay correct — only the online acceptance rate is capped well below what the same budget achieves with a correct target.

Fix

• New TargetFinalNorm (verl_speco/models/target/target_head.py): the verifier's frozen final RMSNorm loaded from the target checkpoint.
• key candidates model.norm.weight / model.language_model.norm.weight / norm.weight
• eps from config.json (rms_norm_eps, incl. nested text_config)
• gemma-style x*(1+w) parameterization folded to w+1 at load time for gemma* / qwen3_5* model types
• fp32, rank-1, requires_grad=False
• DSparkTrainerBackend._build_target_final_norm builds it whenever dspark_l1_loss_alpha > 0; the training step passes weight/eps through to the L1 chunk, which now re-applies the norm before the lm_head: h * rsqrt(mean(h²)+eps) * w → F.linear(·, lm_head).

No config flag — the norm is unconditional whenever the L1 loss is on. The legacy no-norm path is only reachable with dspark_l1_loss_alpha=0 (L1 disabled entirely).

Results (A/B validation)

Single-variable A/B on Qwen3-30B-A3B (single-sample overfit, 125 steps × batch 8 = 1000 learnings, greedy with pinned server- and request-side seeds, drafter deployed with vLLM speculative decoding, k=7):

	A (fix)	B (bug)
Training accuracy / L1 loss	0.993 / 0.019	0.988 / 0.041
Online acceptance rate	0.912	0.619
Mean accepted len / block of 8	6.39	4.33
Draft rounds (3×256 tok)	39	54 (+38%)

• Outputs of both runs are token-identical to non-speculative decoding — the bug costs speed, not correctness.
• Offline ceiling check on the collected hidden states (no training): top-1 agreement 0.843 (raw) vs 0.987 (normed), bounding B's achievable acceptance at ≈0.855 on this sample.
• Per-position acceptance: pos-0 is identical between A and B (92–94%); the whole gap concentrates at deep in-block positions (conditional rates A ≈100% vs B 88%→78%), consistent with the biased target compounding through the drafter's in-block autoregressive rollout.

Tests

• tests/integration/test_target_final_norm.py — from_pretrained: key candidates, nested text_config eps, folding (qwen3 / qwen3_5 / gemma3 parametrized), frozen weight, missing-weight KeyError.
• tests/integration/test_dspark_trainer_backend.py — L1 target numerics match a hand-computed reference with the norm applied; the norm provably changes the target (anti-vacuous assertion); kwargs plumbing is guarded so the norm cannot be silently dropped between forward() and the L1 computation.
• All 18 tests pass. test_dspark_checkpoint_preserves_source_config... fails identically on pristine main branch (pre-existing, non-blocking for this change).