qwen3/dflash: DFlash 2 proposer — top-16 path selector + two-tap dynamic convolution (~+21% acceptance length over DFlash)

[Ma1oneZhang]

What upstream shipped

Inco AI's DFlash 2 (blog, 2026-08-18) keeps the one-pass parallel draft and recovers the two losses it leaves on the table:

• Selection headroom. Per-position Recall@1 is 85.4%, but the target token sits in the drafter's own top-16 candidates 99.5% of the time; an oracle selector would lift acceptance length 4.27 → 6.79 (Qwen3-4B, GSM8K). No autoregressive correction needed.
• Suffix decay. Candidate quality itself decays toward the block end (oracle recall 99.5% → 87.8% by position 7). Upstream's analysis: the drafter's attention spends its mass reading pre-block context and less and less on within-block dependencies (within-block attention share falls 30% → 8% across the 5 layers).

Two cheap, drop-in components fix both — together ~+1.3% draft–verify cycle latency:

1. Path selector. Keep top-16 candidates per position; score every adjacent pair S_t(a,b) = U_t(b) + ⟨A(a)⊙H(h_t), B(b)⟩ (256-dim token embeddings + context gate — a low-rank bilinear attention); a single greedy/sampled walk over the precomputed scores picks the proposed path, and rejection sampling restores the exact target distribution. Scoring stays fully parallel — no extra backbone or LM-head pass. +2.0M params, +0.6% latency on a 5-layer Qwen3-4B-sized drafter; +0.34 (T=0) / +0.47 (T=1) acceptance tokens vs plain DFlash, beating a DSpark Markov-head correction with ~40× fewer params and ~16× lower latency overhead.
2. Two-tap dynamic depthwise convolution before and after each attention and MLP sublayer: Conv(x)_t = k_{t,0}⊙x_t + k_{t,1}⊙x_{t-1} (first position taps the last verified token's hidden). +3% params, +0.7% latency; a 5L+conv drafter nearly matches a 15L drafter on suffix recall, while the 10 extra layers would cost +15.2% latency.

Reported end-to-end (Qwen3.5-4B, mean acceptance length over 5 benchmarks): DFlash 4.92 → DSpark 5.49 → DFlash 2 5.97 (+21% over DFlash, +0.48 over DSpark); 2.7–3.4× autoregressive throughput on Qwen3.8-27B in SGLang. Released drafters include incoai/Qwen3.8-27B-DFlash2.

Why pegainfer is well positioned

Our transaction abstraction (docs/models/qwen3/dflash-speculative-decoding.md) makes the draft↔verify boundary a pure token span; the method-specific part is only the proposer. DFlash 2 changes exactly that part: the conv alters the drafter backbone, and the selector changes which K tokens the proposer emits — still one parallel pass, still a K-length span. The shared core (speculative.rs accept/commit/rollback) and the verify forward are untouched.

It also subsumes the direction of #717 / #718: DSpark in our tree is precisely the class of sequential-Markov-head correction the selector claims to beat at 1/40th the parameters, and this gives #443 a concrete next proposer.

Proposal (phased like the DSpark work)

• Phase 1 — selector on top of existing DFlash weights. Emit a selected path instead of the top-1 chain. Two new proposer-side pieces: batched pairwise scoring (block_size × 16 preds × 16 cands × 256 dims — folds into the already-batched draft pass from draft_logits_batched) and a ≤block_size-step argmax walk (launch-floor candidate, same graph treatment discussed in RFC: adaptive per-request verify windows under load (SPS(B) scheduler, DSpark Phase 3) — design space + CUDA-graph strategy #718).
• Phase 2 — conv drafter. Add the two-tap dynamic depthwise-conv op around drafter sublayers; load the upstream DFlash 2 drafter (or train under our hf-golden-probe gates). Track the fixed-overhead floor — upstream's claim is +1.3% cycle latency for selector+conv combined.
• Phase 3 — sampled losslessness. Greedy stays longest-prefix (already gated by dflash_speculative_gate.rs); under sampling, DFlash 2's rejection sampling restores the exact target distribution, which needs the speculative-sampling accept rule in verify plus a gate broader than the greedy tie-flip band.

Measurement

Once #896 lands, the vllm-bench spec-decode fields give acceptance rate/length A/B for free. Claims to check on our 5090 / 5070 Ti harness (dflash_speculative_perf.rs): mean acceptance length +~1 token at block 7–8 vs current DFlash, and c1/c8/c16 throughput deltas under greedy.

References: DFlash 2 blog · sgl-project/sglang#35371 · vllm-project/vllm#52816

[RicardoMin]

Hi, I'm interested in working on this issue and would love to contribute. Would it be okay if I take on Phase 1 first, implementing the selector on top of the existing DFlash weights?

I'll start by looking into the batched pairwise scoring and the argmax walk, and I'll follow up here if I run into any design questions.

[xiaguan]

@Ma1oneZhang, are you actively working on this proposal or planning to open the implementation PR? @RicardoMin has offered to take Phase 1. Please coordinate ownership here before implementation starts so we do not create two parallel versions of the selector.