Skip to content
TencentPublic

About

No description, website, or topics provided.

Resources

Stars

265 stars

Watchers

1 watching

Forks

Repository files navigation

DSpine

Draft in Parallel, Condition Through Depth: Adjacent Causal Injection for Speculative Decoding

arXiv Project Page Hugging Face Collection License: Apache 2.0

DSpine is a parallel drafter for speculative decoding. Each draft layer passes the predicted feature of each position to the next, allowing token dependencies to be modeled during parallel drafting.

DSpine overview

News

  • [2026-11] DSpine is now supported in vLLM.
  • [2026-11] Added support for Qwen3.8-27B inference.
  • [2026-11] 🍎 Added support for Apple Silicon inference via mlx-lm and SGLang-MLX.
  • [2026-10] 🚀 Training code released in SpecForge.
  • [2026-10] 🤗 Qwen3-8B draft weights released.
  • [2026-10] 💻 Inference code released.
  • [2026-10] 📄 Paper released on arXiv.

Supported Models

Target model Draft model
Qwen3-8B tencent/Qwen3-8B-DSpine-b16
Qwen3.8-27B tencent/Qwen3.8-27B-DSpine-b16

Results

Qwen3-8B

Acceptance length on Qwen3-8B. DSpine, DSpark and DFlash2 are trained on the same Open-PerfectBlend data with the same training budget.

Temp. Method GSM8K MATH-500 HumanEval MBPP LCB MT-Bench Arena-Hard Avg.
T = 0 DFlash† 6.32 7.99 6.56 6.29 6.11 4.26 3.91 5.92
Domino† 9.68 9.40 7.23 6.89 6.40 5.12 4.65 7.05
DSpark 10.00 9.88 7.78 7.35 6.81 5.39 4.87 7.44
DFlash2 10.06 10.04 7.99 7.33 6.96 5.40 5.06 7.55
DSpine 10.55 10.30 8.27 7.66 7.25 5.63 5.13 7.83
T = 1 DFlash† 5.75 6.51 5.52 5.23 4.53 3.77 3.25 4.94
Domino† 8.01 7.13 5.89 5.56 4.59 4.36 3.61 5.59
DSpark 9.33 8.64 6.91 6.66 5.78 4.99 4.25 6.65
DFlash2 9.07 8.42 6.99 6.43 5.67 4.90 4.25 6.53
DSpine 9.83 8.99 7.39 6.97 6.11 5.09 4.49 6.98

† Official checkpoints: z-lab/Qwen3-8B-DFlash-b16 and Huang2020/Qwen3-8B-Domino-b16.

Qwen3.8-27B

We evaluate Qwen3.8-27B on WorkBuddy Bench using trajectories collected in the CodeBuddy sandbox. See the model card for acceptance length and throughput results.

Deployment

SGLang

The SGLang plugin uses SGLang 0.5.22 and requires Python 3.11+ and CUDA.

git clone https://github.com/tencent/dspine.git
cd dspine
python -m pip install -e .

Qwen3-8B

sglang serve \
  --model-path Qwen/Qwen3-8B \
  --speculative-algorithm DSPINE \
  --speculative-draft-model-path tencent/Qwen3-8B-DSpine-b16 \
  --speculative-num-draft-tokens 16 \
  --tp-size 1 \
  --dtype bfloat16 \
  --attention-backend fa3 \
  --page-size 1 \
  --mem-fraction-static 0.65 \
  --context-length 8192 \
  --max-running-requests 64 \
  --cuda-graph-bs-decode 1 2 4 8 16 32 64 \
  --trust-remote-code

Qwen3.8-27B

The implementation uses DSpark-based dynamic verification and ReplaySSM.

Run SPS tuning with your serving settings to tune dynamic verification. Add --speculative-dspark-sps-table-path sps.json to the command below to load the result.

sglang serve \
  --model-path /path/to/Qwen3.8-27B \
  --speculative-algorithm DSPINE \
  --speculative-draft-model-path tencent/Qwen3.8-27B-DSpine-b16 \
  --speculative-num-draft-tokens 16 \
  --tp-size 1 \
  --dtype bfloat16 \
  --attention-backend fa3 \
  --speculative-draft-attention-backend fa3 \
  --linear-attn-backend triton \
  --enable-linear-replayssm-spec \
  --mamba-ssm-dtype float32 \
  --mamba-radix-cache-strategy extra_buffer \
  --max-mamba-cache-size 256 \
  --mem-fraction-static 0.80 \
  --context-length 131072 \
  --max-running-requests 32 \
  --cuda-graph-max-bs-decode 32 \
  --min-free-slots-delay 1 \
  --sampling-backend flashinfer \
  --reasoning-parser qwen3 \
  --tool-call-parser qwen3_coder \
  --trust-remote-code

Use --tp-size N for tensor parallelism and --speculative-draft-attention-backend triton for Triton draft attention.

vLLM

Install vLLM from main, then serve Qwen3.8-27B with the DSpine draft:

vllm serve Qwen/Qwen3.8-27B \
  --language-model-only \
  --speculative-config '{
    "method": "dflash",
    "model": "tencent/Qwen3.8-27B-DSpine-b16",
    "num_speculative_tokens": 15
  }'

Set num_speculative_tokens to the draft block size minus one. For probabilistic draft sampling, add "draft_sample_method": "probabilistic"; for adaptive verification, add "enable_adaptive_verification": true to the speculative config. See the vLLM DSpine guide.

Apple Silicon (MLX)

Install with pip install -e dspine_mlx. See the MLX guide for mlx-lm and SGLang-MLX usage.

Training

See the SpecForge training guide and DSpine online training example.

Roadmap

Acknowledgements

We thank DFlash, SpecForge, SGLang and FlashInfer for their open-source work.

Citation

@article{zhang2026dspine,
  title   = {Draft in Parallel, Condition Through Depth: Adjacent Causal Injection for Speculative Decoding},
  author  = {Zhang, Haohui and Chen, Keyu and Sun, Haocheng and Gu, Weibo and Qiao, Ruizhi and Sun, Xing and Jiang, Bo},
  journal = {arXiv preprint arXiv:2609.36173},
  year    = {2026}
}

License

Apache 2.0

About

No description, website, or topics provided.

Resources

Stars

265 stars

Watchers

1 watching

Forks

Releases

Packages

Contributors

Languages