Add DFlash speculative decoding support to NeMo SpeechLM vLLM plugin

[Slyne]

Important

This PR depends on #15906. Until that prerequisite merges, GitHub may show its SALM vLLM/MTP commits in this draft's diff.

What does this PR do?

Add checkpoint-backed DFlash and DFlash2 speculative-decoding inference support for NeMo SpeechLM models served through vLLM.

Collection: SpeechLM (nemo.collections.speechlm2)

Changelog

• Register the NeMo SpeechLM target with vLLM's SupportsEagle3 hidden-state contract used by DFlash.
• Preserve the SALM audio-conditioned target path while exposing the intermediate hidden states required by the draft model.
• Route published DFlash checkpoints through vLLM speculative decoding.
• Document DFlash2 inference, its V2 model-runner requirement, and its required checkpoint configuration.
• Pin DFlash2 to immutable vLLM commit 3406ec1dae9916f920b90f0dbf90dcf54923d042 from [Spec Decode] DFlash2: local convolution + candidate selector vllm-project/vllm#52816.
• Add focused coverage for plugin registration, configuration routing, weight handling, and multimodal/speculative-decoding contracts.

Scope

This is inference-only support. It does not train, fine-tune, convert, or bundle DFlash/DFlash2 checkpoints. A DFlash2 checkpoint must be trained or fine-tuned separately for the target Lightning backbone.

Usage

vllm serve /path/to/vllm-ready-speechlm-checkpoint \
  --trust-remote-code \
  --speculative-config '{
    "method": "dflash",
    "model": "nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4-DFlash",
    "num_speculative_tokens": 6
  }'

For DFlash2, install the pinned runtime documented in docs/source/speechlm2/vllm_dflash.rst and point model at a separately trained checkpoint declaring DFlash2DraftModel.

Validation

• Focused unit tests: 116 passed
  • tests/collections/speechlm2/test_to_hf.py
  • tests/collections/speechlm2/test_vllm_plugin.py
• Repository pre-commit hooks passed for the complete branch diff; all changed Python files pass Black 24.10.0.
• git diff --check upstream/main...HEAD passed.
• DFlash1 was validated end-to-end with the Lightning SALM proxy and published trained draft checkpoint.
• DFlash2 inference plumbing was exercised through vLLM's V2 model runner; no production DFlash2 checkpoint is included in this PR.

Before this PR is ready for review

• Contributor guidelines followed
• Necessary tests added
• Necessary documentation added
• Optional vLLM imports remain plugin-scoped
• New feature
• Prerequisite Add MTP speculative-decoding support to NeMo SpeechLM vLLM plugin #15906 merged

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[Slyne]

Close this PR. Please refer to #16239.