Skip to content

fix(sglang): request hidden states for every trainable drafter algorithm - #65

Open
khazic wants to merge 1 commit into
verl-project:mainfrom
khazic:khazic/fix/sglang-hidden-state-gating-per-algorithm
Open

fix(sglang): request hidden states for every trainable drafter algorithm#65
khazic wants to merge 1 commit into
verl-project:mainfrom
khazic:khazic/fix/sglang-hidden-state-gating-per-algorithm

Conversation

@khazic

@khazic khazic commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Problem

The SGLang request gate carries its own copy of the algorithm-to-hidden-state-layout mapping:

def _drafter_uses_eagle_last_hidden(drafter_cfg):
    algorithm = str(drafter_cfg.get("speculative_algorithm", "") or "").upper()
    return bool(algorithm == "EAGLE3" and ...)

def _drafter_uses_dflash_aux_hidden(drafter_cfg):
    algorithm = str(drafter_cfg.get("speculative_algorithm", "") or "").upper()
    return bool(algorithm in {"DFLASH", "DSPARK"} and ...)

Both sets have drifted from what speco_worker can actually build a training backend for:

  • EAGLE-1 / EAGLE-2 / P-EAGLE all need last_hidden_states. Eagle1TrainerBackend.compute_loss and PEagleTrainerBackend.compute_loss both raise ValueError("... requires last_hidden_states") without it, and both build_model implementations reject use_logits=True, so there is no fallback.
  • Domino consumes the same aux context layers as DFlash. DominoTrainerBackend subclasses DFlashTrainerBackend and its compute_loss splits batch["hidden_states"] into num_context_layers.

None of the four appear in either set. On an SGLang rollout with collect_hidden_states_from_sgl, they request nothing at all and drafter training is starved.

EAGLE-1/2 are especially easy to miss here: Eagle1TrainerBackend.model_type deliberately reports "eagle3" so it can reuse the EAGLE-3 data plumbing, but this gate reads the raw config string, so the masquerade never reaches it.

Fix

oldlogprob_layer_ids.resolve_drafter_hidden_states_layout is already the shared answer to this exact question. speco_worker and speco_ray_trainer both use it, its docstring spells out the split ("DFlash-family drafters (DFlash, DSpark, Domino) consume the raw aux context layers, while EAGLE-family drafters also need the final hidden state"), and DFLASH_FAMILY_ALGORITHMS already lists DOMINO.

Route the SGLang gate through it instead of keeping a second copy that can drift again. The two predicates keep their names and their mutual exclusivity, and the shared collection preconditions move into one helper.

oldlogprob_layer_ids imports nothing from verl_speco, so there is no import cycle.

Validation

Ran a before/after repro on both main and this branch. For every algorithm speco_worker accepts, it asks what layout the trainer needs (via the shared resolver) and what the SGLang gate actually requests.

Before/after repro output
=================== before: main (333b754) ===================
algorithm  trainer needs          sglang requests        verdict
----------------------------------------------------------------
EAGLE1     eagle3_aux_plus_last   NOTHING                MISMATCH
EAGLE2     eagle3_aux_plus_last   NOTHING                MISMATCH
EAGLE3     eagle3_aux_plus_last   last_hidden            ok
PEAGLE     eagle3_aux_plus_last   NOTHING                MISMATCH
DFLASH     dflash_aux             dflash_aux_hidden      ok
DSPARK     dflash_aux_plus_last   dflash_aux_hidden      ok
DOMINO     dflash_aux             NOTHING                MISMATCH

[verdict]
          algorithms with no hidden states: EAGLE1, EAGLE2, PEAGLE, DOMINO
          RESULT: drafter training is starved on SGLang (bug present)

=================== after: this branch ===================
algorithm  trainer needs          sglang requests        verdict
----------------------------------------------------------------
EAGLE1     eagle3_aux_plus_last   last_hidden            ok
EAGLE2     eagle3_aux_plus_last   last_hidden            ok
EAGLE3     eagle3_aux_plus_last   last_hidden            ok
PEAGLE     eagle3_aux_plus_last   last_hidden            ok
DFLASH     dflash_aux             dflash_aux_hidden      ok
DSPARK     dflash_aux_plus_last   dflash_aux_hidden      ok
DOMINO     dflash_aux             dflash_aux_hidden      ok

[verdict]
          every algorithm requests the layout its trainer needs
          RESULT: gate matches the shared resolver (fixed)

Tests

Three new tests in tests/integration/test_sglang_adapter_contract.py:

  • test_every_trainable_drafter_requests_its_hidden_state_layout walks all seven algorithms through the real request path and pins which custom param each one gets.
  • test_last_hidden_env_follows_the_shared_layout_resolver covers VERL_SGLANG_DRAFTER_RETURN_LAST_HIDDEN for the same seven.
  • test_hidden_state_gates_stay_off_without_drafter_collection pins the preconditions that must keep both gates off (enable=False, enable_drafter_training=False, use_logits=True).

Full CPU suite (tests/integration tests/compat tests/config tests/examples) run on both sides:

Test suite before/after
### BASELINE (origin/main, 333b754) ###
FAILED tests/integration/test_drafter_runtime_control_contract.py::test_target_head_sync_defers_for_all_lm_head_drafters[DSPARK-veomni-npu-veomni_lm_head_full]
FAILED tests/integration/test_drafter_runtime_control_contract.py::test_target_head_sync_defers_for_all_lm_head_drafters[DFLASH-veomni-npu-veomni_lm_head_sparse]
FAILED tests/integration/test_drafter_runtime_control_contract.py::test_target_head_sync_defers_for_all_lm_head_drafters[EAGLE3-veomni-cuda-veomni_lm_head_full]
FAILED tests/integration/test_drafter_runtime_control_contract.py::test_target_head_sync_defers_for_all_lm_head_drafters[EAGLE1-fsdp-npu-engine_full_param]
FAILED tests/integration/test_drafter_runtime_control_contract.py::test_target_head_sync_defers_for_all_lm_head_drafters[DOMINO-fsdp2-cuda-engine_full_param]
FAILED tests/integration/test_drafter_runtime_control_contract.py::test_target_head_transfer_waits_after_actor_update
FAILED tests/integration/test_drafter_runtime_control_contract.py::test_async_publish_sets_pending_ref_and_waits_before_next_publish
FAILED tests/integration/test_dspark_trainer_backend.py::test_dspark_checkpoint_preserves_source_config_and_vllm_weight_names
FAILED tests/integration/test_verl_npu_vllm_compat.py::test_factory_fused_moe_survives_verl_npu_patch_import
9 failed, 245 passed, 2 warnings in 33.60s

### THIS BRANCH ###
FAILED tests/integration/test_drafter_runtime_control_contract.py::test_target_head_sync_defers_for_all_lm_head_drafters[DSPARK-veomni-npu-veomni_lm_head_full]
FAILED tests/integration/test_drafter_runtime_control_contract.py::test_target_head_sync_defers_for_all_lm_head_drafters[DFLASH-veomni-npu-veomni_lm_head_sparse]
FAILED tests/integration/test_drafter_runtime_control_contract.py::test_target_head_sync_defers_for_all_lm_head_drafters[EAGLE3-veomni-cuda-veomni_lm_head_full]
FAILED tests/integration/test_drafter_runtime_control_contract.py::test_target_head_sync_defers_for_all_lm_head_drafters[EAGLE1-fsdp-npu-engine_full_param]
FAILED tests/integration/test_drafter_runtime_control_contract.py::test_target_head_sync_defers_for_all_lm_head_drafters[DOMINO-fsdp2-cuda-engine_full_param]
FAILED tests/integration/test_drafter_runtime_control_contract.py::test_target_head_transfer_waits_after_actor_update
FAILED tests/integration/test_drafter_runtime_control_contract.py::test_async_publish_sets_pending_ref_and_waits_before_next_publish
FAILED tests/integration/test_dspark_trainer_backend.py::test_dspark_checkpoint_preserves_source_config_and_vllm_weight_names
FAILED tests/integration/test_verl_npu_vllm_compat.py::test_factory_fused_moe_survives_verl_npu_patch_import
9 failed, 260 passed, 2 warnings in 22.04s

The same 9 tests fail on main and on this branch. They need optional dependencies (VeOmni, the NPU vLLM stack) that are absent in this environment, so they are pre-existing and unrelated. The +15 on this branch are the new parametrized tests above.

The SGLang request gate carried its own copy of the algorithm-to-layout mapping:

    algorithm == "EAGLE3"              -> last_hidden
    algorithm in {"DFLASH", "DSPARK"}  -> dflash aux hidden

Both sets have drifted from what speco_worker can actually train. EAGLE-1/2 and
P-EAGLE need last_hidden_states (their compute_loss raises without it, and both
reject use_logits=True), and Domino consumes the same aux context layers as
DFlash. None of the four appear in either set, so on an SGLang rollout with
collect_hidden_states_from_sgl they request nothing and drafter training is
starved.

EAGLE-1/2 are especially easy to miss here: Eagle1TrainerBackend.model_type
deliberately reports "eagle3" so it can reuse the EAGLE-3 data plumbing, but
this gate reads the raw config string instead, so the masquerade does not reach
it.

oldlogprob_layer_ids.resolve_drafter_hidden_states_layout is already the shared
answer to this question (speco_worker and speco_ray_trainer both use it, and it
lists DOMINO in DFLASH_FAMILY_ALGORITHMS). Route the SGLang gate through it
instead of keeping a second copy that can drift again.

Signed-off-by: khazic <khazzz1c@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant