refactor(qwen35): split TP response contract into tp_executor/responses.rs - #1036
Open
CAICAIIs wants to merge 4 commits into
Open
refactor(qwen35): split TP response contract into tp_executor/responses.rs#1036CAICAIIs wants to merge 4 commits into
CAICAIIs wants to merge 4 commits into
Conversation
CAICAIIs
force-pushed
the
feat/qwen35-tp-responses-split
branch
2 times, most recently
from
September 8, 2026 06:52
5de248a to
8039d67
Compare
CAICAIIs
force-pushed
the
feat/qwen35-tp-responses-split
branch
2 times, most recently
from
September 8, 2026 09:23
80b544a to
7b788fc
Compare
Move the per-rank worker runtime out of the tp_executor.rs God module:
TpWorker + spawn/drop lifecycle, TpStartupGate, the NCCL startup watchdog,
TpWorkerState/TpWorkerPrepared and its command loop (run/respond/
execute_*/precapture/drop), the slot bookkeeping helpers, the shared
decode-row view/sampling helpers, recurrent-capacity math, and the worker
CublasThreadGuard binding. The entry keeps command/reply types, the
executor orchestration (including the precapture sweep), plan validators,
and response validation, reaching worker items via the same `use super::*`
style as scheduler/{backend,tp}. Rebased over pegainfer-project#720/pegainfer-project#722. Behavior
unchanged; the watchdog disarm test moves into worker.rs with its
implementation.
Signed-off-by: CAICAIIs <3360776475@qq.com>
CAICAIIs
force-pushed
the
feat/qwen35-tp-responses-split
branch
from
September 10, 2026 03:43
7b788fc to
363facb
Compare
The tp_executor entry and the worker module carried two near-identical condvar gates: TpCommandStartGate (Pending/Execute/Cancel, first-write resolve returning whether it won) and TpStartupGate (Pending/Connect/ Cancel, silent set). Both broadcast one go/cancel decision that is resolved exactly once, so they collapse into TpGate + TpGateDecision with the first-write-wins resolve; startup treats Go as connect and asserts on the returned decision instead of a bool. Behavior unchanged. Signed-off-by: CAICAIIs <3360776475@qq.com>
The scheduler and the TP worker carried byte-identical CUDA/cuBLAS thread bindings (guard + set_device + bind_to_thread + cublas_init); the worker split moved one of the two copies, making the duplication visible. Both now call crate::cublas_thread::bind_model_thread with a role tag (the failure messages are unchanged); the scheduler keeps its tune_decode_gemm_algos step at its own call site. Behavior unchanged. Signed-off-by: CAICAIIs <3360776475@qq.com>
…es.rs Move the reply envelope (TpWorkerReply/TpWorkerResponse), bounded response collection, per-command response validation, and the step-timeout/abort helpers out of tp_executor.rs; the three response-contract tests move with them. Behavior unchanged, with two behavior-preserving consolidations in the moved code: validate_exact_rank_responses folds its push loop into an iterator pipeline, and the three rank-0 payload validators (prefill / decode / unified) collapse into one validate_primary_responses helper over payload extractors, keeping the exact prior error messages. Rebased over pegainfer-project#1004/pegainfer-project#1005/pegainfer-project#1033/pegainfer-project#1038 (the precapture sweep keeps using validate_ack_responses, which stays production-visible). Signed-off-by: CAICAIIs <3360776475@qq.com>
CAICAIIs
force-pushed
the
feat/qwen35-tp-responses-split
branch
from
September 10, 2026 05:19
363facb to
676daed
Compare
Collaborator
|
@codex check this please |
|
To use Codex here, create a Codex account and connect to github. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Split the TP response contract out of
tp_executor.rsintotp_executor/responses.rs, following #worker-split. Behavior unchanged.Stacked PR 2/3 — based on #1035 (worker split); open against
mainbecause stacked cross-fork bases are not supported, so its Files-changed view includes #1035 until that merges. Merge #1035 first. Rebuilt over #1004/#1005/#1033/#1038: the precapture sweep keepsvalidate_ack_responseson the production path, so it stays visible rather than test-gated.Structure
responses.rs(394 lines) owns the reply envelope (TpWorkerReply/TpWorkerResponse), bounded response collection with the 300s step timeout, the poison-on-failure contract, and per-command response validation.tp_executor.rskeeps the executor orchestration, commands, plan validators, and tests that don't exercise the response contract. The three response-contract tests (runtime_response_failure_poisons_executor,runtime_response_collection_fails_fast_when_peer_never_responds,disconnected_runtime_response_poisons_executor) move with the contract they pin.Behavior-preserving consolidations (functional style)
validate_exact_rank_responses: the push loop folds into aninto_iter().map(..).collect::<Result<Vec<_>>>()pipeline over the same checks (count, rank range, duplicate ranks, completeness), same error strings.validate_prefill/decode/unified_responses) collapse into onevalidate_primary_responseshelper over payload extractors, keeping the exact prior error messages (operation + payload noun cover all three wording variants).Defense table (all inherited — pure move)
responses.rsrecv_runtime_responseresponses.rscollect_runtime_responsesresponses.rsrecv_runtime_responseresponses.rsvalidate_exact_rank_responsesresponses.rsvalidate_primary_responsesValidation
cargo fmt --checkclean;cargo clippy -p pegainfer-qwen35 --features qwen35 --all-targets -- -D warningsclean.cargo test --libgreen (moved response tests included);tp2_*GPU tests pass on 2×A100;e2e_schedulergreen.