fix(frontend): unify stop-token semantics across model schedulers

docs/benchmarks/qwen3-8b-pd-vs-mix-h200.md

@@ -17,7 +17,6 @@ vllm-bench \
   --multi-turn --multi-turn-num-turns 5 \
   --random-input-len 4096 --per-turn-input-len 1024 --random-output-len 128 \
   --num-prompts 20 --multi-turn-concurrency 10 \
-  --extra-body '{"min_tokens":1}' \
   --temperature 0
 ```
 

docs/index.md

@@ -171,6 +171,7 @@ Organized by domain (model line / subsystem / playbook / lesson) instead of by l
 | `subsystems/frontend/frontend-architecture.md` | `pegainfer-frontend` owns everything north of the model schedulers. Two contract generations coexist: the step contract (qwen3 + pegainfer-sim migrated) and the legacy `EngineHandle`/`TokenEvent` path (other five lines). Next: migrate glm52, then delete the legacy contract. |
 | `subsystems/frontend/simulated-inference-engine.md` | CPU-only simulated model crate on the step contract (`SimScheduler` → `LaunchedEngine::Stepped`) for vLLM/OpenAI frontend and `vllm bench serve` validation without CUDA or weights. |
 | `subsystems/frontend/sim-step-contract.md` | Cut `pegainfer-sim` from the legacy `EngineHandle`/`TokenEvent` path onto the step contract. |
+| `subsystems/frontend/stop-token-policy.md` | Shared EOS/explicit-stop contract: preserve the triggering token and typed stop cause across migrated schedulers; retain the legacy sentinel only as a compatibility fallback. |
 | `subsystems/frontend/sim-high-concurrency-bench.md` | Same-session A/B vs main: feat TPOT ~30–180× better, TTFT worse and linear in C; E2EL/throughput win at c=64 and c=1024. |
 | `subsystems/frontend/cpu-profiling-baseline.md` | Frontend CPU profiling baseline using `pegainfer-sim` with fixed TTFT=5ms/TPOT=12ms: 200 req / concurrency=16 shows ~150ms TTFT overhead (no dominant hotspot), heap allocation ~10%, stream polling ~7.5%, IPC ~1%; reproducible benchmark command and perf evidence documented. |
 | `subsystems/frontend/startup-time.md` | Qwen3-4B warm startup-to-ready: frontend tokenizer load runs concurrently with the engine load (HTTP still binds only after the engine registers); mmap teardown is paid at the end of load since #377; pinned-staging upload (2026-07) cuts warm ready 5.22s → 4.66s on sm_89, and the remaining floor is the engine's own post-load startup work. |

docs/models/kimi-k2/sampling.md

@@ -1,8 +1,8 @@
 # Kimi-K2 sampling: param surface and design
 
-**TL;DR**: temperature/top_k/top_p are honored on TP1/DP8 via one batched FlashInfer pass (greedy rows keep the in-graph argmax, zero perf cost); TP8 rejects non-greedy explicitly; everything else on the OpenAI surface is documented below — nothing is silently ignored anymore (#237).
+**TL;DR**: temperature/top_k/top_p are honored on TP1/DP8 via one batched FlashInfer pass (greedy rows keep the in-graph argmax, zero perf cost); TP8 rejects non-greedy explicitly; unsupported sampling fields are rejected or explicitly documented, and request stop-token IDs are carried through the shared stop policy (#237).
 
-Last touched: 2026-06
+Last touched: 2026-08
 
 ## Param surface (`/v1/completions`)
 
@@ -20,10 +20,14 @@ scheduler/worker.
 | `top_p` = 0 / out of range | **rejected** (HTTP 500, see below) | rejected | engine (`lifecycle.rs validate_sampling_params`) |
 | `top_k` ≥ 1 | **honored** (`top_k=1` routes greedy) | rejected if non-greedy | engine |
 | `top_k` = 0 | all tokens (disabled) | — | frontend maps 0 → -1; protocol type is `u32`, negatives don't parse |
-| `seed` | **accepted, ignored** — engine seed is fixed at 42, per-request seed is dropped at `convert_sampling` | same | frontend |
+| `seed` | greedy requests are **accepted, ignored**; non-greedy per-request seeds are rejected until row-local seed wiring lands | same | frontend |
 | `logprobs` | honored; for sampled rows the logprob follows the **sampled** token (reported rank is a placeholder, see PR #96) | honored (greedy only) | engine |
 | `max_tokens`, `echo`, `stop` (EOS) | honored | honored | engine / frontend |
-| `min_p`, `frequency_penalty`, `presence_penalty`, `repetition_penalty`, `logit_bias`, `min_tokens`, `prompt_logprobs`, custom `stop_token_ids` | **accepted, ignored** — dropped at `convert_sampling`, never reach the engine | same | frontend (all models, not kimi-specific) |
+| `min_p` | **honored** when in `[0, 1)`; out-of-range values are rejected | same | frontend / engine |
+| `frequency_penalty`, `presence_penalty`, `repetition_penalty` | defaults are accepted; non-default values are rejected because no matching sampler path exists | same | shared frontend wire validation |
+| `logit_bias`, `prompt_logprobs` | **accepted, ignored** — no engine-side implementation yet | same | frontend (all models, not kimi-specific) |
+| `stop_token_ids` | **honored** independently of EOS; the matching token is preserved and reported as the stop cause | same | shared `StopPolicy` |
+| `min_tokens` | **rejected** — the current scheduler contracts do not carry the threshold needed to mask EOS/stop IDs | same | shared frontend wire validation |
 
 Rejection UX pitfall: an engine-side rejection surfaces as a generic HTTP 500
 (`"Internal server error"`). The real message ("top_p must be in (0, 1]…",

docs/models/qwen3/model-crate.md

@@ -157,7 +157,7 @@ pub fn start_engine(
 ### Step 7: Retire ModelForward and Fix Length Limit
 - Deleted `pegainfer_core::model::{ModelForward, GenerationState}` and removed the root `src/model.rs` re-export.
 - Deleted the Qwen3 `forward.rs` compatibility path. Qwen3 tests that used it now build their baselines from `batch_prefill(bs=1)` plus `batch_decode(bs=1)`, so they exercise the same phase APIs as production.
-- Fixed Qwen3 decode length-limit handling by adding `DecodeEffect::EmitAndFinish`. EOS behavior is unchanged: EOS finishes without emitting the stop token. Length limit now emits the sampled final token, then sends `Finished { finish_reason: Length }`.
+- Fixed Qwen3 decode length-limit handling by adding `DecodeEffect::EmitAndFinish`. At that point EOS still finished without emitting the stop token; the current typed stop contract emits the trigger token and its `StopCause` together. Length-limited output likewise retains the sampled final token before `Finished { finish_reason: Length }`.
 - Regenerated `test_data/Qwen3-4B.json` because every length-limited golden output now includes the final requested token.
 - Re-ran `bench_serving snapshot` on the CUDA validation host and pulled back `bench_snapshots/rtx-5090/qwen3-4b.json`; `decode_heavy (1024,256)` now records `generated_tokens min=max=avg=256`.
 - Performance stayed within noise on RTX 5090:

docs/subsystems/frontend/frontend-architecture.md

@@ -93,7 +93,7 @@ All six lines are onboarded. Adding a model line = write `model_line.rs` in the
 
 ## Protocol stacks
 
-**`vllm` (current default, fleet-proven).** Impersonates a vLLM EngineCore process over in-process ZMQ/msgpack because upstream `vllm-server` assumes the engine is a separate process. HTTP routes, OpenAI types, tokenizer, chat templates, Prometheus live in the external `vllm-server`/`vllm-metrics`/`vllm-text` crates. `SteppedEngineBridge` translates each `RequestUpdate` 1:1 into an EngineCore output (wall-clock timestamps are reconstructed from the contract's `Instant`s via a per-bridge unix anchor; a `Finished{Stop}` appends the stop sentinel token, which is how usage keeps counting the suppressed EOS).
+**`vllm` (current default, fleet-proven).** Impersonates a vLLM EngineCore process over in-process ZMQ/msgpack because upstream `vllm-server` assumes the engine is a separate process. HTTP routes, OpenAI types, tokenizer, chat templates, Prometheus live in the external `vllm-server`/`vllm-metrics`/`vllm-text` crates. `SteppedEngineBridge` translates each `RequestUpdate` 1:1 into an EngineCore output (wall-clock timestamps are reconstructed from the contract's `Instant`s via a per-bridge unix anchor). The typed step contract carries the trigger token and `StopCause` together; only the legacy bridge retains synthetic sentinel fallback for producers that do not yet report a cause.
 
 **`dynamo` (planned second stack).** dynamo's `lib/llm` in-process path removes the wire protocol entirely (`EngineConfig::InProcessTokens` + `run_input`). The step contract was shaped so this stack can consume `StepOutputs` directly without impersonation overhead. Decision gate: prototype, A/B against the vllm stack, let TTFT/step-overhead numbers pick the default.
 

docs/subsystems/frontend/stop-token-policy.md

@@ -0,0 +1,177 @@
+# Stop-Token Policy Contract
+
+> **TL;DR:** Keep EOS policy, explicit request stop IDs, generated tokens, and
+> the concrete stop cause separate from wire parsing through every migrated
+> scheduler; preserve the trigger token and drop only speculative suffixes.
+>
+> **Last touched:** 2026-08
+
+## Preparation
+
+- **Read:**
+  - `docs/index.md` — frontend has two contract generations; Qwen3 and the
+    simulator use the step contract while the remaining model lines use the
+    legacy event path.
+  - `docs/subsystems/frontend/frontend-architecture.md` — migration must keep
+    the legacy bridge compatible until each model has its own lifecycle tests.
+  - `pegainfer-frontend/src/engine/stop.rs` — shared `StopPolicy` and
+    `StopCause` implementation.
+- **Plan:**
+  1. Audit every scheduler's prefill, decode, speculative, and P/D terminal
+     paths for policy propagation and trigger-token ordering.
+  2. Add CPU contract coverage where a multi-token or handoff path can lose the
+     trigger or miscount completion tokens.
+  3. Run formatting, frontend/model checks, and GPU tests where the server
+     toolchain supports them; record environmental blockers separately.
+
+## Contract
+
+`StopPolicy` has two independent inputs:
+
+- `eos`: `ModelDefault`, an explicit primary EOS ID, or `Ignore`;
+- `token_ids`: explicit request stop IDs, active regardless of `eos`.
+
+`StopPolicy::classify` gives EOS precedence when the same ID appears in both
+sets. A scheduler emits the sampled token first, then emits `Finished` with:
+
+- `StopCause::Eos(id)` for EOS stops (wire `stop_reason` remains absent);
+- `StopCause::Token(id)` for explicit request stops (wire `stop_reason` is the
+  actual token ID);
+- `None` for length stops.
+
+The completion count is incremented exactly once for every emitted token,
+including the trigger. For speculative spans, only the prefix through the
+first terminal token is committed; later candidates are discarded.
+
+## Migration Matrix
+
+| Model/path | Policy carried | Prefill | Decode/speculative | Terminal evidence |
+| --- | --- | --- | --- | --- |
+| Qwen3 step contract | yes | migrated | migrated, including speculative verify suffix truncation | `StopCause` |
+| Qwen3.5 legacy scheduler | yes | migrated | migrated | `StopCause` |
+| DeepSeek-V2-Lite | yes | migrated | migrated | `StopCause` |
+| Kimi-K2 TP/DP | yes | migrated | migrated | `StopCause` |
+| GLM5.2 | yes | migrated | decode, DSpark/MTP, and native P/D handoff migrated | `StopCause` / native handoff cause |
+| K3 | yes | migrated | scheduler span truncation migrated; CUDA build gate pending | `StopCause` |
+| Legacy bridge | compatibility | N/A | consumes typed cause when present; sentinel only for old `None` producers | typed cause or fallback |
+
+The legacy bridge's `StopCause::None` sentinel branch is intentionally retained
+until every old producer is migrated. It must never run when a real
+`StopCause` is present.
+
+`min_tokens` is a separate, still-unimplemented sampling contract: vLLM
+requires it to mask EOS and explicit stop IDs until the requested completion
+count, but the shared request types do not carry that threshold. Both bridge
+generations therefore reject a non-zero value at the common wire-validation
+boundary instead of letting legacy models silently ignore it. Implementing the
+masking semantics is a follow-up that must add the field to the scheduler
+contract and every sampler path.
+
+## Execution Log
+
+### Shared and model migration
+
+- Added the shared `StopPolicy` / `StopCause` types and threaded them through
+  the request, ledger, step, and event contracts.
+- Updated wire conversion so `ignore_eos` does not disable explicit
+  `stop_token_ids`.
+- Updated Qwen3, Qwen3.5, DeepSeek-V2-Lite, Kimi-K2, GLM5.2, K3, and Qwen3
+  speculative paths to emit the trigger token before terminal metadata.
+- Added native GLM5.2 P/D serialization of the typed stop cause and preserved
+  the anchor/cause distinction.
+- Unified `min_tokens` handling at the shared vLLM wire boundary; legacy and
+  stepped bridges now fail closed with the same error until sampler masking is
+  implemented.
+
+### K3 multi-token contract tests
+
+- Extended `pegainfer-k3/src/scheduler/tests.rs` with a scripted
+  `decode_many` fixture.
+- Added tests for an explicit stop in the middle of a span, a length cap in the
+  middle of a span, and EOS precedence over an overlapping explicit stop ID.
+- The tests assert emitted IDs, suffix removal, `StopCause`, completion count,
+  and slot release.
+
+### Verification
+
+- `cargo fmt --all` — pass (run from the Linux login shell).
+- `git diff --check` — pass.
+- `cargo test --release -p pegainfer-frontend --lib` — **77 passed, 0 failed**.
+  This includes both legacy and stepped `min_tokens` rejection tests, typed
+  explicit-stop mapping, EOS handling, and the shared wire-policy tests.
+- `cargo test --release -p pegainfer-qwen3 --lib` — **93 passed, 0 failed**.
+  This covers prefill/decode, speculative-span truncation, EOS/explicit-stop
+  precedence, and request cleanup.
+- `cargo test --release -p pegainfer-deepseek-v2-lite --lib` —
+  **5 passed, 0 failed**.
+- `cargo test --release -p pegainfer-sim --tests -- --test-threads=1` —
+  **22 passed, 0 failed** (7 unit, 12 frontend HTTP, 3 tool-call round-trip).
+  The serial test flag is required because several simulator tests bind a
+  fixed localhost port.
+- `cargo build --release -p pegainfer-server --bin pegainfer` — pass when the
+  user-local `protoc-31.1` and CUDA 12.8 library paths are selected (see the
+  command below). The default login environment links the obsolete system
+  CUDA 10.1 libraries and is not a valid build environment for this tree.
+- `cargo test --release -p pegainfer-k3 --lib scheduler::tests` — blocked before
+  Rust test compilation by the server CUDA toolchain: the installed headers do
+  not define `CUmemFabricHandle`, `CU_MEM_HANDLE_TYPE_FABRIC`, or
+  `CU_DEVICE_ATTRIBUTE_HANDLE_TYPE_FABRIC_SUPPORTED`; TileLang is also absent.
+  This is a mainline K3 environment prerequisite, not a stop-policy compiler
+  error.
+
+### Qwen3 HTTP smoke
+
+- Server: Qwen3-0.6B at `/home/ricardo.zheng/models/Qwen3/Qwen3-0.6B`, served
+  as `qwen3-0.6b` on `127.0.0.1:18080`; `/v1/models` returned HTTP 200 and the
+  expected model metadata.
+- `ignore_eos=true`, `max_tokens=8`: HTTP 200, `finish_reason=length`,
+  `stop_reason=null`, `completion_tokens=8`. This proves ignored EOS does not
+  terminate the request early.
+- `stop_token_ids=[0..151935]`, `ignore_eos=true`: HTTP 200,
+  `finish_reason=stop`, `stop_reason=12095`, `completion_tokens=1`. The first
+  generated token was preserved and reported as the actual explicit stop ID.
+- `min_tokens=1`: HTTP 500 with the standard OpenAI error envelope. The server
+  log contains the detailed rejection, and the request is rejected before
+  scheduler submission by the shared wire validator.
+
+## Remaining Risks
+
+- K3's real DSpark executor performs speculative KV work before the scheduler
+  applies request stop policy. Terminal slot release resets that state, so no
+  suffix reaches the next request, but this is extra work rather than early
+  executor-side truncation.
+- The legacy sentinel fallback must be removed only after each remaining
+  producer has a resolver and real HTTP lifecycle gate.
+- A full GPU K3 test should be rerun after the server upgrades CUDA headers and
+  installs TileLang.
+
+## Next Step
+
+HTTP smoke is complete. Keep the legacy sentinel fallback as a separate
+migration after maintainer feedback; it is not required for the typed-cause
+paths covered here.
+
+## Debrief
+
+- **Outcome:** The shared stop-policy migration is implemented and verified at
+  unit, model-crate, simulator HTTP, and real Qwen3 HTTP levels. EOS, explicit
+  stop IDs, trigger-token preservation, completion counts, speculative suffix
+  truncation, and fail-closed `min_tokens` behavior are covered.
+- **Environment caveats:** The default login shell selects obsolete CUDA 10.1
+  libraries and `protoc 3.6.1`; the successful build used the user-local
+  `protoc-31.1` and CUDA 12.8 paths recorded below. K3 GPU validation remains
+  blocked by missing CUDA fabric headers and TileLang.
+- **Follow-up:** Before opening a PR, perform the final diff review, decide
+  which local-only `.codex` artifacts stay untracked, then stage only the
+  intended source and documentation files. Do not remove the legacy sentinel
+  fallback in this change.
+
+Linux build environment used for the successful checks:
+
+```bash
+export PROTOC=/database/ricardo.zheng/.local/opt/protoc-31.1/bin/protoc
+export CUDA_HOME=/usr/local/cuda-12.8
+export LIBRARY_PATH=/usr/local/cuda-12.8/lib64:/usr/local/cuda-12.8/targets/x86_64-linux/lib:/usr/lib/x86_64-linux-gnu
+export LD_LIBRARY_PATH=/usr/local/cuda-12.8/lib64:/usr/local/cuda-12.8/targets/x86_64-linux/lib:/usr/lib/x86_64-linux-gnu
+export RUSTFLAGS=-Lnative=/usr/local/cuda-12.8/lib64
+```