feat(vllm-model): supply the previous call's exact training tokens

[ananthsub]

Supplies a request-time resolved parent's exact cumulative tokens to compatible vLLM generation backends and records generation-time proof that the backend used them.

Prefix-supply control flow

sequenceDiagram
    participant H as Harness
    participant M as Gym vLLM model server
    participant L as LineageStore
    participant V as vLLM backend
    participant S as TokenSink

    H->>M: continuation request
    M->>L: resolve(request items)
    alt RESOLVED
        L-->>M: parent call + exact cumulative tokens
        M->>V: generation request with required prefix token IDs
        V-->>M: response + generation-time prompt token IDs
        M->>M: compare observed prompt prefix with requested prefix
        M->>S: put(TokenEntry, prefix_requested=true, prefix_supplied=proof))
    else ROOT or UNRESOLVED
        L-->>M: no proven parent tokens
        M->>V: ordinary generation request
        V-->>M: response
        M->>S: put(TokenEntry, prefix_requested=false)
    end

Loading

Summary

• Consumes only a typed RESOLVED lineage result. ROOT, UNRESOLVED, and lookup failures leave the ordinary generation request unchanged.
• Requests the exact cumulative token sequence captured for the verified parent, avoiding a second chat-template rendering at the continuation boundary.
• Records prefix_requested separately from prefix_supplied; supply is true only when generation-time prompt-token evidence proves the requested prefix was used.
• Fails startup for unsupported completions-API paths when supply is enabled, so a training run cannot silently claim inconclusive prefix use.
• Keeps prefix diagnostics concurrency-safe and preserves metadata through capture-context construction.
• Advances the token-entry schema to version 4 for the prefix-request/proof fields layered on feat(token-id-capture): resolve each call's parent at request time #2180's schema version 3.
• Covers resolved, root, unresolved, unsupported backend, completions API, and generation-proof paths.

Depends on #2180. Documentation and adapter tests are in #2349.

[copy-pr-bot]

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[ananthsub]

/claude review

[claude]

SHIP — no correctness or async-safety concerns.

Reviewed the full diff plus the PR-head app.py, sink.py, records.py, the resolve_parent call sites in base_responses_api_model.py, and the lineage/client APIs. This is a tight, well-tested opt-in feature (prefix supply for chained training trajectories).

What I checked and why it's safe:

• Fail-closed verification. _verify_generation_prefix requires generation-time prompt_token_ids proving the engine applied the prefix and raises RuntimeError (propagates to a 500, fails the run) if the backend ignores or omits it. A misconfigured backend fails loud instead of silently recording a re-rendered prompt against a spliced generation — correct for training-data integrity.
• Conservative supply. Only fires on a unique verified parent resolved once before dispatch; otherwise the request is forwarded untouched (cold cache, never a wrong prefix). Fork/ambiguous/fingerprint-miss cases are covered by tests.
• Tokenize path. Correctly skipped only when supply verified, substituting the proven prompt ids; TestPrefixSupplyReachesTokenize guards the both-calls-must-agree hazard.
• Config guard. supply_prefix_token_ids ⇒ return_token_id_information enforced by a model_validator, which the verification path relies on.
• Schema v3. Backward-compatible — prefix_supplied defaults False, and _refuse_a_newer_record still refuses genuinely-newer records loudly.

Two NOTEs (author's call, both fail safe, not merge-blocking):

• completions-API path. _apply_prefix_supply runs inside _preprocess_chat_completion_create_params, which the use_completions_api=true path never invokes (per the comment at app.py:1140). Enabling both flags together makes supply silently no-op (records via tokenize, still correct — just no chaining benefit and no accounting signal). Given the effort spent on auditability, a one-line config warning or validator note when both are set would prevent an inconclusive experiment.
• _prefix_supply_counts accounting. In-process counter mutated without a lock under concurrent requests; the += reads/writes have no intervening await so it's effectively fine under the GIL, and it's diagnostic-only. No correctness impact.

Test coverage is real behavioral assertions, not pass-throughs. LGTM.

[ananthsub]

/claude review

[claude]

Review incomplete: the PR revision changed during analysis. The base SHA captured at the start of this review (7c505f01bfaa163f65cd276d815666f059469620) no longer matches the current base (4df7a80816817fd97cbcdde44ae5ad18833c0092), so the diff I analyzed may no longer reflect the head under review. No inline findings are being posted. Please re-run the review against the updated revision.

[ananthsub]

/claude review