feat(qwen3): vLLM-prefill P/D — cross-engine KV compat, verified end-to-end

[xiaguan]

What

Qwen3 decode nodes can now consume KV prefilled by a vLLM + PegaKVConnector peer over the pegaflow P2P mesh: vLLM prefills and seals blocks under its own prefix-cache hashes; openinfer-D derives byte-identical keys, discovers the blocks via the MetaServer, RDMA-fetches and restores them, and decodes — zero vLLM/connector changes.

• openinfer-kv-offload/src/vllm_hash.rs — VllmBlockHasher: replicates vLLM's xxhash_cbor block-hash chain (hand-rolled canonical CBOR + xxh3_128, NONE_HASH from PYTHONHASHSEED). Golden vectors captured from a real vLLM process.
• D-side wait window — a cold request whose query misses re-queries (5ms/req throttle) until the producer's registration lands (--kv-pd-miss-wait-ms, default 5000), bridging the P→D save/publish race. remote_fetch.rs decision cascade extended with wait_on_miss; 12 unit tests cover the branch matrix.
• Unhappy path is loud and bounded (toxic-review hardening): startup fingerprint log (seed/namespace/block_size/NONE_HASH/KV geometry — diff it against the P side), warn on every exhausted miss window, and a miss breaker — after 3 consecutive exhausted windows (or 15s hard timeouts) new requests skip the wait until a remote hit re-arms it. Flags validate at parse time (seed must be decimal u32, namespace 8-hex); miss-wait >= 15s deadline is rejected at startup.
• Server flags — --kv-pd-vllm-seed, --kv-pd-vllm-namespace, --kv-pd-miss-wait-ms (all gated on the P2P mesh flags).

Compatibility contract (verified on hardware)

P's blocks are loadable by D iff: same namespace (an 8-hex digest that does not include block size), same block granularity + hash scheme (P must run --block-size <D's page size> and --prefix-caching-hash-algo xxhash_cbor with a pinned PYTHONHASHSEED), equal per-layer slot counts (36 for Qwen3-8B on both sides), and compatible per-slot K/V segment layout — the last is absorbed by pegaflow novitalabs/pegaflow#382 (contiguous device layouts loading split-KV-sealed blocks). Full derivation in docs/models/glm52/pd-vllm-prefill.md §3.1.

Evidence (jz node 34, H200 ×8: vLLM-P on GPU0, openinfer-D on GPU1)

Correctness — 3-prompt greedy smoke (short / cross-block / block-aligned) via the P/D router vs direct-D baseline: all outputs byte-identical. Content-addressed delta reuse confirmed: the aligned prompt (sharing a 480-token prefix with the previous one) fetched exactly 1 incremental block.

TTFT (unique-prefix cold runs, median of 3; overhead = PD via router − P direct):

prompt tokens	P direct	P/D	handoff overhead
~470	34.1 ms	47.7 ms	+14 ms
~1790	54.9 ms	105.9 ms	+51 ms
~7070	203.9 ms	351.0 ms	+147 ms

The 7k overhead is dominated by physically moving ~1 GiB of KV (RDMA fetch alone: 992 MiB in 41.8 ms, 23.7 GiB/s); pipelining P-side save / discovery / H2D is follow-up territory.

Failure injection — P killed, 4 sequential cold requests: first 3 wait the 5s window (one warn each), breaker opens, 4th fails over in 38 ms; P restored, one routed hit re-arms waiting.

Not in this PR (tracked in the design doc)

• Tail-block connector extension + router t1-forwarding (D currently computes the ≤ block_size partial tail locally).
• Strict no-prefill mode (429/500 on miss instead of scratch fallback).
• Namespace↔model-identity canary handshake (the namespace digest carries no model identity; the startup fingerprint is the current guard).

Dependencies

Runtime-correct only with pegaflow ≥ #382; builds fine against the currently pinned rev (the fix is load-path-only). I'll bump the pegaflow-core rev here once #382 merges.

Test plan

• cargo test --release -p openinfer-kv-offload -p openinfer-qwen3 --lib — golden vectors + remote-fetch branch matrix, all green.
• e2e smoke + TTFT + failure injection on node 34 as above (/data/pd-stack/{stack,smoke,failtest}.sh, ttft.py).

🤖 Generated with Claude Code