Migrate qwen3 onto openinfer-kv-store (retire the remote-fetch state machine)

[xiaguan]

Context

PR #825 merged openinfer-kv-store: a self-contained KV read/write orchestration crate (BlockPool absorbed from openinfer-kv-cache, direct deps on kvbm-logical + pegaflow-core; PegaflowHost + KvStoreBuilder::rank_with_offload; resolve_prefix/seal/retire with a single terminal KvPrefix; real-GPU + io_uring-SSD test suites, no mocks). Design doc: docs/subsystems/kv-cache/design.md. This issue tracks migration plan step 2: the qwen3 first migration.

Goal

Replace qwen3's hand-written offload orchestration — the PrefetchPhase (RemoteFetch → Loading → Committed) / remote_fetch_action machinery in openinfer-qwen3/src/executor.rs (~2k lines) — with KvStore::resolve_prefix / seal / retire. Keep the old path behind a switch until bench alignment, then delete it.

Mapping that already exists (verified against the skeleton)

• remote_fetch_action decision cascade → resolve re-query loop (release lease before every wait, deadline backstop)
• saved_cursor → SaveCursor; fire-and-forget save_sealed_blocks → seal(..., SaveClass::Cacheable)
• flush_on_finish / release_finished_events barrier → SaveClass::Handoff + KvStore::flush_saves
• Drop-request blocking until in-flight H2D DMA settles → store-owned (the reservation rides the detached load task)
• REMOTE_FETCH_DEADLINE 15 s / REMOTE_REQUERY_INTERVAL 5 ms → with_resolve_deadline / with_requery_interval (same defaults)

Migration checklist

• Intake: stop discarding _kv_prefix in openinfer-qwen3/src/scheduler.rs (SubmittedRequest = (GenerateRequest, KvPrefix) is already plumbed through EngineHandle::submit_resolved); bind the rank before resolve; drop the hold right after match_and_add_prefix
• Save hook: replace the six save_sealed_blocks call sites with per-boundary seal; drop_request → retire
• Scheduler admission: subtract store.pinned_blocks(rank) from the usable budget; keep the prefetched_blocks credit so a resolved hit is never double-counted
• KV event feed parity: take_kv_store_events + lineage→seq-hash translation stays model-side for now
• Absorb the runtime toggles: prefix_cache_enabled / l1_retention_disabled → resolve short-circuit / no-match at admission / reset-on-release
• Echo requests keep skipping probe/prefetch entirely
• TP: keep today's single-GPU offload assert; per-rank pool wiring is a later step

Sequencing with the vLLM P/D cut

Recorded decision: vLLM P/D interop will be removed wholesale in a separate PR (vllm_compat state, VllmBlockHasher, the miss_wait window, and kv-store's page_first registration mode + its test). Landing that deletion FIRST shrinks this migration's surface: no hasher/key-scheme pluggability is needed on the store side, and ResolvePolicy::wait_for_full_hit remains the only all-or-nothing consumer.

Deferred design decisions (deliberately not pre-built)

• Admission arbitration: the skeleton's set_admission_floor was dropped as unconsumed API. Its return shape should be admission-managed budgeting — resolved-hit credits and reservations in the scheduler's own ledger (qwen3's reserve_floor precedent) — not a dedicated watermark API. The in-pool TOCTOU close-out (async reserve_blocks) is discussed alongside it.
• miss-breaker (stop parking after consecutive cold misses): store vs scheduler placement — decide during this migration.
• Handoff lease release timing (on D-side fetch completion vs on admission assert): pin pegaflow lease semantics during the glm52 P/D step.

Verification

• openinfer-qwen3/tests/kv_offload_cpu_hit.rs (save → flush → CPU-hit restore → logits) green on the new path
• hf_golden_gate accuracy gates green
• Bench parity old-vs-new behind the switch on prefix-warm workloads (TTFT), then remove the old path in the same PR train

Design doc open-questions section carries the full context; glm52 (step 3) and qwen35 (step 4) follow the same pattern.

[HasonoCell]

@xiaguan Hi! I'd like to explore this. Could you assign this to me?

[xiaguan]

Hi @HasonoCell, thanks! This change is very large — not a good first contribution. Would suggest starting with a smaller issue first;

[HasonoCell]

Hi @HasonoCell, thanks! This change is very large — not a good first contribution. Would suggest starting with a smaller issue first;

Sure! I will pick a smaller issue that is good for a first contribution, thanks for your reply!