prune: summary anchor can land mid-conversation (role:system after a user message) — 400 on strict OpenAI backends

[ranxianglei]

What

collectSummaryAnchors (src/prune.ts, dist/index.js:136-165) sets insertAt = earliest ?? 0 where earliest is the index of the block's earliest covered message in the input messages. rebuildMessages (dist/index.js:166-189) then inserts the rendered summary — renderSummary produces role: "system" (dist/index.js:190-201) — at that index.

Whenever the covered range does NOT start at the conversation head, the anchor sits after one or more still-visible messages, and the rebuilt message list carries a system message mid-conversation:

[user(m00001), assistant(m00002), system(summary), user(m00011), ...]

This is not limited to multi-segment compress: any compress whose range starts after the head (the normal "fold the old stuff, keep the tail" pattern) produces it. The only safe shape is a range starting at index 0.

Note the kernel already preserves the first user message unconditionally (rebuildMessages, dist/index.js:176-179: if (index === firstUserIndex && firstUserIndex >= 0) { result.push(...); continue; }), so even a head-anchored range m00001–mNNN leaves m00001 visible — the summary lands at index 0, ahead of it, which is fine. The problem is exclusively anchors at index > 0.

Impact

• OpenAI chat-wire hosts: coreToOpenai serializes role: "system" verbatim at its position, so strict OpenAI-compatible backends reject every subsequent request. Reproduced in production with sglang (qwen3.8-27B): 400 System message must be at the beginning. on ALL requests after the compress; the session is stuck until the provider is switched or the proxy bypassed. See 多段 compress 夹 user 消息时 T1 摘要 (role:system) 落在对话中部, sglang 报 400 "System message must be at the beginning" billion-context#355.
• In-process hosts (pi/opencode plugins) that consume core messages directly carry the mid-conversation system message into whatever wire they build.

Suggested fix

Clamp the anchor to the first user message in collectSummaryAnchors — firstUserIndex is already computed in prune() (dist/index.js:117-119) but only passed to rebuildMessages:

insertAt: Math.min(earliest ?? 0, firstUserIndex)

This folds every summary into the leading system prefix. Semantics are unchanged: a summary is a stand-in for the folded history, and a leading system prefix is accepted by both OpenAI and Anthropic (the anthropic wire already maps non-assistant roles to user, so it is unaffected today).

Alternative considered: render summaries as role: "user" — works everywhere but changes role semantics for every host.

Interim mitigation

billion-context PR ranxianglei/billion-context#356 hoists mid-conversation system/developer messages to the leading prefix at the OpenAI wire layer (both prepareOpenai and the compress-loop adapter), so proxy users are covered without a kernel release. This issue tracks the kernel-level root fix so all hosts benefit.

[ranxianglei]

🤖 ework agent · vllm-qwen/qwen3.8-27b

Fixed in PR #171 → #171

Placement fix (src/prune.ts) — every summary anchor is clamped to the first non-system message, so rendered summaries always land in the leading system prefix and never mid-conversation. I clamped to the first non-system message rather than firstUserIndex so it also covers hosts that lead with a non-user, non-system message (clamping to firstUserIndex alone would still leave a system-after-assistant 400 in that shape). Summaries keep their chronological order via an unclamped orderKey, so multi-summary ordering is unchanged apart from moving into the prefix.

One thing the minimal proposal misses — insertAt: Math.min(earliest ?? 0, firstUserIndex) on its own regresses block consumption. blockVisibleInRange detects a block in-range via its summary position or earliest surviving raw; once the summary moves to the head and the block's raws are all pruned, neither is in the range, so a message-ref compress spanning a pruned region no longer folds the block in (broke the promote-after-prune "m-ref range spanning a pruned region" case — multi-tier distillation). So blockVisibleInRange now falls back to the block's content refs when no raw survives in the view, gated on rawIndex === null so a block whose anchor raw is merely outside the range (but present) isn't wrongly treated as nested (that gate keeps the "covered by non-nested block" validation rejecting correctly).

Tests — 2 new regression tests pin the wire-validity invariant (no system message after a non-system message), including the leading-system-message case; 3 existing tests updated because their exact position assertions encoded the old buggy placement. Full suite 542 pass, typecheck + build clean.

I also verified end-to-end (prune → coreToOpenai): the wire now serializes system user user assistant instead of the old user system user assistant that triggered the sglang 400.