fix(#728): proactive nudge for upstreams that never report usage

CHANGELOG.md

@@ -7,6 +7,7 @@ Versions follow the merge of a `*_release-v*` branch; CI publishes to npm on tag
 
 ### Fixes
 
+- **Proactive nudge fires on upstreams that NEVER report usage — no more unbounded climb to the hard limit (#728)**: ChatGPT-login-style backends omit `usage.input_tokens` entirely (`response.completed` carries none), so `lastInputTokens` stayed 0 for the whole session — and the kernel's `decideNudge` is structurally unfireable at tokenCount == 0 (growth reference falls back to tokenCount itself → growth ≡ 0; mass-ready and all pressure bands require usage ≥ their pct lines). Proactive compression therefore never fired and contexts climbed unbounded until preflight's hard-limit emergency path took over (#726 reached ~1.32M tokens against a 1.05M window before the ~630K-token emergency summarization — which then 502-looped). `effectiveTokenCount` fed 0 for explicit-identity zero-baseline sessions on the "self-heals via the next measured usage report" assumption, which breaks for never-reporting upstreams. The fix mirrors how OpenAI handles exactly this in codex itself (server-observed usage first-class, local estimator fallback underneath — verified against `openai/codex` sources during triage): each `prepare*` now records the char-count upper bound of THAT turn's outbound payload (post-fold processed messages + system/tools overhead + image tokens) into `session.stats.localInputEstimate`, and `effectiveTokenCount` feeds it ONLY while `lastInputTokens == 0`, capped by the current request's inbound upper bound so a stale-high reading can't outlive a client-side shrink. Real usage always takes precedence the moment any report lands (same invariant as #604's `armFailureShrink` exception — the upper-bound estimator only errs early, compresses earlier, never later); turn 1 of a fresh explicit session is byte-identical to before (nothing measured yet → still 0); anonymous-prefix-affinity forks keep their raw-inbound-bound regime untouched; the estimate self-corrects after every fold (post-fold outbound payload shrinks → next reading drops, no inflation even though proxy clients re-send full raw history every turn); it is cleared on the native-compaction-boundary reset. Net effect: silent-backend sessions get incremental compression from the moment the local estimate crosses the nudge thresholds instead of one giant emergency fold at the wall. Rebase onto current master exposed an interaction with #817's per-body preflight dead-end cooldown key: the nudge text injected into the system prompt changes every turn, so keying the cooldown on the sha256 of the proxy-REBUILT wire body misses on identical client retries once the fallback arms the nudge — every retry re-burned the doomed summarization walk, defeating #726's quota protection on exactly the silent backends this fix targets. The cooldown identity now hashes the client's raw inbound request body instead, so proxy-side injections (nudge/system/tools/tags) can no longer defeat it. Tests pin: silent-backend multi-turn (turn 1 idle → turn 2 nudges, preflight silent), real-usage precedence (reported tokens beat a ~90% payload), stale-high cap (shrunk history does not re-trigger).
 - **Bump acp-kernel 0.0.74 → 0.0.77; `acp_cache` held off every wire surface until adoption (#800)**: the pin jumps three kernel releases — 0.0.75 (cache-reconciliation core: `acp_cache` tool + `buildCacheReport`; recommend tail-below-gate fix), 0.0.76 (Google/Gemini wire codec; recommend range integrity), 0.0.77 (nudge ranges ordered/merged by position instead of ref number — fixes #887). Kernel ≥0.0.75 ships a fifth proxy tool, `acp_cache`, but this proxy has no execution path for it: an intercepted call fell through `executeProxyTool` into the unknown-tool branch, which reports SUCCESS with `[Unknown proxy tool: acp_cache]` — a model calling it would get a plausible-looking result that did nothing. Until adoption lands (feature #800, owner decision pending), `src/compress-tool.ts` derives `PROXY_TOOL_NAMES` and the four `BILI_ACP_TOOLS_*` arrays from the kernel constants minus `UNADOPTED_KERNEL_TOOLS` (`{acp_cache}`) — the single choke point every serving surface consumes (Responses/OpenAI/Anthropic tool arrays, plugin manifest + allowed list, absorb gate, Responses-adapter classification), so the tool stays off the wire everywhere without touching call sites. `MUTATING_PROXY_TOOLS`/`READONLY_PROXY_TOOLS` stay raw re-exports (classification sets only; `acp_cache` is correctly classified readonly and no serving surface reads them). The #841 parity test (`tests/search-context-cross-session.test.ts`) extends the "no other tool may change" contract against the FILTERED kernel side only, so a future BILI-side leak of an unadopted tool still fails.
 - **A render tag wrapped around the model's whole turn no longer reaches the client as orphan markup (PR #872)**: a model imitating the render tag can write its entire turn — tool call included — where the opening's attributes are still open (recorded shape: `<acp tokens="1" text="text` immediately followed by the turn's own markup). The span escapes every tag pattern at once: an attribute class stops at `<`, and the value opened with `"` is never closed, so no close tag is ever written. The filter dropped the opening and released the payload, and the agent received a completed turn of orphan `parameter`/`invoke`/`calls` closers with no tool call in it — it stalled until nudged by hand. The stripper now recognises the wrapped shape by form (the attribute list runs into a `<`, or carries an odd number of quotes) and swallows the span whole, to a loose close tag or to the end of the turn, in `stripAcpTags` and in the streaming filter alike; the turn then reaches the client empty, which the degenerate-turn retry (#870, PR #871) re-asks for. The #644 rule is unchanged: an over-long tail after a *plain* opening is still released as content.
 

src/server.ts

@@ -1528,6 +1528,7 @@ async function handle(
                         prepared,
                         runPrepare,
                         req,
+                        bodyBuffer,
                         res,
                         opts,
                         core,
@@ -1821,13 +1822,35 @@ function diagNudge(turn: { nudge?: { shouldInject: boolean; reason: string; cont
 // anonymously (prefix-affinity forks/reloads, #553): they carry the full raw
 // history but no measurement yet, so feeding 0 blinds the nudge (usage 0%,
 // growth ref 0) and no compression trigger fires until overflow. Explicit-
-// identity zero-baseline sessions stay at 0 — first-turn or post-native-
-// compaction payloads that are small by construction and self-heal via the
-// next measured usage report.
+// identity zero-baseline sessions previously stayed at 0 on the assumption
+// that they "self-heal via the next measured usage report" — an assumption
+// that breaks for upstreams that NEVER report usage (ChatGPT-login backends,
+// #728): lastInputTokens stays 0 for the whole session, and the kernel's
+// decideNudge is structurally unfireable at tokenCount == 0 (growth ref falls
+// back to tokenCount itself → growth ≡ 0; firstSightMassReady/pressure bands
+// all require usage >= their pct lines). #728 fix: such sessions fall back to
+// the PREVIOUS turn's locally-measured outbound payload upper bound
+// (session.stats.localInputEstimate, recorded in prepare* each turn). The
+// estimator only errs EARLY (char-count upper bound → compress earlier, never
+// later), is active only while lastInputTokens == 0 (a real usage report takes
+// precedence immediately — same invariant as #604's armFailureShrink
+// exception), and self-corrects after every fold (the post-fold payload
+// shrinks → the next estimate drops). Turn 1 of a fresh explicit session
+// still feeds 0 (nothing measured yet; nothing pending either), so
+// first-turn behavior is byte-identical to pre-#728.
 function effectiveTokenCount(session: Session, msgs: CoreMessage[]): number {
     if (session.stats.lastInputTokens > 0) return session.stats.lastInputTokens;
-    if (!session.metadata.anonymousPrefixAffinity) return 0;
-    return estimateCoreMessagesUpper(msgs);
+    if (session.metadata.anonymousPrefixAffinity) return estimateCoreMessagesUpper(msgs);
+    const est = session.stats.localInputEstimate ?? 0;
+    if (est <= 0) return 0;
+    // Cap by THIS request's inbound upper bound: the recorded estimate lags by
+    // one turn, so after a client-side shrink (native compaction echo, history
+    // edit) the previous turn's payload can be larger than what is in front of
+    // us now — never claim more context than the current request could hold.
+    // In steady state est <= raw bound always (the outbound fold is never
+    // larger than the inbound history), so this is a no-op there.
+    const raw = estimateCoreMessagesUpper(msgs);
+    return Math.min(est, raw);
 }
 
 function prepareAnthropic(
@@ -1956,6 +1979,16 @@ function prepareAnthropic(
     // identity chain (#268), not part of the Anthropic Messages API — strip it
     // so the real upstream never sees a field it doesn't know.
     delete (rebuilt as Record<string, unknown>).prompt_cache_key;
+    // #728: record the char-count upper bound of THIS turn's outbound payload
+    // (post-fold messages + system/tools overhead + images) as the fallback
+    // token source for upstreams that never report usage — read only while
+    // lastInputTokens == 0 (effectiveTokenCount). When the kernel transform
+    // above failed, processedMessages is empty and the forwarded body is the
+    // UNPROCESSED projection — measure that instead so the fallback isn't
+    // blinded to a system+tools-only floor.
+    session.stats.localInputEstimate = estimateCoreMessagesUpper(processedMessages.length > 0 ? processedMessages : originalMessages)
+        + countSystemAndToolsTokens(extractSystem(systemOut), toolsOut)
+        + imageTokensInParsedBody("anthropic", rebuilt);
     return { body: JSON.stringify(rebuilt), session, processedMessages, originalMessages, anthropicSystem: parsed.system, protocol: "anthropic", stream, compressInjected: injectTools, pluginMode, nudge, prompts, surface, renderTags: "text-only" } as Prepared;
 }
 
@@ -2113,6 +2146,15 @@ function prepareOpenai(
     if (!isTitleGen && openaiOutboundSystem !== undefined) {
         session.metadata.systemPromptTokens = countSystemAndToolsTokens(openaiOutboundSystem, toolsOut);
     }
+    // #728: record this turn's outbound payload upper bound as the fallback
+    // token source for upstreams that never report usage (see effectiveTokenCount).
+    // Title-gen side requests are skipped like the overhead row above — their
+    // tiny payload would clobber the conversation's measurement.
+    if (!isTitleGen) {
+        session.stats.localInputEstimate = estimateCoreMessagesUpper(processedMessages.length > 0 ? processedMessages : originalMessages)
+            + countSystemAndToolsTokens(openaiOutboundSystem || openaiSystemText, toolsOut)
+            + imageTokensInParsedBody("openai", rebuilt);
+    }
     snapshotMessages(session, originalMessages);
     markDirty(session);
     return { body: JSON.stringify(rebuilt), session, processedMessages, originalMessages, protocol: "openai", stream, compressInjected: injectTools, pluginMode, nudge, prompts, surface, openaiSystemText, renderTags: "text-only" } as Prepared;
@@ -2373,6 +2415,15 @@ function prepareResponses(
     if (transformOk) {
         session.metadata.systemPromptTokens = countSystemAndToolsTokens(responsesDevContent ?? "", toolsOut);
     }
+    // #728: record this turn's outbound payload upper bound as the fallback
+    // token source for upstreams that never report usage (see effectiveTokenCount).
+    // Compaction-trigger requests are the compression mechanism itself — no
+    // incremental decision hangs off them, so don't leave a stale reading.
+    if (!isCompactionTrigger) {
+        session.stats.localInputEstimate = estimateCoreMessagesUpper(processedMessages.length > 0 ? processedMessages : originalMessages)
+            + countSystemAndToolsTokens(responsesDevContent ?? "", toolsOut)
+            + imageTokensInParsedBody("responses", rebuilt);
+    }
     snapshotMessages(session, originalMessages);
     markDirty(session);
     return {
@@ -2829,6 +2880,7 @@ async function preflightCompressIfNeeded(
     prepared: Prepared,
     runPrepare: () => Prepared,
     req: http.IncomingMessage,
+    inboundBody: Buffer,
     res: http.ServerResponse,
     opts: ProxyOptions,
     core: CompressionCore,
@@ -2950,7 +3002,12 @@ async function preflightCompressIfNeeded(
     // stale-baseline fit): those return without running the walk, so the
     // cooldown must not convert a fitting payload into a false fail-fast while
     // a marker from a larger earlier request is still warm.
-    const deadEndKey = `${model}\u0000${limit}\u0000${createHash("sha256").update(prepared.body).digest("hex")}`;
+    // #726 identity is the CLIENT request, not the rebuilt wire body: proxy-side
+    // injections vary turn to turn (the nudge text changes every turn; #728's
+    // silent-backend fallback arms the nudge on exactly these never-report-usage
+    // upstreams), so hashing prepared.body misses the cooldown on retry and
+    // re-burns upstream quota — the failure #726 exists to prevent.
+    const deadEndKey = `${model}\u0000${limit}\u0000${createHash("sha256").update(inboundBody).digest("hex")}`;
     const deadEnd = session.metadata.preflightDeadEnd;
     if (deadEnd && typeof deadEnd === "object") {
         const de = deadEnd as Record<string, unknown>;

src/session.ts

@@ -107,6 +107,16 @@ export type Session = {
         pendingFoldUsage?: boolean;
         /** Current in-context (uncompressed) token count at last processTurn. */
         contextTokens: number;
+        /** #728: char-count upper bound of the LAST turn's outbound payload
+         *  (post-fold processed messages + system/tools overhead + images),
+         *  recorded locally in prepare* each turn. Read ONLY while
+         *  lastInputTokens == 0, as the fallback tokenCount for upstreams
+         *  that never report usage (ChatGPT-login-style backends — see
+         *  effectiveTokenCount in server.ts). Self-correcting: a successful
+         *  fold shrinks the next turn's payload and thus the estimate.
+         *  Cleared by resetSessionCompression (native-compaction boundary).
+         *  Persisted (survives restart like the rest of stats). */
+        localInputEstimate?: number;
     };
     /** Free-form escape hatch for future fields not yet promoted to typed
      *  members. Persisted as-is (must be JSON-serializable). Use sparingly —
@@ -378,6 +388,10 @@ export function resetSessionCompression(session: Session): void {
     session.stats.lastInputTokens = 0;
     // #857: a zeroed baseline carries no provenance — drop any stale flag.
     delete session.stats.lastInputTokensSource;
+    // #728: the pre-compaction outbound payload is gone — the old estimate
+    // (measured against the pre-compaction wire) would read high and blind
+    // the nudge fallback early; let the next prepare* re-measure.
+    session.stats.localInputEstimate = 0;
     session.stats.contextTokens = 0;
     session.metadata.nativeCompactionAt = Date.now();
     markDirty(session);