perf: avoid transform work that scales with compression history (#384)

devlog/2026-09-11_perf-transform-history/DESIGN.md

@@ -0,0 +1,128 @@
+# DESIGN - Avoid transform work that scales with compression history
+
+- Task ID: `2026-09-11_perf-transform-history`
+- Issue: https://github.com/ranxianglei/opencode-acp/issues/384
+
+## Problem shape
+
+The message-transform hook runs on **every LLM call**. Several of its steps —
+and the compress tool's candidate validation — rebuilt lookup structures whose
+size grows with *total compression history* (`messageIds.byRef` is never
+reclaimed between compactions; `blocksById` keeps consumed blocks forever for
+decompression/fork recovery). Cost therefore scaled O(history) per transform
+instead of O(visible messages + active blocks).
+
+## Design decisions
+
+### 1. Request-scoped boundary lookup memo (RC1)
+
+`buildBoundaryLookup()` maps every known ref/bid to a `BoundaryReference`. It
+depends only on `(searchContext.rawIndexById, state.messageIds.byRef, active
+blocks)` — all immutable **within one request** (a single compress tool call
+or one transform). So it is built once per request:
+
+- `SearchContext.boundaryLookup?: BoundaryLookup` (new OPTIONAL field,
+  `lib/compress/types.ts`). `buildSearchContext()` fills it eagerly;
+  `resolveBoundaryIds()` falls back to `context.boundaryLookup ??= buildBoundaryLookup(...)`
+  so hand-built contexts (tests, `search_context` tool) keep working unchanged.
+- No invalidation machinery needed: the context object is request-scoped by
+  construction and never mutated after creation.
+
+### 2. Fast token estimate in candidate selection (RC1b)
+
+`resolveSelection()` only needs token counts to report range sizes and feed
+quality-gate estimates — it does NOT need exact BPE counts. The codebase
+already uses `chars/4` as its estimation convention everywhere else; the hot
+path alone called the real Anthropic tokenizer (~27 ms per ~1 KB measured),
+which at 500–1000 messages dominated candidate planning. New
+`estimateAllMessageTokensFast()` = `Math.round(countMessageCharacters(msg) / 4)`.
+Exact counting stays where correctness requires it (`getCurrentTokenUsage`'s
+rare fallback path).
+
+### 3. Lazy nudge analysis (RC2)
+
+`injectCompressNudges()` computed `estimateContextComposition`,
+`computeProtectedRefs`, and `buildCompressibleRanges` (each an O(all messages)
+pass that stringifies tool outputs) on every transform even when nothing could
+be emitted. Analysis now runs iff:
+
+```
+needsNudgeAnalysis = nudgeAllowed || emergencyOverride || tierTriggerPossible
+```
+
+- `nudgeAllowed` — growth/floor gate says a T1 nudge may fire
+- `emergencyOverride` — opencode compaction happened; the "nothing to
+  compress" check must run to decide between emergency notice vs silence
+- `tierTriggerPossible` — tier-1/tier-2 summary usage crossed the growth
+  threshold, so a T2/T3 trigger could emit
+
+Downstream consumers of the now-possibly-null structures are null-guarded.
+No behavior change when analysis WAS needed; quiet turns simply skip it.
+
+### 4. Structure-version invalidation for derived indexes (RC3)
+
+Two per-transform paths replayed ALL blocks (active + inactive):
+`syncCompressionBlocks()` (sort + liveness recompute + anchor map rebuild) and
+`hideConsumedCompressCalls()` (consumed-call index rebuild). Block liveness and
+the consumed-call structure change ONLY at three mutation sites (found by grep
+audit of `.active =`, `deactivatedByUser*` assignments, `blocksById.set/delete`):
+
+1. `compress/state.ts` → `applyCompressionState()` (new block + consumption deactivations)
+2. `gc/merge.ts` → `mergeMarkedBlocks()` (merged block + source deactivations)
+3. `decompress-logic.ts` → `deactivateCompressionTarget()` (+ deep BFS deactivation)
+
+Design: `PruneMessagesState.structureVersion: number` bumped via
+`bumpPruneStructureVersion()` at exactly those sites.
+
+- **sync**: steady-state fast path skips the full replay when
+  `structureVersion === lastSyncedStructureVersion`; it still updates the
+  anchor-presence bookkeeping cheaply. Any version mismatch (or first run /
+  post-load) triggers the original full replay, which then records the version.
+  Full replay remains the source of truth — the fast path only defers it.
+- **hide-consumed**: its derived index (`allBlockCallIds`,
+  `liveRangeKeysByCallId`, `activeCallIds`) is cached in
+  `PruneMessagesState.hideConsumedIndex = { version, ... }`; rebuilt only when
+  the version changed since the cache was built.
+
+Correctness argument: both consumers derive exclusively from block fields that
+only those three sites mutate, so version equality implies index validity.
+
+### 5. Ordered, coalescing save queue (RC4)
+
+`saveSessionState()` callers are fire-and-forget (`.catch(() => {})`) and fire
+in bursts within one transform (sync deactivation, batch cleanup, nudge
+anchors, compaction reset, tool finalize). Independent async whole-file writes
+could settle out of request order (stale snapshot overwrites fresh) and bursts
+produced N redundant serializations + writes.
+
+Queue semantics (per key = `sessionId \u0000 storageDir`):
+
+1. Snapshot serialized **synchronously at enqueue time** — a queued entry can
+   never read mutated state later.
+2. Synchronous bursts pile onto one batch; batch drains on `setImmediate`
+   (macrotask boundary, so same-tick enqueues coalesce).
+3. A batch writes **only the latest snapshot** (earlier ones are strictly
+   stale); all waiters in the batch share the outcome.
+4. FIFO across batches: entries arriving during a write form the next batch,
+   so order is preserved end-to-end.
+5. Failure isolation: a failed write rejects its batch's waiters (callers
+   already swallow via `.catch`) but subsequent batches proceed normally.
+6. Idle queues delete themselves (no unbounded Map growth).
+
+## Persisted-state compatibility
+
+All new state fields (`structureVersion`, `lastSyncedStructureVersion`,
+`hideConsumedIndex`, `SearchContext.boundaryLookup`) are **transient**: they
+are excluded from `serializePruneMessagesState()`, so on-disk format is
+unchanged. On load, absent fields default safely — `structureVersion` starts
+at 0 which forces one full sync replay and one hide-consumed rebuild, i.e.
+first transform after restart behaves exactly like pre-change code. No
+migration needed; old state files load unchanged.
+
+## What deliberately did NOT change
+
+- Candidate executor validation, protection semantics (Bug 39 hard-exclusion,
+  protected recent window), decompression, fork recovery: untouched.
+- `messageIds` ref allocation/reclamation policy: untouched (refs still grow
+  until compaction — that's a separate concern).
+- Exact-token accounting paths: untouched.

devlog/2026-09-11_perf-transform-history/REQ.md

@@ -0,0 +1,90 @@
+# REQ - Avoid transform work that scales with compression history
+
+- Task ID: `2026-09-11_perf-transform-history`
+- Home Repo: `opencode-acp`
+- Created: 2026-09-11
+- Status: InProgress
+- Priority: P1
+- Owner: ranxianglei
+- References: https://github.com/ranxianglei/opencode-acp/issues/384
+
+## 1. Background & Problem Statement
+
+- **Context**: Long ACP sessions accumulate many compression blocks, byMessageId
+  entries, and message-ref history. Several per-transform (every LLM call) code
+  paths rebuild global lookup structures or replay all historical blocks, so
+  cost grows with *total compression history* instead of staying bounded by
+  visible context.
+- **Current behavior (symptom)**: OpenCode becomes slow in long ACP sessions,
+  especially after many compressions. Isolated probe measured adaptive candidate
+  planning at ~3.2 ms for 100 messages, ~34 ms for 500, ~105 ms for 1,000.
+- **Expected behavior**: Per-transform work bounded by visible message count +
+  active block count; candidate planning ≤ 20 ms at 1,000 messages.
+- **Impact**: Latency on every LLM call in long sessions; user-visible stalls.
+
+## 2. Reproduction (if applicable)
+
+- **Environment**: Node 22/24, linux
+- **Minimal reproduction steps**:
+  1) Run a session with many compressions (large `blocksById` / `byMessageId` /
+     `messageIds.byRef`).
+  2) Observe per-transform time in `logs/acp/context/<session>/<ts>.json`
+     timestamps growing with history size.
+  3) Benchmark harness: `node --import tsx scripts/bench-candidate-planning.ts`
+     (added in this task) reproduces the scaling at N=100/500/1000.
+- **Relevant configuration**: defaults; scales regardless of mode.
+
+## 3. Constraints & Non-Goals
+
+- **Constraints**:
+  - Backward compatibility: persisted state format unchanged (new fields are
+    transient, not serialized — serialization is explicit-field).
+  - Preserve: candidate executor validation, protection semantics, decompression,
+    fork recovery, Bug 34 auto-swap, #247 tool-pair adjustment, [PATCH Bug 3]
+    anchor-survival semantics, [FIX #60]/[FIX Bug 6] save semantics.
+  - Performance requirement: candidate planning at 1,000 messages ≤ 20 ms.
+- **Non-Goals**:
+  - No changes to nudge thresholds/floors or prompt text.
+  - No changes to quality-gate, gc merge policy, or truncation limits.
+  - No new dependencies.
+
+## 4. Acceptance Criteria (must be testable)
+
+- **Correctness**:
+  - [ ] Full existing test suite passes (no behavioral regressions).
+  - [ ] syncCompressionBlocks incremental path produces identical state to full
+    replay for unchanged structure (property/equivalence tests).
+  - [ ] hideConsumedCompressCalls index cache invalidates on every block
+    mutation site (version bump coverage).
+  - [ ] saveSessionState writes are ordered per session; latest snapshot wins;
+    awaited callers observe durability; errors still propagate ([FIX Bug 6]).
+- **Performance / Stability**:
+  - [ ] Candidate planning benchmark at 1,000 messages ≤ 20 ms (median), with
+    before/after evidence recorded in WORKLOG.md.
+  - [ ] Steady-state transform (sync + hide-consumed) no longer iterates
+    inactive blocks when structure is unchanged.
+- **Regression**:
+  - [ ] New/modified test cases added to test suite and passing.
+  - [ ] Dual-agent review of lib/ changes + test review (AGENTS.md §5.3/§5.6).
+
+## 5. Proposed Approach (optional)
+
+- **Affected modules & entry files**:
+  - `lib/compress/types.ts`, `lib/compress/search.ts` — request-scoped
+    boundary lookup (RC1): memoize `buildBoundaryLookup` on `SearchContext`.
+  - `lib/messages/inject/inject.ts` — lazy no-nudge analysis (RC2): skip
+    composition/protected/range computation when no T1/T2/T3 nudge can fire.
+  - `lib/state/types.ts`, `lib/state/utils.ts`, `lib/messages/sync.ts`,
+    `lib/compress/hide-consumed.ts`, `lib/compress/state.ts`,
+    `lib/gc/merge.ts`, `lib/compress/decompress-logic.ts`
+    — verified-state synchronization + transient derived indexes (RC3):
+    transient `structureVersion` counter bumped at block-mutation sites; sync
+    and hide-consumed skip full replay / reuse cached index when unchanged.
+  - `lib/state/persistence.ts` — ordered state-save coalescing (RC4):
+    per-session FIFO queue, snapshot captured at enqueue, coalesced writes.
+- **Risks**:
+  - Version-counter omission at a mutation site → stale cache/skip. Mitigated
+    by grepping all liveness mutators (done) + equivalence tests.
+  - Save-queue timing changes vs tests that mutate XDG dirs mid-flight.
+    Mitigated by capturing file path at enqueue time.
+- **Rollback strategy**: revert the PR commits; no persisted-format change to migrate.

devlog/2026-09-11_perf-transform-history/WORKLOG.md

@@ -0,0 +1,64 @@
+# WORKLOG - Avoid transform work that scales with compression history
+
+- Task ID: `2026-09-11_perf-transform-history`
+- Branch: `2026-09-11_perf-transform-history`
+- Issue: https://github.com/ranxianglei/opencode-acp/issues/384
+- Status: Complete (pending review)
+
+## Root causes verified in code
+
+| # | Claim | Verified at | Fix |
+|---|-------|-------------|-----|
+| RC1 | Candidate validation rebuilds boundary lookup per draft | `lib/compress/search.ts` — `resolveBoundaryIds()` called `buildBoundaryLookup(context, state)` on **every** call; `buildBoundaryLookup` iterates the entire unbounded `messageIds.byRef` map | Request-scoped memo: `SearchContext.boundaryLookup?: BoundaryLookup` populated by `buildSearchContext()`, lazily filled via `??=` in `resolveBoundaryIds()` — built once per compress call regardless of draft count |
+| RC1b | Token estimation used the real Anthropic BPE tokenizer per message in `resolveSelection` | `lib/compress/search.ts` — `countAllMessageTokens(rawMessage)` per selected message; micro-bench: BPE ≈ 27 ms per ~1 KB of text | New `estimateAllMessageTokensFast()` in `lib/token-utils.ts` = `Math.round(chars / 4)` using the existing `countMessageCharacters()` convention (same estimator family used across the codebase for estimates); exact BPE retained only where correctness requires it (`getCurrentTokenUsage` rare fallback) |
+| RC2 | T1 nudge analysis computed full context/range data even when no nudge can fire | `lib/messages/inject/inject.ts` — `estimateContextComposition`, `computeProtectedRefs`, `buildCompressibleRanges` ran unconditionally every transform; each stringifies tool outputs over ALL messages | Gate: `const needsNudgeAnalysis = nudgeAllowed \|\| emergencyOverride \|\| tierTriggerPossible` — heavy analysis runs only when a nudge, tier trigger, or emergency notice could actually be emitted; downstream consumers null-guarded |
+| RC3 | Stable transforms replayed historical inactive blocks and rebuilt immutable consumed-call indexes | `lib/messages/sync.ts` — `syncCompressionBlocks()` sorted + replayed **all** blocks (active and inactive) every transform; `lib/compress/hide-consumed.ts` — rebuilt `allBlockCallIds`/`liveRangeKeysByCallId`/`activeCallIds` from all blocks every transform | Transient derived indexes with structural invalidation: new `PruneMessagesState.structureVersion: number` bumped at every block-mutation site (`compress/state.ts:applyCompressionState`, `gc/merge.ts:mergeMarkedBlocks`, `decompress-logic.ts:deactivateCompressionTarget`) via `bumpPruneStructureVersion()` in `state/utils.ts`; sync keeps `lastSyncedStructureVersion` + anchor bookkeeping and skips full replay when unchanged; hide-consumed caches its index in `PruneMessagesState.hideConsumedIndex` keyed by version |
+| RC4 | Fire-and-forget saves raced and produced redundant whole-file writes | `lib/state/persistence.ts` — every caller did `saveSessionState(...).catch(() => {})` independently; overlapping writes could settle out of order (stale overwrite) and bursts wrote the file N times | Ordered, coalescing per-session queue inside `saveSessionState`: snapshot serialized synchronously at enqueue time (no stale reads), batches drain on `setImmediate`, each batch writes ONLY the latest snapshot, FIFO across batches, waiters of a failed batch reject without poisoning subsequent saves. Key = `sessionId \u0000 storageDir` so custom-storage sessions isolate correctly |
+
+## Key files
+
+- `lib/compress/search.ts` — boundary lookup memoization (RC1), fast token estimate in `resolveSelection` (RC1b)
+- `lib/compress/types.ts` — `SearchContext.boundaryLookup` optional field (backward compatible: hand-built contexts still work via lazy fill)
+- `lib/token-utils.ts` — `estimateAllMessageTokensFast()`
+- `lib/messages/inject/inject.ts` — `needsNudgeAnalysis` gate + null guards (RC2)
+- `lib/messages/sync.ts` — incremental steady-state path (RC3)
+- `lib/compress/hide-consumed.ts` — derived-index cache (RC3)
+- `lib/state/types.ts` — `structureVersion`, `hideConsumedIndex`, `lastSyncedStructureVersion` fields (transient; NOT serialized by `serializePruneMessagesState`, so persisted-state format unchanged)
+- `lib/state/utils.ts` — `bumpPruneStructureVersion()`
+- `lib/compress/state.ts`, `lib/gc/merge.ts`, `lib/compress/decompress-logic.ts` — version bumps at mutation sites
+- `lib/state/persistence.ts` — ordered coalescing save queue (RC4)
+- `scripts/bench-candidate-planning.ts` — new benchmark harness (workload: N visible messages, H ≈ N/2 historical blocks, only newest 20 active, full ref history)
+- Tests: `tests/sync.test.ts` (+4), `tests/hide-consumed.test.ts` (+3), `tests/compress-search.test.ts` (+4), `tests/token-counting.test.ts` (+3), `tests/persistence.test.ts` (+4), `tests/inject.test.ts` (+2, §5.7 multi-turn production-config cycle + emergency-path preservation)
+
+## Benchmark evidence
+
+Same machine, same harness, median of 7 reps. Baseline = pristine master (`5135dfd`) run in a throwaway worktree the day of the fix.
+
+```
+Baseline (master):                          Fixed (this branch):
+msgs  hist  A(ms)     B(ms)  C(ms)   msgs  hist  A(ms)  B(ms) C(ms)
+ 100    50   1608.62   0.06   0.04    100    50   0.40   0.01  0.03
+ 500   250   6798.66   0.40   0.18    500   250   1.09   0.06  0.14
+1000   500  13618.01   0.66   0.10   1000   500   1.16   0.07  0.07
+```
+
+- A = candidate planning (`buildSearchContext` + `resolveRanges` × 10 drafts)
+- B = steady-state `syncCompressionBlocks`; C = `hideConsumedCompressCalls`
+- **A @1000: 13618 ms → 1.16 ms (≈11,700×)** — acceptance target ≤ 20 ms met with ~17× headroom
+- A @500: 6798.7 → 1.09 ms (≈6,200×); A @100: 1608.6 → 0.40 ms (≈4,000×)
+- B+C steady-state flat vs history size (0.76 → 0.14 ms @1000); both now bounded by ACTIVE block count, not total history
+
+Note: absolute baseline numbers differ from the isolated probe in the issue (3.2/34/105 ms) because this harness models a denser workload (realistic tool-output sizes, 10 drafts per call, full un-reclaimed ref history); before/after are measured on the identical harness.
+
+## Test results
+
+- Full suite: **1151 tests, 0 failures** (`node --import tsx --test tests/*.test.ts`), up from 1131 pre-change
+- `npm run typecheck`: clean
+- New tests specifically guard the regression modes: sync full-replay-vs-incremental equivalence + shared-anchor ordering, hide-consumed cache reuse/invalidation identity, boundary-lookup memoization + lazy backward-compat, fast-estimator exactness, save-queue burst coalescing / no-stale-overwrite / failure isolation / storageDir isolation, §5.7 multi-turn growth cycle with `preserveRecentMessages > 0` asserting both `shouldInjectThisTurn` AND `lastPerMessageNudgeTokens`/`lastNudgeShownTokens` after each turn
+
+## Lessons learned
+
+- The dominant cost was not the "obvious" O(blocks) loops but per-draft rebuilds of O(ref-history) structures plus a real tokenizer call inside a hot loop — profile before optimizing.
+- Derived indexes must be invalidated at EVERY mutation site; the three block-mutation sites (compress apply, batch merge, decompress deactivation) were found by grepping `.active =` / `deactivatedByUser*` assignments and `blocksById.set/delete`.
+- Save-ordering bugs are invisible to single-save tests; behavioral coverage must burst saves synchronously and assert which snapshot won.
+- Transient state fields must stay OUT of `serializePruneMessagesState` to keep persisted-state compatibility; load-time reconstruction defaults them safely (version 0 forces one full replay after restart — correct by construction).

lib/compress/decompress-logic.ts

@@ -1,5 +1,6 @@
 import type { CompressionBlock, PruneMessagesState, WithParts } from "../state"
 import { parseBlockRef } from "../message-ids"
+import { bumpPruneStructureVersion } from "../state/utils"
 import type { CompressionTarget } from "../commands/compression-targets"
 
 export function parseBlockIdArg(arg: string): number | null {
@@ -145,6 +146,12 @@ export function deactivateCompressionTarget(
             }
         }
     }
+
+    // [Issue #384] Block liveness changed — invalidate sync + hide-consumed
+    // caches derived from previous versions.
+    if (target.blocks.length > 0) {
+        bumpPruneStructureVersion(messagesState)
+    }
 }
 
 export interface RestoredMessagesResult {