Skip to content

(fix) bound per-session kernel-state and nudge-dedup caches with LRU eviction - #121

Open
ranxianglei wants to merge 1 commit into
Tyan66666:mainfrom
ranxianglei:2026-09-06_state-lru-eviction
Open

(fix) bound per-session kernel-state and nudge-dedup caches with LRU eviction#121
ranxianglei wants to merge 1 commit into
Tyan66666:mainfrom
ranxianglei:2026-09-06_state-lru-eviction

Conversation

@ranxianglei

Copy link
Copy Markdown
Contributor

Fixes #113.

Problem
Two per-session caches grow without bound for the lifetime of the engine process:

  • AcpStateStore.states (src/state.ts) — one full acp-kernel CompressionState per session id, including every block summary; delete() had zero callers.
  • lastNudgeTurn (src/index.ts) — one entry per session id that ever received a nudge.

A long-lived server handling many sessions leaks memory proportional to total historical session count × per-session state size. (compressCallIdsToHide is intentionally left alone: it is bounded by in-flight compress calls and self-cleans on tool/result arrival.)

Cause
Plain Maps with only insert paths; nothing evicts idle sessions.

Fix

  • New src/lru.ts: LruMap<K,V> (extends Map, API-compatible drop-in) + DEFAULT_SESSION_CACHE_LIMIT = 512. Recency refreshes on both get and set; over-cap inserts evict the coldest entry.
  • AcpStateStore.states is now an LRU map capped at 512 sessions (constructor takes an optional limit for tests).
  • lastNudgeTurn is now an LRU map with the same cap. Worst case after eviction of a cold session: one extra non-emergency nudge when it returns (per-turn dedup restarts) — bounded and self-healing.
  • Why eviction is lossless for block identity: stateFor already rehydrates from the durable log (the same path a restart uses), and rehydration is deterministic — bN ids are recorded in the durable event or synthesised in ledger order, and message refs are re-derived from the append-only full-log array (allLogMessages includes shadowed originals), so numbering reproduces exactly.
  • Secondary fix found while making eviction routine: rehydration set blocks/nextBlockId but not nextRunId, so the first compression after a restart (or eviction) reused run id r1 colliding with a rehydrated block. Rehydration now continues run ids after the rehydrated max (nextRunIdAfter).
  • Kernel fields that do reset on eviction — tokenSnapshot, nudge cadence state, stats counters — all self-heal on the session's next turn; documented on the field.

Verification

  • New tests:
    • tests/lru.test.ts (4): cap enforcement, recency refresh on get AND set, delete semantics, cap clamping. The re-set test caught a real bug in the first implementation (Map.set on an existing key does not move it to MRU position — fixed by delete+re-insert).
    • tests/state.test.ts "M2: store cap evicts cold sessions losslessly": cap-1 store, compress two distinct sessions, touch warm (evicts cold), then assert cold's rehydrated state has identical bN id, identical effectiveMessageIds, and continued nextBlockId/nextRunId — indistinguishable from the live state.
  • npm run typecheck clean; npm test 213 pass / 0 fail (208 baseline + 5 new); npm run build clean (dist committed per AGENTS.md §4).

Built against main @ 6348af8 (v0.2.20).

@ranxianglei

Copy link
Copy Markdown
Contributor Author

我来协助看一下这个问题,请分析并回复处理结果。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants