perf(runtime): cap long-lived runtime caches to stop daemon memory drift#201
Merged
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 849c9a2902
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
LIU9293
force-pushed
the
feat/bounded-runtime-caches-7697
branch
from
April 21, 2026 04:03
849c9a2 to
f3c3097
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f3c309766b
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
The daemon is a long-lived singleton, so several in-memory caches that never shrink leak steadily for the life of the process: - liveEventHistory entries retain entire raw SDK events (tool outputs, file contents, subagent replies can be 2-50KB each) during a turn. - finalizedMessages / rateLimitedMessages / rateLimitErrors in the message-update layer record one entry per finalized message, forever. - newSessions in each CLI agent client (claude/gemini/goose/kilo/kiro/ qwen) records abandoned session ids that are never consumed. This change introduces two fixes: 1) Deep-truncate raw event payloads before pushing them into the live event history (4KB per string, default). The live-status UI only renders tool input fields (30-200 char previews) and never the raw tool output, so the truncation is invisible to the renderer. Exception: assistant `text` parts inside `message.part.updated` events (and their `reasoning` / `thinking` siblings) are kept verbatim because they feed `state.currentText`, which `request-run.ts` publishes to Slack as the final reply on `stop` and tool-only turns — truncating them would post a `...[truncated N bytes]` marker to the user. The truncator now accepts a `preserveStringAtPath` predicate and the request runner uses it to whitelist `properties.part.text` for those part types. A stability test replays six captured harness fixtures and asserts the rendered status bytes are identical with and without truncation, and a new regression test proves the full truncate → fold → publish pipeline returns the assistant reply verbatim even for 50KB final messages. 2) Ship a generic BoundedSet / BoundedMap in @/utils with Set/Map compatible APIs (FIFO eviction on overflow, no order refresh on re-set — matching the bespoke copy that used to live in message-updates.ts). Apply them to finalizedMessages (5000), rateLimitedMessages (5000), rateLimitErrors (5000), and each agent's newSessions (1000). Eviction is safe in every case: stale message ids receive no further updates, and an evicted session id simply loses its 'this is a new session' flag. Note on sessionProviders: the agent adapter's session → provider lookup is intentionally NOT capped. It is used by abortSession, subscribeToSession, ensureSession, and the task / cron schedulers to dispatch to the correct agent client, so evicting entries would misroute calls for still-valid non-opencode sessions once the cap is exceeded. Its growth is bounded in practice by the number of unique session ids the daemon has observed. Tests: - packages/utils/test/event-truncation.test.ts (11 cases) - packages/utils/test/final-reply-preservation.test.ts (4 cases) - packages/utils/test/bounded-collections.test.ts (12 cases) - packages/live-status-harness/test/truncation-stability.test.ts (6 fixtures x render-stability assertions) - Full suite: 320 pass, 1 skip, 0 fail.
LIU9293
force-pushed
the
feat/bounded-runtime-caches-7697
branch
from
April 21, 2026 04:13
f3c3097 to
07e9267
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The daemon runs forever, so a handful of in-memory caches that never shrink were leaking steadily across its lifetime. This PR caps the ones that are safe to cap and also trims the largest single contributor (raw SDK event payloads) before they land in the live-event history.
What changed
1) Truncate raw event payloads before storing them in
liveEventHistorypackages/utils/event-truncation.ts: generic recursive truncator (4KB per string default).packages/core/kernel/request-run.ts: calltruncateEventPayload()before push.2) Shared
BoundedSet/BoundedMappackages/utils/bounded-collections.ts: Set/Map-compatible FIFO containers.message-updates.ts).packages/core/runtime/message-updates.tsto reuse them — removes duplicated classes.3) Apply
BoundedSet/BoundedMapto the caches where eviction is safefinalizedMessagescore/runtime/message-updates.tsrateLimitedMessagesrateLimitErrorsnewSessions(per agent)agents/{claude,gemini,goose,kilo,kiro,qwen}/client.tsagents/runtime/cli-session.ts: widen thenewSessions?param to a structural{ add }contract so agent clients can pass either a rawSetor aBoundedSetwithout forcing a cross-package dep here.Explicitly NOT capped:
sessionProvidersThe agent adapter's session → provider map is used by
abortSession,subscribeToSession,ensureSession, and the task / cron schedulers to dispatch calls to the correct agent client. Evicting entries and falling back to the default provider would misroute calls for still-valid non-opencode sessions once the cap is exceeded (flagged by Codex review on the first revision of this PR). It stays an unbounded correctness index; its growth is bounded in practice by the number of unique session ids the daemon has observed.Expected memory impact
newSessionssets; at worst each now stays under its cap forever instead of growing with every new message / session id.Tests
packages/utils/test/event-truncation.test.ts— 9 unit cases.packages/utils/test/bounded-collections.test.ts— 12 unit cases (FIFO order, no-refresh-on-re-set, delete/clear, Set/Map-compatibleadd/setreturn value).packages/live-status-harness/test/truncation-stability.test.ts— replays 6 captured harness fixtures and asserts rendered status is byte-identical with and without truncation.bun test: 314 pass, 1 skip, 0 fail.tsc --noEmit: no new errors (pre-existingweb-uiSvelte module-resolution noise is unrelated).