feat: memory compaction — progressive summarization for stored memories#343
Merged
AliceLJY merged 1 commit intoCortexReach:masterfrom Mar 25, 2026
Merged
Conversation
Adds a `MemoryCompactor` that periodically consolidates semantically
similar old memories into single refined entries, inspired by the
progressive summarization pattern (MemOS). Over time, related memory
fragments are merged rather than accumulated, reducing retrieval noise
and keeping the LanceDB index lean.
Key additions:
- `src/memory-compactor.ts`: pure, dependency-free compaction module
with cosine-similarity clustering, greedy seed expansion, and
rule-based merge (dedup lines, max importance, plurality category)
- `store.ts`: new `fetchForCompaction()` method that fetches old entries
with vectors (intentionally omitted from `list()` for performance)
- `index.ts`: `memory_compact` management tool (requires
`enableManagementTools: true`) + optional auto-compaction at
`gateway_start` with configurable cooldown
- `openclaw.plugin.json`: `memoryCompaction` config schema + uiHints
- `test/memory-compactor.test.mjs`: 23 tests, 100% pass
Config example:
memoryCompaction:
enabled: true # auto-run at gateway_start
minAgeDays: 7 # only touch memories ≥ 7 days old
similarityThreshold: 0.88
cooldownHours: 24
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
e730804 to
5a5b0d2
Compare
AliceLJY
added a commit
to AliceLJY/memory-lancedb-pro
that referenced
this pull request
Mar 25, 2026
Combines the following rebased changes onto latest cortexreach/master: - feat: add session compression and adaptive extraction throttling - fix: disable extraction throttle and session compression in smart-extractor-branches test - fix: address Codex review for session compression - fix: address PR CortexReach#318 review — align opt-in defaults, CJK-aware scoring, rate limiter docs Session compression scores conversation texts before auto-capture and drops low-signal content; extraction throttling adds a sliding-window rate limiter and a low-value conversation skip gate. Both features default to opt-in (disabled) to avoid surprising existing users. Resolves merge conflicts with memory-compaction (CortexReach#343) and recall-mode-v2 (CortexReach#342) that landed on master since the original branch.
|
Mar 26 18:03:18 OpenClaw node[1962]: 2026-03-26T18:03:18.841+00:00 [plugins] plugin tool name conflict (memory-lancedb-pro): memory_compact |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
src/memory-compactor.ts(new): Pure, dependency-free compaction module implementing the progressive summarization pattern. Clusters semantically similar memories using greedy cosine-similarity expansion (seeded by highest-importance entry), then merges each cluster into a single refined entry — deduplicating lines, taking max importance, plurality-voting on category.store.ts: AddsfetchForCompaction(maxTimestamp, scopeFilter, limit)— fetches old entries with raw vectors (intentionally omitted fromlist()for performance; compaction needs them for similarity computation).index.ts: Wires inmemory_compactMCP tool (gated byenableManagementTools) withdry_run/min_age_days/similarity_threshold/scopesparameters. Also registers an optionalgateway_startauto-compaction hook with configurable cooldown.openclaw.plugin.json:memoryCompactionconfig schema + uiHints.test/memory-compactor.test.mjs: 23 tests coveringcosineSimilarity,buildClusters,buildMergedEntry, andrunCompaction(including dry-run, empty store, orthogonal entries, scan limit).Motivation
Without compaction, the LanceDB index accumulates fragments of the same idea across many sessions. Related memories like "prefers dark mode", "always uses dark theme", "dark mode on all apps" remain as three separate entries — each returned at retrieval time, wasting context budget and diluting precision. Progressive summarization collapses these into one high-quality entry, so the index gets more refined over time rather than noisier.
Config example
Manual use (requires
enableManagementTools: true)Test plan
node --test test/memory-compactor.test.mjs— 23/23 passmemoryCompaction.enabled: truein a live OpenClaw instance; confirmgateway_startlog shows compaction statsmemory_compactwithdry_run: true, verify cluster count makes sense for your LanceDBmemoryCompactionis absent from config🤖 Generated with Claude Code