Skip to content

feat: memory compaction — progressive summarization for stored memories#343

Merged
AliceLJY merged 1 commit intoCortexReach:masterfrom
AliceLJY:feat/memory-compaction
Mar 25, 2026
Merged

feat: memory compaction — progressive summarization for stored memories#343
AliceLJY merged 1 commit intoCortexReach:masterfrom
AliceLJY:feat/memory-compaction

Conversation

@AliceLJY
Copy link
Collaborator

Summary

  • src/memory-compactor.ts (new): Pure, dependency-free compaction module implementing the progressive summarization pattern. Clusters semantically similar memories using greedy cosine-similarity expansion (seeded by highest-importance entry), then merges each cluster into a single refined entry — deduplicating lines, taking max importance, plurality-voting on category.
  • store.ts: Adds fetchForCompaction(maxTimestamp, scopeFilter, limit) — fetches old entries with raw vectors (intentionally omitted from list() for performance; compaction needs them for similarity computation).
  • index.ts: Wires in memory_compact MCP tool (gated by enableManagementTools) with dry_run / min_age_days / similarity_threshold / scopes parameters. Also registers an optional gateway_start auto-compaction hook with configurable cooldown.
  • openclaw.plugin.json: memoryCompaction config schema + uiHints.
  • test/memory-compactor.test.mjs: 23 tests covering cosineSimilarity, buildClusters, buildMergedEntry, and runCompaction (including dry-run, empty store, orthogonal entries, scan limit).

Motivation

Without compaction, the LanceDB index accumulates fragments of the same idea across many sessions. Related memories like "prefers dark mode", "always uses dark theme", "dark mode on all apps" remain as three separate entries — each returned at retrieval time, wasting context budget and diluting precision. Progressive summarization collapses these into one high-quality entry, so the index gets more refined over time rather than noisier.

Config example

"memoryCompaction": {
  "enabled": true,
  "minAgeDays": 7,
  "similarityThreshold": 0.88,
  "cooldownHours": 24
}

Manual use (requires enableManagementTools: true)

// Preview without writing
memory_compact { dry_run: true }

// Run for specific scopes
memory_compact { scopes: ["project:myapp"], similarity_threshold: 0.92 }

Test plan

  • node --test test/memory-compactor.test.mjs — 23/23 pass
  • Manual: enable memoryCompaction.enabled: true in a live OpenClaw instance; confirm gateway_start log shows compaction stats
  • Manual: call memory_compact with dry_run: true, verify cluster count makes sense for your LanceDB
  • Verify existing auto-capture / auto-recall behavior is unchanged when memoryCompaction is absent from config

🤖 Generated with Claude Code

Adds a `MemoryCompactor` that periodically consolidates semantically
similar old memories into single refined entries, inspired by the
progressive summarization pattern (MemOS). Over time, related memory
fragments are merged rather than accumulated, reducing retrieval noise
and keeping the LanceDB index lean.

Key additions:
- `src/memory-compactor.ts`: pure, dependency-free compaction module
  with cosine-similarity clustering, greedy seed expansion, and
  rule-based merge (dedup lines, max importance, plurality category)
- `store.ts`: new `fetchForCompaction()` method that fetches old entries
  with vectors (intentionally omitted from `list()` for performance)
- `index.ts`: `memory_compact` management tool (requires
  `enableManagementTools: true`) + optional auto-compaction at
  `gateway_start` with configurable cooldown
- `openclaw.plugin.json`: `memoryCompaction` config schema + uiHints
- `test/memory-compactor.test.mjs`: 23 tests, 100% pass

Config example:
  memoryCompaction:
    enabled: true        # auto-run at gateway_start
    minAgeDays: 7        # only touch memories ≥ 7 days old
    similarityThreshold: 0.88
    cooldownHours: 24

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@AliceLJY AliceLJY force-pushed the feat/memory-compaction branch from e730804 to 5a5b0d2 Compare March 25, 2026 15:49
@AliceLJY AliceLJY merged commit 4097c4a into CortexReach:master Mar 25, 2026
3 checks passed
AliceLJY added a commit to AliceLJY/memory-lancedb-pro that referenced this pull request Mar 25, 2026
Combines the following rebased changes onto latest cortexreach/master:
- feat: add session compression and adaptive extraction throttling
- fix: disable extraction throttle and session compression in smart-extractor-branches test
- fix: address Codex review for session compression
- fix: address PR CortexReach#318 review — align opt-in defaults, CJK-aware scoring, rate limiter docs

Session compression scores conversation texts before auto-capture and
drops low-signal content; extraction throttling adds a sliding-window
rate limiter and a low-value conversation skip gate.

Both features default to opt-in (disabled) to avoid surprising existing
users. Resolves merge conflicts with memory-compaction (CortexReach#343) and
recall-mode-v2 (CortexReach#342) that landed on master since the original branch.
@NW15D
Copy link

NW15D commented Mar 26, 2026

Mar 26 18:03:18 OpenClaw node[1962]: 2026-03-26T18:03:18.841+00:00 [plugins] plugin tool name conflict (memory-lancedb-pro): memory_compact
it's ok ? it conflict with core_tools openclaw.
i use github version of plugin

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants