Skip to content

feat: /acp compact — dedicated compression model (session shared-prefix + models.json) - #241

Open
ranxianglei wants to merge 2 commits into
masterfrom
feat/acp-compact-compression-model
Open

ranxianglei wants to merge 2 commits into
masterfrom
feat/acp-compact-compression-model

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Summary

Implements #240: a /acp compact command that offloads compress summary-writing to a separate call, in two modes:

  • session — the session's own model, in a separate call that reuses the session prompt prefix (system prompt + active tools + the exact transformed messages Pi just sent). Because the prefix matches what the main model already sent, the provider prompt cache keeps the input cheap, and the compression reasoning stays out of the main model's context.
  • <id> — a cheaper models.json model (reuses its baseUrl/apiKey; a different cache namespace, so no prefix sharing, but a lower per-token price).

Unset → the main model writes summaries (unchanged default). Any call failure (network/timeout/API/empty) → fallback to the main model's summary for that range — the session is never interrupted.

Changes

  • src/compress-model.ts: CompressionModelClient (listModels / resolveModel / summarize / summarizeContext), SESSION_MODEL_REF.
  • src/compress-tool.ts: branch on session vs models.json; build the shared-prefix Context; per-range instruction appended to the captured prefix.
  • src/index.ts: capture the ACP-transformed messages in the context transform; pass pi to the tool.
  • src/runtime.ts: per-session setLastSentMessages / getLastSentMessages.
  • src/commands.ts: /acp compact [session|<id>|reset] + status display.
  • src/user-config.ts / src/config.ts: compressionModelId (+ saveCompressionModelId writer).
  • Tests: 22 new (compress-model + acp-compact, incl. 4 session tests). 450/450 pass.
  • Docs: README + CONFIGURATION.

Related

  • Follow-up (the primary lever — Prompt Cache stability / batch rollover) is being tracked as a separate issue.

ework-agent added 2 commits August 28, 2026 01:04
Add an optional dedicated model that writes compress summaries instead
of the main model, reusing credentials from ~/.pi/agent/models.json.

- /acp compact [list | <id> | reset] command
- compressionModelId persisted to ~/.pi/acp.json
- CompressionModelClient (list/resolve/summarize via ModelRuntime)
- handleCompress routes to the model with guaranteed fallback to the
  main model on any failure (never blocks the session)
- 18 new tests (models.json read, mock SSE API, fallback)
- README.md + CONFIGURATION.md docs
…n model

Add a 'session' mode to the dedicated compression model: instead of only
supporting a cheaper models.json model, /acp compact session now uses the
session's own model in a separate call that reuses the session prompt prefix
(system prompt + active tools + the exact transformed messages Pi just sent),
so the provider prompt cache keeps the input cheap and the compression
reasoning stays out of the main model's context.

- runtime: per-session capture of the transformed messages (setLastSentMessages)
- index: capture the rebuilt messages in the context transform; pass pi to the tool
- compress-model: SESSION_MODEL_REF + summarizeContext (takes a built Context)
- compress-tool: branch on session vs models.json; build the shared-prefix
  Context; per-range instruction appended to the captured prefix
- commands: /acp compact session + status display
- tests: 4 new session tests (set/status/shared-prefix success/fallback)
- docs: README + CONFIGURATION for the session mode
@github-actions

Copy link
Copy Markdown

📦 Built Extension Artifact

Branch: feat/acp-compact-compression-model (c5e7de8)

Option A — Install from npm PR tag (recommended)

pi install npm:billion-context-pi@pr-241

Each push to this PR publishes a new version under the pr-241 npm tag.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pi-pr241.tgz
pi install ./package

This comment is automatically updated on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 ework agent · vllm-qwen/qwen3.8-27b

[bot] Reviewing this PR now. Branch feat/acp-compact-compression-model checked out locally (2 commits: 631d51e + c5e7de8). Running typecheck, tests, and build, and reading through the diff — I'll report findings here shortly.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 ework agent · vllm-qwen/qwen3.8-27b

[bot] Review complete — LGTM, all CI checks green. Merging is left to you (human-only per AGENTS.md).

CI (run on branch feat/acp-compact-compression-model, commits 631d51e + c5e7de8)

Check Result
npm run typecheck ✅ pass
npm test 450/450 pass (22 new: 12 in tests/compress-model.test.ts, 10 in tests/acp-compact.test.ts)
npm run build dist/index.js 535.59 KB, self-contained — zero runtime refs to pi-ai (type-only imports erased)

Verified integration points

  • Pins intact: acp-kernel 0.0.46, pi-coding-agent 0.83.0 unchanged; new devDep @earendil-works/pi-ai pinned exactly to 0.83.0 (type-only, so devDep placement is correct). The 1.3K-line lockfile diff is just pi-ai's transitive deps (AWS/Anthropic SDKs) — expected.
  • Shared-prefix capture (src/index.ts:367): runtime.setLastSentMessages(sid, rebuilt) stores exactly the array returned as { messages: rebuilt } — the final ACP-transformed messages (tags + injected nudge). Correct source.
  • Prefix fidelity: getSystemPromptText(ctx) uses ctx.getSystemPrompt() (same prompt Pi sends), convertToLlm(AgentMessage[]): Message[] exists in pi-coding-agent 0.83.0, and collectActiveTools's {name, description, parameters} mapping matches pi-ai's Tool/Context shapes exactly. So the session-mode request should be byte-identical to the main model's prefix → cache hit.
  • Fallbacks: per-range try/catch → main-model summary retained on any failure; unresolvable ref, missing session model, and empty response all degrade gracefully with a ⚠️ note in the panel. Session never blocked.
  • Tests: the 4 session tests use a real mock SSE server and assert the shared prefix ("lorem ipsum") is actually present in the outbound request body, plus the unreachable-model fallback path. Good coverage of the load-bearing behavior.
  • Docs: README + CONFIGURATION cover both modes, the fallback guarantee, resolution rules, and the global-file scope.

Non-blocking observations

  1. Fallback quality cliff: when a compression model is set, the tool description tells the main model to pass a minimal placeholder ("compressed"). If the compression call fails for that range, the block's stored summary is that placeholder — much weaker than the old "main model writes full summaries" behavior, and the only recovery is decompress + re-compress. The session is never interrupted (as designed), but consider either (a) stating this tradeoff explicitly in the tool description/docs, or (b) asking the main model for a 1–2 sentence real summary instead of a bare placeholder so the fallback is still useful.
  2. Project-level override edge case: /acp compact writes the global ~/.pi/acp.json, but loadUserConfig lets a project .pi/acp.json override per-field. If a project file sets compressionModelId, the next reloadConfig (forced by lastUserConfigKey = undefined) re-reads and the project value wins over the command's in-memory update. CONFIGURATION documents the scope, but the command's "applies across sessions" message could mislead in that corner.
  3. Memory: lastSentMessages holds a full copy of the transformed messages per session until clearSessionTracking — fine, just noting it's a duplicate of data already in memory.
  4. Minor: commandArgString(args: string) is typed string but handles arrays (test convenience) — the type is a slight lie; harmless.
  5. truncateContent(content, 120000) caps range content only in models.json mode; session mode sends the full (cached) context with no cap. Inconsistent but each is reasonable for its mode.

None of these block the merge. Nice work — the session-mode design (isolation without a cheaper model, via cache) is the interesting part.

@ranxianglei

Copy link
Copy Markdown
Owner Author

解决冲突 更新最新代码

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

上游同步完成,冲突已解决,已更新到最新代码。

现状feat/rollover-cache-stability 已合并 origin/master @ 2650a99(v0.1.70 时期,acp-kernel 0.0.69),merge commit 16c4c3a,已推送。PR #243 现在基于最新 master。

冲突(12 个文件,均保留双方特性后解决)

  • CHANGELOG.md:保留 master 新条目 + 本功能条目
  • src/config.ts:AdapterConfig 字段合并(rollover + promptSections/nudgeSections/toolPrompts/delegatePrompt)
  • src/compress-tool.ts:imports 合并;rollover 分支改用 compress 摘要双重转义(\uXXXX)入库+渲染;幻觉用户原话被摘要永久化为 CURRENT TASK 导致循环复发 #309 之后的 sanitizedRanges(延迟记录与摘要净化两者都保留);panel 格式跟随 master 的 block-span 新格式
  • src/index.ts(5 处):imports 合并;restore-merge 块置于 reasoning-drop/degeneration-collapse 之前;wireSystemPrompt 在 pack-aware base 之后追加 ROLLOVER_PROMPT_SECTION
  • src/runtime.ts / src/state.ts / src/status-tool.ts:接口 / 持久化 / 状态面板合并
  • 测试文件:tests/density-usage-fixes.test.ts 随 master 删除 density 一并 git rm;其余保留双方断言

移植工作:被 master 移除的 calibrateTokens/density API → 改用 #289sentViewTokenCount/adjustedTokenCount;master 新增测试文件的 57 处 createAcpExtension 调用点补了 rollover: false pin(legacy 行为测试不受 rollover 默认开启影响)。

调试发现:验证时发现 master 新增的 host gate(isUnsupportedHost)要求 sessionManager 暴露 buildContextEntries()——我的 rollover 测试 fake 缺该方法,导致 session_start 提前返回、跳过 absorb 注册。这是本地偶发失败的根因(已按 master 的 host contract 修正),与合并本身无关。

验证:typecheck 干净;全量测试绿 ×3(823 tests / 817 pass / 0 fail / 6 skip);构建成功(dist 554 KB 自包含)。注意:本机磁盘 97% 且外部进程会间歇性删除 /tmp 下的文件(观察到 TAP 文件与测试状态目录在命令之间被清掉),本地全量跑偶发的瞬时失败与此无关——以 CI 为准。

未改版本号(release-branch-only 规则);PR #243 等你 review 和合并。

一句话总结:把 PR #243 同步到最新 master(v0.1.70 / acp-kernel 0.0.69),解决了 12 个文件的冲突并保留双方特性,本地全量测试通过,待人工合并。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant