Skip to content

fix: stabilize cache-aware context handling#410

Closed
quan020406 wants to merge 1 commit into
TencentCloud:mainfrom
quan020406:fix/prompt-cache-show-injected
Closed

fix: stabilize cache-aware context handling#410
quan020406 wants to merge 1 commit into
TencentCloud:mainfrom
quan020406:fix/prompt-cache-show-injected

Conversation

@quan020406

Copy link
Copy Markdown

Description | 描述

本 PR 针对 #120 中的 prompt cache 不稳定问题,调整插件上下文构建方式,减少每轮动态数据对可复用 prompt 前缀的影响。

核心目标:

  • 稳定会话事实通过确定性 session snapshot 输出。
  • 动态记忆默认按需工具召回,而不是每轮自动 prepend。
  • 大工具结果在进入 prompt-facing history 前先卸载,避免长期污染历史。
  • Mermaid/MMD 任务状态使用 snapshot/delta 结构,降低旧消息反复变化的风险。
  • 为后续 Cache Epoch 长上下文治理补齐基础合约和测试。

主要改动:

  • 新增 src/core/session/session-snapshot.ts,用于构建确定性 session snapshot。
  • 默认启用 recall.mode = "tool-only",同时保留 recall.mode = "auto" 兼容旧的自动召回路径。
  • 拆分 showInjectedpersistInjected 语义;当 persistInjected=false 时,在持久化消息前清理 <relevant-memories> 注入块。
  • 新增工具结果 normalize/offload 逻辑,大结果写入 refs/*.md,历史中只保留 compact metadata。
  • 新增 tdai_offload_read,支持按 result_ref、query 和行号范围受控读取完整结果片段。
  • 新增 Cache Epoch helper,用于判断长上下文压缩边界。
  • 新增 Mermaid/MMD task snapshot 和 delta helper。
  • 补充相关回归测试,覆盖 session snapshot、tool-only recall、注入清理、工具结果卸载、ref 读取、epoch 判断和 task snapshot/delta。

Related Issue | 关联 Issue

Fix #120

Change Type | 修改类型

  • Bug fix | Bug 修复
  • New feature | 新功能
  • Documentation update | 文档更新
  • Code optimization | 代码优化

Self-test Checklist | 自测清单

  • Verified locally | 本地验证通过
  • No existing features affected | 无影响现有功能

本地验证命令:

npm.cmd test
npm.cmd run build:plugin
node --check dist\index.mjs
git diff --check

验证结果:

  • 13 个测试文件通过。
  • 82 个测试通过。
  • 插件构建通过。
  • dist/index.mjs 语法检查通过。
  • diff whitespace 检查通过。

额外做过本地长上下文验证:

  • 48 turn。
  • 覆盖普通多轮对话、记忆召回、工具密集任务、长文件/搜索任务、Mermaid/MMD 任务追踪、issue 方案沉淀等 6 类任务。
  • 生成 384 条 provider rows 和 8 条 gate rows。
  • 本地测得输入 token 降低约 83.9% - 86.5%
  • 本地 LCP 约 97.8%
  • result_ref 恢复链路检查为 333/333

上述长上下文数据仅作为本地验证证据,不作为公开 benchmark 结论。Toolathlon、SWE-bench / Claw-SWE-Bench、WideSearch 等应作为后续独立 benchmark gate。

Additional Notes | 其他说明

兼容性说明:

  • 旧的自动召回路径仍可通过 recall.mode = "auto" 启用。
  • showInjected 仍控制当前轮是否展示注入召回内容。
  • persistInjected=false 现在会阻止注入召回块进入持久化对话历史。
  • tdai_offload_read 只允许读取安全的 refs/*.md 相对路径,并应用读取预算限制。

@Maxwell-Code07

Copy link
Copy Markdown
Collaborator

Thank you for submitting this PR and participating in Tencent Rhino-bird Open-source Training Program!
We have successfully received your submission. The program is currently in full swing, and we will complete the Code Review for you as soon as possible. Please keep an eye on the status notifications for this PR so you can follow up promptly once the review feedback is provided.
Thanks again for your contribution and open-source spirit! 🚀

@YOMXXX

YOMXXX commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Triage note: #375 is now the canonical scoped runtime mitigation for #120.

This PR is much broader than the immediate #120 fix: it changes recall mode defaults, adds session snapshots, tool-result offload/read tooling, cache epoch helpers, and task snapshot/delta helpers. Those may be useful independently, but they should not be merged as a single duplicate #120 implementation.

Recommendation: split this into separate PRs after #375: one for any remaining recall-cache behavior, one for tool-result offload/read, and one for cache-epoch/task snapshot helpers. The current PR is too large for the corrected #120 scope.

@quan020406

Copy link
Copy Markdown
Author

Thanks for the triage. I have split #410 into three scoped PRs based on the requested boundaries:

The recall PR excludes behavior already covered by #375. The other follow-up PRs are scoped independently for narrower review.

Local docs, logs, experiment artifacts, machine-specific config, and credentials are not included in the submitted diffs.

@YOMXXX

YOMXXX commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Follow-up cleanup note now that the split PRs exist:

This large PR should be considered superseded for review purposes. The immediate #120 runtime mitigation is already covered by the approved #375, and the remaining parts have been split into narrower follow-ups:

Recommendation: close this PR and continue review on the split PRs only. Keeping #410 open alongside the split follow-ups makes the #120 queue look duplicated and increases the chance that the same behavior is reviewed twice.

@quan020406 quan020406 closed this Jul 10, 2026
@quan020406 quan020406 reopened this Jul 10, 2026
@quan020406

Copy link
Copy Markdown
Author

Acknowledged, and thanks for the clarification.

#410 will be treated as superseded going forward. The immediate runtime mitigation directly related to #120 is tracked in #375, and the remaining parts from #410 have been split into #446, #447, and #448. Further discussion and review will continue on the split PRs.

@quan020406 quan020406 closed this Jul 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

prependContext + showInjected 导致 OpenAI-compatible provider 前缀缓存命中率退化 / Prompt cache hit rate regression

3 participants