Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
## Unreleased (master, since v0.1.38)
- **fix(degeneration): thinking/text 单字符退化熔断 + 一次性恢复通知(closes #351)** — 长会话末尾模型偶发退化为单字符长连击(实测:thinking 块末尾 4655 个连续「【」,跨轮升级直至 turn abort、会话停死)。根因链已代码级验证:pi 的 openai-completions 转换把历史 assistant thinking 在**每个后续请求**中回传 provider(`reasoning_content`,或 requiresThinkingAsText 时转纯文本),aborted turn 的部分消息又持久化在会话日志里 → 退化尾部随每轮 prompt 重放 → 模型看到自己上一轮以数千个重复字符结尾 → 续写偏置再次触发退化 → 连环 abort。新增 `src/degeneration.ts`:每个 context 事件对出站视图的 assistant text/thinking 块折叠 ≥`minRun`(默认 200,下限 8,codepoint/代理对安全)的单 codepoint 连击为短标记(保留 ≤3 份样本;纯函数、幂等——标记固定文案无相邻重复码点、fail-safe;持久化历史不改,toolCall 参数不动以免与实际执行脱钩);当最近一条 assistant 消息已退化时追加一次性 `[ACP recovery notice]`(位置自限:模型产出新 turn 后自动消失,无持久状态不累积,#223 教训)。检测走持久化 originals 而非出站视图:thinking-only aborted turn 会被 projectMessage 丢弃(空文本在 OpenAI 兼容 provider 400),但它仍是模型的"上一轮",通知必须照发。acp.json 新键 `degenerationGuard`(boolean 或 `{enabled,minRun}`,默认开;`false` 为 kill-switch)。附带修复:`repetitionGuard` 此前不在 user-config KNOWN 白名单内,acp.json 中配置被静默丢弃(dead key),本次补入。测试 `tests/degeneration.test.ts`(31 例:单元 + context transform 端到端 wiring)
- **fix(reasoning): 闭合判定改按回合证据——无用户消息的长 agent 会话不再永久保留 compress thinking(closes #348)** — 原门控“compress 调用之后存在真实用户消息才算闭合”在长 agent 会话不可达(整个会话只有开头 1–2 条用户消息,后续 30 个 compress 全部被永久视为活跃回合,观察会话 0 次触发,thinking 地板 20.6K/8.4K/10.6K 字符全部滞留)。现在闭合判定改为:消息内**每个** compress toolCall 的 toolResult(role `toolResult`、`toolCallId` 匹配)已出现在更晚位置,且其后至少还有一条消息(回合已实际推进)。安全门不变:结果未返回或结果仍是最后一条消息(在飞中)绝不动;nudge 在 drop 之后才注入,不可能光当“结果后的消息”闭合在飞回合;per-provider `compress.providers.<name>.reasoning.drop=false` 逃生阀保留(GLM 等 reasoning 回显模型)。测试重写 + 新增 #348 场景(无用户消息的助手链闭合、result 悬置、result 在 call 之前、多 toolCall 部分闭合、误 id 不闭合)
- **feat(contract): ACP 压缩块机器可读契约(closes #368)** — sidecar 顶层新增 `schemaVersion`+`producer`(缺失=v1;未知高版本 bcp 侧「宽松加载已知字段 + 每文件记一次日志」,downgrade 不丢状态;仅破坏性变更升版本,其余只增不改);新增 `billion-context-pi/contract` subpath 导出(`BcpBlockV1` 类型、`SCHEMA_VERSION` 常量、`createSidecarEnvelope`)+ `schema/bcp-block-v1.json`(JSON Schema,随包发布),供下游跨进程/跨重启稳定读取块结构而无需绑定私有字段;README 记录 tmp+rename 原子写承诺(下游可安全用 mtime/size 做水位并整文件替换)。纯加法、零行为变更,不改 sidecar 位置与 pi 集成。测试:`tests/contract.test.ts` + `tests/state.test.ts`
- **fix(overflow): output headroom 预留按窗口比例封顶,默认 25%(closes #207)** — `reserveOutputHeadroom` 原按模型注册表 maxTokens **全额**预留输出预算:maxTokens 占窗口比例大的模型(qwen3.8-27b:262144 窗口 / 131072 maxTokens)输入预算被砍半,kernel 75% 强制压缩带在完整窗口 ~37% 处触发(host pct 仅 ~34%,两个口径不同加剧误导)。现在预留量 = min(maxTokens, `outputHeadroomMaxPct` × window):新增 acp.json 配置键 `outputHeadroomMaxPct`(默认 0.25,接受比例或 `"N%"`;0 完全禁用预留,≥1 恢复旧的全额行为)。小预留不受影响(同窗口 int4 版 32K maxTokens 保持原样),超出预留的超长回复溢出一次后由既有 overflow self-heal(learned window + armed emergency)下一轮恢复。可观测性:`[turn]` 日志新增 `fullWindow` 字段(仅当 limit 被预留削减时出现,= 本轮 recenter 后的完整窗口),消除 pct(完整窗口口径)vs limit(预留后口径)混淆;`output-headroom` 事件日志新增 `cap` 字段;`/acp` 面板与 `acp_status` 分母经 `applyOutputHeadroom` 同步使用同一封顶值(#267 统一口径不回归)
- **fix(state): in-memory(无 session 文件)会话压缩后状态不再丢失(closes #322)** — `src/state.ts` 的 cache 更新原本在 no-file early return 之后,导致没有 session 文件的 in-memory 会话压缩后状态不被保存(恢复/重启时丢失)。现在将 cache 更新移到 early return 之前,确保 in-memory 会话也持久化压缩状态。测试:`tests/state.test.ts` + `tests/compress-tool.test.ts` 扩展
- **fix(nudge): 增长基线刻度混用复位(closes #267)** — token 计量在估算值 ↔ provider 真实 usage 锚点间翻转(hostFloor 激活/失活)时,增长增量跨翻转相减是伪度量:旧刻度基线对新刻度 tokenCount 相减,凭空多出 "+35K 假增长",绕过 cadence 节流乱发 nudge(或反向漏发)。现在翻转时统一重置全部增长基线:kernel 侧 `lastNudgeShownTokens` / `lastPerMessageNudgeTokens` / `lastShownByTier`(0.0.55 per-tier cadence 基线)+ 扩展侧 `clearNudgeTokenStamps()`(#316 引入的同轮 re-inject 刻度戳,旧刻度戳同样会伪造满地板增长触发伪重注入);usage bands 的 floor-stale 行为保留不动。附带:② `/acp` 面板与 `acp_status` 百分比分母统一为与实时仲裁相同的 `window − maxOutput`(新增 `src/overflow-selfheal.ts` `applyOutputHeadroom`,此前面板按全窗口报告而 nudge bands 按 headroom 后窗口仲裁);③ auto-update 限流与只读停止标记按安装位置 sha256 隔离(多副本互不干扰)。实现:`src/runtime.ts` `noteTokenScale`/`dropTokenScale`,`src/index.ts` 翻转检测重置,`src/update.ts` `locationKey`;测试 `tests/growth-scale-flip.test.ts` + `tests/update.test.ts` 扩展
Expand Down
25 changes: 25 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -242,6 +242,31 @@ Restore them next to each other on the target machine. For clone/fork children,

> If you import only the `.jsonl`, ACP's log-replay fallback rebuilds the state automatically on the next session (see above). Copying the pair is still preferred — it is exact, while the replay re-derives token snapshots and can only restore what the transcript records.

### Machine-readable contract for compression blocks (#368)

Downstream tools that read a session's compressed blocks (cross-session search, memory indexes, …) should rely on this stable boundary instead of globbing private fields:

**Versioned sidecar.** Every `<session>.acp.json` carries a top-level envelope:

```json
{ "schemaVersion": 1, "producer": { "name": "billion-context-pi", "version": "<semver>" }, "blocks": [ … ], … }
```

Semantics:
- **Missing `schemaVersion` == v1.** Files written before this field existed are treated as version 1.
- **Unknown / higher version.** billion-context-pi stays *tolerant* when loading a higher `schemaVersion` (reads only the fields it knows, logs `unknown-schema-version` once per file) so a downgrade never silently drops state. **Consumers parsing the file themselves must do the opposite:** skip a file/block whose `schemaVersion` they don't understand, log once, and never rewrite it.
- **Bump only on breakage.** `schemaVersion` changes solely when a listed field is renamed, removed, or re-semanticked. New optional fields are additive and never bump the number.

**Public exports.** Import the pinned types/constants without pulling in the Pi host integration:

```ts
import { SCHEMA_VERSION, PRODUCER_NAME, BcpBlockV1, createSidecarEnvelope } from "billion-context-pi/contract";
```

A machine-readable JSON Schema for one block ships at [`schema/bcp-block-v1.json`](./schema/bcp-block-v1.json) (included in the npm package — point your validator at it). Each `blocks[]` element is a `BcpBlockV1`: the required stable core is `blockId`, `summary`, `tier`, `compressedTokens`, `createdAt` (**epoch milliseconds**, not ISO 8601); `topic` / `startRef` / `endRef` / `effectiveMessageIds` are optional coverage metadata. Extra properties are permitted — real blocks also carry internal pruning/index fields that are intentionally outside the contract.

**Atomic writes.** The sidecar is written as a temp file + `rename()` over the target, so a reader never observes a half-written file: watermark on mtime/size and always replace the whole file. The **file remains the source of truth** across process restarts.

## Built on acp-kernel

The compression engine is [`acp-kernel`](https://github.com/ranxianglei/acp-kernel) — a platform-agnostic, MIT-licensed library with 208 tests. It's bundled inline into `dist/index.js`, so there are zero runtime dependencies.
Expand Down
5 changes: 5 additions & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,10 @@
".": {
"types": "./dist/index.d.ts",
"import": "./dist/index.js"
},
"./contract": {
"types": "./dist/contract.d.ts",
"import": "./dist/contract.js"
}
},
"pi": {
Expand All @@ -17,6 +21,7 @@
},
"files": [
"dist",
"schema",
"README.md",
"LICENSE"
],
Expand Down
50 changes: 50 additions & 0 deletions schema/bcp-block-v1.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
{
"$schema": "http://json-schema.org/draft-07/schema#",
"$id": "https://github.com/ranxianglei/billion-context-pi/schema/bcp-block-v1.json",
"title": "BcpBlockV1",
"description": "One ACP compression block as persisted in a <session>.acp.json sidecar (an element of its top-level \"blocks\" array). Only the listed properties are part of the v1 stability contract: evolution is additive-only (new optional fields may appear), and renaming/removing/re-semanticking a listed field requires bumping the sidecar's schemaVersion. additionalProperties are permitted because real blocks carry extra internal pruning/index fields that are intentionally outside the contract.",
"type": "object",
"properties": {
"blockId": {
"type": "string",
"description": "Idempotency key. Stable across restarts; never reused."
},
"summary": {
"type": "string",
"description": "The final accepted summary text."
},
"tier": {
"type": "integer",
"enum": [1, 2, 3],
"description": "Compression tier."
},
"compressedTokens": {
"type": "integer",
"minimum": 0,
"description": "Tokens of original content compressed by this block."
},
"createdAt": {
"type": "integer",
"description": "Creation time, epoch MILLISECONDS (not an ISO string)."
},
"topic": {
"type": "string",
"description": "Short human label for the compressed range."
},
"startRef": {
"type": "string",
"description": "Model-facing start ref, e.g. m00005. Absent when unknown."
},
"endRef": {
"type": "string",
"description": "Model-facing end ref, e.g. m00020. Absent when unknown."
},
"effectiveMessageIds": {
"type": "array",
"items": { "type": "string" },
"description": "Raw message ids covered by this block (authoritative coverage set)."
}
},
"required": ["blockId", "summary", "tier", "compressedTokens", "createdAt"],
"additionalProperties": true
}
56 changes: 56 additions & 0 deletions src/contract.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
// Machine-readable contract for ACP compression blocks (#368). Zero runtime deps;
// safe to import across process/restart boundaries. The sidecar is the source of
// truth — this module only pins the versioned, additive-only block shape.

export const SCHEMA_VERSION = 1 as const;

export const PRODUCER_NAME = "billion-context-pi";

export interface ProducerInfo {
name: string;
/** Semver of the billion-context-pi build that wrote the sidecar. Omitted when unknown (dev/test). */
version?: string;
}

export interface SidecarEnvelope {
schemaVersion: number;
producer: ProducerInfo;
}

/** Build the top-level envelope written to every `<session>.acp.json`. */
export function createSidecarEnvelope(version?: string): SidecarEnvelope {
return {
schemaVersion: SCHEMA_VERSION,
producer: { name: PRODUCER_NAME, ...(version ? { version } : {}) },
};
}

/**
* One compression block as persisted in a sidecar's `blocks[]` element.
*
* v1 stability contract: listed fields are guaranteed present with stable names
* and semantics; evolution is additive-only (new optional fields may appear).
* Renaming/removing/re-semanticking any field requires bumping `SCHEMA_VERSION`.
* Fields not listed here are internal pruning/index details — not part of the
* contract and may change freely.
*/
export interface BcpBlockV1 {
/** Idempotency key. Stable across restarts; never reused. */
blockId: string;
/** The final accepted summary text. */
summary: string;
/** Compression tier (1 | 2 | 3). */
tier: number;
/** Tokens of original content compressed by this block. */
compressedTokens: number;
/** Creation time — epoch MILLISECONDS (not an ISO string). */
createdAt: number;
/** Short human label for the compressed range. */
topic?: string;
/** Model-facing start ref, e.g. "m00005". Absent when unknown. */
startRef?: string;
/** Model-facing end ref, e.g. "m00020". Absent when unknown. */
endRef?: string;
/** Raw message ids covered by this block (authoritative coverage set). */
effectiveMessageIds?: string[];
}
22 changes: 20 additions & 2 deletions src/state.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,18 @@ import { promises as fs } from "node:fs";
import * as path from "node:path";
import { createInitialState, type CompressionState } from "acp-kernel";
import { logError, logInfo, logWarn } from "./log.js";
import { createSidecarEnvelope, SCHEMA_VERSION } from "./contract.js";

const STATE_SUFFIX = ".acp.json";

declare const CURRENT_VERSION: string;

function producerVersion(): string | undefined {
return typeof CURRENT_VERSION !== "undefined" ? CURRENT_VERSION : undefined;
}

const warnedUnknownVersions = new Set<string>();

export interface LiveRefOrigin {
rawId: string;
identity: string;
Expand Down Expand Up @@ -60,7 +69,15 @@ export class SessionStateStore {
if (file) {
try {
const raw = await fs.readFile(file, "utf8");
const parsed = JSON.parse(raw) as CompressionState & { liveRefOrigins?: unknown };
const parsed = JSON.parse(raw) as CompressionState & { liveRefOrigins?: unknown; schemaVersion?: unknown };
// Tolerant forward-compat (#368): a higher schemaVersion means a future bcp wrote this
// file — we still load known fields below (refusing would drop state on downgrade) but
// log once per file so the downgrade is observable. Missing schemaVersion == v1.
const sv = typeof parsed?.schemaVersion === "number" ? parsed.schemaVersion : SCHEMA_VERSION;
if (sv > SCHEMA_VERSION && !warnedUnknownVersions.has(file)) {
warnedUnknownVersions.add(file);
logWarn("state", { event: "unknown-schema-version", file, found: sv, supported: SCHEMA_VERSION });
}
if (parsed && Array.isArray(parsed.blocks)) {
state = mergeInitialState(parsed);
liveRefOrigins = parseLiveRefOrigins(parsed.liveRefOrigins);
Expand Down Expand Up @@ -100,7 +117,8 @@ export class SessionStateStore {
});
const tmp = path.join(dir, `.acp-tmp-${path.basename(file)}`);
try {
await fs.writeFile(tmp, JSON.stringify({ ...state, liveRefOrigins }), "utf8");
const envelope = createSidecarEnvelope(producerVersion());
await fs.writeFile(tmp, JSON.stringify({ ...envelope, ...state, liveRefOrigins }), "utf8");
await fs.rename(tmp, file);
} catch (e) {
logError("state", { event: "save-failed", file, error: e instanceof Error ? e.message : String(e) });
Expand Down
52 changes: 52 additions & 0 deletions tests/contract.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
import { test } from "node:test";
import assert from "node:assert/strict";
import { readFile } from "node:fs/promises";
import * as path from "node:path";
import { fileURLToPath } from "node:url";
import { SCHEMA_VERSION, PRODUCER_NAME, createSidecarEnvelope, type BcpBlockV1 } from "../src/contract.js";

const here = path.dirname(fileURLToPath(import.meta.url));

// Must stay in sync with src/contract.ts BcpBlockV1 required fields and schema/bcp-block-v1.json.
const REQUIRED = ["blockId", "summary", "tier", "compressedTokens", "createdAt"];
const OPTIONAL = ["topic", "startRef", "endRef", "effectiveMessageIds"];

test("SCHEMA_VERSION and PRODUCER_NAME are stable", () => {
assert.equal(SCHEMA_VERSION, 1);
assert.equal(PRODUCER_NAME, "billion-context-pi");
});

test("createSidecarEnvelope: includes version when provided", () => {
assert.deepEqual(createSidecarEnvelope("0.1.65"), {
schemaVersion: 1,
producer: { name: "billion-context-pi", version: "0.1.65" },
});
});

test("createSidecarEnvelope: omits version when unknown (dev/test build)", () => {
const env = createSidecarEnvelope(undefined);
assert.equal(env.schemaVersion, 1);
assert.equal(env.producer.name, "billion-context-pi");
assert.ok(!("version" in env.producer), "version key must be absent, not undefined");
});

test("schema/bcp-block-v1.json stays in sync with the BcpBlockV1 contract", async () => {
const raw = await readFile(path.join(here, "..", "schema", "bcp-block-v1.json"), "utf8");
const schema = JSON.parse(raw) as {
type: string;
required?: string[];
additionalProperties?: boolean;
properties: Record<string, unknown>;
};
assert.equal(schema.type, "object");
assert.deepEqual([...(schema.required ?? [])].sort(), [...REQUIRED].sort());
assert.equal(schema.additionalProperties, true, "internal fields outside the contract must remain allowed");
for (const key of [...REQUIRED, ...OPTIONAL]) {
assert.ok(schema.properties[key], `schema must define property ${key}`);
}
});

test("BcpBlockV1 exposes exactly the documented required core", () => {
const block: BcpBlockV1 = { blockId: "b1", summary: "s", tier: 1, compressedTokens: 10, createdAt: 172 };
for (const key of REQUIRED) assert.ok(key in block, `BcpBlockV1 requires ${key}`);
});
Loading
Loading