test: deterministic fake-upstream real-codex compression E2E (#686)

[ranxianglei]

What

The "real regression" requested in #686 — compression actually happens inside real codex, driven by a deterministic fake Responses-API upstream. No model, no tokens, no external secrets, so it can gate every PR.

Distinct from the two earlier #686 follow-ups:

• PR test: add Windows codex overlay MCP-registration check (#686) #688 = Windows MCP-registration layer (config-level, headless).
• PR test: add containerized real-codex E2E (docker) (#686) #697 = full lifecycle in Docker against a real model upstream (needs secrets).
• This = real codex + real in-process compression, but with a fake upstream → deterministic and secret-free → runnable on every PR.

How it works

• tests/e2e/fake-upstream.mjs — standalone fake serving the two wire contracts verified against codex-cli 0.147.0:
  • stream:true → Responses SSE assistant turn;
  • stream:false → plain JSON body that bili's summarization call parses (extractSummaryText → json.output[].content[].text).
  • Recognises bili summarization calls by their exact TASK text in instructions; replies with a faithful summary preserving the sentinel numbers found in the segment, dropping bulk filler. Every /v1/responses request is logged to FAKE_REQLOG as the assertion oracle.
• tests/e2e/e2e-codex-fake.test.ts — gated by ACP_TEST_E2E_FAKE=1:
  • overflow (window 12k): warms up planting sentinels, then pushes ~10KB of bulky filler on every load turn via resume --last. Asserts: a real compression event fires (log), summarization is called repeatedly (oracle), the forwarded payload stays bounded despite heavy injection (compression folds the bulk instead of letting it accumulate), sentinels survive into the final payload, and the [Compressed conversation section] block is present.
  • control (window 60k): one small turn asserts no compression and no summarization calls (no false positives).

Local usage

npm i -g @openai/codex@latest   # or set E2E_CODEX_BIN
ACP_TEST_E2E_FAKE=1 npm run test:e2e:fake

Validation (this sandbox, Linux)

Ran against the real codex-cli 0.147.0: typecheck clean · build ok · 2/2 pass (~64s). The overflow run shows repeated preflight compressed events and a forwarded payload that grows only ~1KB across four ~10KB filler injections — i.e. compression is genuinely folding context inside codex.

Caveat / review note

The fake encodes the exact SSE/JSON contract of codex-cli 0.147.0. If @openai/codex changes its wire format on a future @latest bump, this test fails loudly (by design) and the fake needs updating — same maintenance trait as the existing tests/e2e/e2e-codex.test.ts. Nothing pinned, matching that convention.

Scope vs #686

Covers the "real compression really happens in codex + assert behaviors" ask. The Windows-specific acceptance items (concurrent-window livePidHoldsOverlay warning, interactive compress-tool visibility) still need a human on a real Windows box — tracked under #688 / #683.

Refs #686

---

中文摘要：按你"用 fake api 造数据、让压缩真的在 codex 里发生、并断言一系列行为"的要求，加了一个确定性假上游 E2E（真实 codex + 进程内真压缩，零 token/零密钥），可作每次 PR 的门禁；已在本地对真实 codex-cli 0.147.0 验证 typecheck/build/2 用例全绿。

[github-actions]

📦 Built Package Artifact

Branch: 2026-09-11_fake-api-e2e (97df100)

Option A — Install from npm PR tag (recommended)

npm install -g billion-context@pr-715

Each push to this PR publishes a new version under the pr-715 npm tag.

Option B — Download artifact

1. Download the artifact from the Actions run
2. Extract the tarball and install:

tar xzf billion-context-pr715.tgz
npm install -g package

---

This comment is automatically updated on each push.

[ranxianglei]

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 Reviewed PR #715 against real codex-cli 0.147.0 in this sandbox. Verdict: mergeable — reproduced green end-to-end and verified every load-bearing assumption against source. No defects found that need a commit.

What I ran (all green)

• typecheck clean · build ok · default unit suite 1371 pass / 0 fail / 2 skip.
• E2E (ACP_TEST_E2E_FAKE=1): 2/2 pass in ~64s — matches your claim.

The test is not vacuous — oracle artifacts confirm real in-process compression

I inspected the fake's request log after the run:

• overflow dir: 10 requests = 5 summarization + 5 forwarded. Forwarded payload sizes [80420, 80660, 80944, 81228, 81512] → grew only ~1.1KB across four ~10KB filler injections (peak/min ratio 1.014, well under the 1.3 bound). Final payload carries [Compressed conversation section] and all three planted sentinels (4781 / 2903 / 6577). So compression genuinely folds the bulk instead of accumulating it.
• control dir (60k window): 1 request, 0 summarization calls → no false positive.

If the filler weren't being folded, turn 5 would be ~+40KB and the ratio assertion would fail — so the bound has real discriminating power.

Load-bearing assumptions verified against source

• src/preflight.ts:227 — Responses summaryPayload puts the summary TASK into instructions, which is exactly what the fake's isSummaryRequest inspects.
• src/preflight.ts:312 — the TASK text contains both regex alternatives the fake matches ("must be compressed because the session context exceeds" + "Write a tier-1 compression summary").
• BILI_LAUNCHER_MODEL_WINDOWS (src/server.ts:162) and --no-auto-update (src/cli.ts:174) are real; the /bili/<absolute-url> base_url matches the documented tunnel convention (README).
• Diff is clean: 4 purpose-related files, package.json adds only the test:e2e:fake script (no version bump), no lockfile/generated churn. Code follows §3 (precise type assertions, no as any/@ts-ignore).

Non-blocking notes

• This becomes a mandatory gate on every PR. That's sound here because it's secret-free and self-contained (spawns its own fake upstream), unlike ci-e2e.yml which needs secrets + an unreachable localhost upstream and stays manual-only. The one real cost is already acknowledged in the PR: @openai/codex@latest wire-format drift will fail all PRs until the fake is updated. Flagging so you're consciously accepting that maintenance tradeoff for repo-wide gating.
• The peak <= minLen * 1.3 bound passed with a comfortable margin (1.014); it holds because codex's fixed scaffolding dominates the baseline and the folded filler is small relative to it. Left as-is.
• Scope vs E2E verify on real Windows: codex accepts CODEX_HOME overlay and registers the bili MCP server (#681 / PR #683) #686 looks right: Windows-specific items are correctly deferred to test: add Windows codex overlay MCP-registration check (#686) #688/fix: deliver codex MCP server via CODEX_HOME overlay on Windows (#681) #683.

Duplicate screening

Distinct from the other two #686 follow-ups — #688 (Windows MCP-registration, config-level/headless) and #697 (Docker full lifecycle vs a real model, needs secrets). This one is the deterministic fake-upstream variant (real codex + in-process compression, zero tokens/secrets). Not a duplicate; keep open.

---

中文摘要：在真实 codex-cli 0.147.0 上复现了 typecheck/build/单测(1371 过)/E2E(2/2 约 64s)全绿，并核对假上游的断言确实有效（溢出用例转发体四次注入仅涨约 1.1KB、保留全部哨兵值，对照用例零压缩），关键假设逐条对源码核实无误，diff 干净无版本号改动，可合并；唯一需你确认的是它成为每次 PR 的强制门禁后，codex 线格式漂移会阻塞所有 PR，这点维护成本请知悉。