fix(daemon): full ACP tool transcript and richer usage telemetry#6057
fix(daemon): full ACP tool transcript and richer usage telemetry#6057mrcfps wants to merge 17 commits into
Conversation
Map all ACP tool_call frames to canonical tool_use/tool_result for run.events, PostHog run_finished, and Langfuse spans. Preserve artifact_count via Write/Edit normalization, tighten path extraction, forward truncated results, and emit real tool startedAt. Enrich usage scanning (thought_tokens, anthropic vs openai cache semantics), canonical tool-family PostHog props, analytics recovery finish snapshot, and shared content-tool redaction.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 40539a649d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
lefarcen
added
size/XXL
lefarcen
added
the
needs-validation
|
🧪 Queued for QA validation — this PR changes live run events / contract behavior that can affect what users see in chat when ACP tools stream, so we'll keep the QA pass in the loop before merge. Nothing needed from you right now; we'll update here once validation is done. Thanks for the cleanup on the observability path. |
Normalize recognized ACP kinds (e.g. kind:read + name:read_file → Read) before emitting tool transcripts so Langfuse content-tool redaction matches canonical families and does not leak file contents. Generated-By: looper 0.11.0 (runner=fixer, agent=grok-build)
|
Looper fixer round complete — 980bfd4
🔁 Powered by Looper · runner=fixer · agent=grok-build · An autonomous AI dev team for your GitHub repos. |
|
To use Codex here, create an environment for this repo. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 980bfd4df6
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Generated-By: looper 0.11.0 (runner=fixer, agent=grok-build)
|
Looper fixer round complete — eb4911d
🔁 Powered by Looper · runner=fixer · agent=grok-build · An autonomous AI dev team for your GitHub repos. |
|
To use Codex here, create an environment for this repo. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: eb4911d9a7
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
When a session dies mid-tool (timeout, exit, RPC error), emit pending tool_use/tool_result pairs with isError so transcripts stay complete. Generated-By: looper 0.11.0 (runner=fixer, agent=grok-build)
|
Looper fixer round complete — 859f1bd
🔁 Powered by Looper · runner=fixer · agent=grok-build · An autonomous AI dev team for your GitHub repos. |
|
To use Codex here, create an environment for this repo. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 859f1bd839
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Execute tools can dump private files (e.g. cat .env); replace Bash tool_result content with a length summary before the canonical transcript ships so Langfuse cannot receive raw execute output. Generated-By: looper 0.11.0 (runner=fixer, agent=grok-build)
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d592a37b26
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
Looper fixer round complete — d592a37
🔁 Powered by Looper · runner=fixer · agent=grok-build · An autonomous AI dev team for your GitHub repos. |
|
To use Codex here, create an environment for this repo. |
Extend local-path masking for ACP Bash partial redaction to cover /home and /root (not only /Users and Windows Users), so Linux home paths do not leak into Langfuse tool inputs when content telemetry is on. Generated-By: looper 0.11.0 (runner=fixer, agent=grok-build)
|
Looper fixer round complete — 8437d4b
🔁 Powered by Looper · runner=fixer · agent=grok-build · An autonomous AI dev team for your GitHub repos. |
|
To use Codex here, create an environment for this repo. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8437d4be17
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
Looper fixer round complete — 67dfa84
🔁 Powered by Looper · runner=fixer · agent=grok-build · An autonomous AI dev team for your GitHub repos. |
|
To use Codex here, create an environment for this repo. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 67dfa84ac9
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Require both input and output before ending the reverse usage scan so output-only or input-only trailing frames still merge earlier counters. Flush deferred open ACP tools as errored on abort so user-canceled runs keep complete tool transcripts for Langfuse/PostHog. Generated-By: looper 0.11.0 (runner=fixer, agent=grok-build)
|
Looper fixer round complete — 527eb10
🔁 Powered by Looper · runner=fixer · agent=grok-build · An autonomous AI dev team for your GitHub repos. |
|
To use Codex here, create an environment for this repo. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 527eb10fb9
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Adapters can embed user paths in toolCallId (e.g. read:/home/alice/.env). Hash those values at tool_use/tool_result emit so Langfuse span ids and metadata.toolCallId cannot leak secrets, while keeping the raw id as the local frame-correlation map key. Generated-By: looper 0.11.0 (runner=fixer, agent=grok-build)
|
Looper fixer round complete — 49272eb
🔁 Powered by Looper · runner=fixer · agent=grok-build · An autonomous AI dev team for your GitHub repos. |
|
To use Codex here, create an environment for this repo. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 49272eb38e
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Identifier-shaped secrets (sk-proj-*, ghp_*, JWTs) passed the prior allowlist because they contain no path characters. Hash every non-empty adapter toolCallId before tool_use/tool_result emission so Langfuse span ids and metadata.toolCallId cannot leak tokens. Generated-By: looper 0.11.0 (runner=fixer, agent=grok-build)
|
Looper fixer round complete — 1180633
🔁 Powered by Looper · runner=fixer · agent=grok-build · An autonomous AI dev team for your GitHub repos. |
|
To use Codex here, create an environment for this repo. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 118063343a
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
When a newer usage frame already has input+output but omits cache counters, continue scanning older frames so cache_read/cache_creation still merge into run_finished and Langfuse. Generated-By: looper 0.11.0 (runner=fixer, agent=grok-build)
|
Looper fixer round complete — 137e1c2
🔁 Powered by Looper · runner=fixer · agent=grok-build · An autonomous AI dev team for your GitHub repos. |
|
To use Codex here, create an environment for this repo. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 137e1c28b2
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Keep acpToolRunEventState entries after flushOpenAcpTools so a racy tool_call_update cannot recreate the same id and emit a second tool_use/tool_result pair after timeout, abort, or prompt flush. Generated-By: looper 0.11.0 (runner=fixer, agent=grok-build)
|
Looper fixer round complete — 7c35252
🔁 Powered by Looper · runner=fixer · agent=grok-build · An autonomous AI dev team for your GitHub repos. |
|
To use Codex here, create an environment for this repo. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7c35252545
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Custom ACP tools with kind:other must not emit Bash/shell/execute/terminal names, which would unlock Langfuse partial redaction and leak rawInput. Generated-By: looper 0.11.0 (runner=fixer, agent=grok-build)
|
Looper fixer round complete — 953fd8d
🔁 Powered by Looper · runner=fixer · agent=grok-build · An autonomous AI dev team for your GitHub repos. |
|
To use Codex here, create an environment for this repo. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 953fd8d5f8
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
ACP tools missing kind must not preserve name/title Bash labels that unlock Langfuse partial redaction; only recognized execute-family kinds may emit Bash. Generated-By: looper 0.11.0 (runner=fixer, agent=grok-build)
|
Looper fixer round complete — d0958e5
🔁 Powered by Looper · runner=fixer · agent=grok-build · An autonomous AI dev team for your GitHub repos. |
|
To use Codex here, create an environment for this repo. |
2 participants
Why
Local Open Design Prerelease AMR/ACP runs stored incomplete tool transcripts (only synthetic
Writepairs with garbage paths and empty results) and weak token usage for PostHog/Langfuse. Investigation of failed/succeeded prerelease runs showed ~475 ACP tool frames collapsing to 19 fake Writes, and usage often missing or only partially reported. This blocks observability of real agent workflows (bash/read/search loops) and token accounting.Author use case: fix trace data storage and reporting after a prerelease handoff so internal Langfuse/PostHog debugging matches what the agent actually did — without changing UI code.
What users will see
No intentional product UI change. Chat may show more accurate tool cards for ACP agents (AMR, Hermes, etc.) because the daemon now emits real
tool_use/tool_resultevents instead of a write-only shim. Analytics/telemetry for operators become complete: full tool families, thought tokens, and correct cache accounting onrun_finishedand Langfuse traces.Surface area
apps/weborapps/desktop(including Electron menu bar)odsubcommand or flag, newtools-dev/tools-pack/tools-prflag, or newOD_*env var/api/*endpoint, new SSE event, or changed shape inpackages/contractsskills/,design-systems/,design-templates/, orcraft/, or change to the skills protocolTRANSLATIONS.mdfor the locale workflow)package.json(dependenciesordevDependencies); workspace-packagepackage.jsonfiles are out of scope. Include a paragraph on what we get vs. what bytes we ship (seeCONTRIBUTING.md→ Code style)Contract note: optional
startedAton live/persistedtool_usefor ACP tool duration; no new endpoints.Screenshots
N/A — no UI surface changes.
Bug fix verification
apps/daemon/tests/acp.test.ts(full tool transcript, path tighten, kind authority, think-only, exactly-once)apps/daemon/tests/acp-format-usage.test.ts(usage aliases + anthropic/openai cache semantics)apps/daemon/tests/amr-acp-integration.test.ts(AMR no-output / think-only)apps/daemon/tests/run-analytics-observability.test.ts(unique tool counts, canonical tool families, thought_tokens)apps/daemon/tests/langfuse-bridge.test.ts/langfuse-trace.test.ts(content redaction, thoughtTokens, startedAt)Validation
pnpm --filter @open-design/daemon exec vitest run -c vitest.config.ts tests/acp.test.ts tests/acp-format-usage.test.ts tests/run-analytics-observability.test.ts tests/langfuse-bridge.test.ts tests/amr-acp-integration.test.ts— 177 passedpnpm --filter @open-design/daemon typecheck— passpnpm guard— passlangfuse-tracereportRunFeedbackfailures on main (default relay URL env) are unrelatedSummary of fix
tool_use+tool_resultper id with real names/inputs/results; Write/Edit normalize forcountNewArtifacts; sticky kind + thinkOnly;startedAtfrom first frame.formatUsage; preserve anthropic vs openai cache fields;thought_tokensthrough PostHog + Langfuse; usage-before-fail when prompt result carries usage.startedAt.