Repository navigation
Show token usage from ACP agents (registry and local command) on the Usage page #32083
Replies: 4 comments
|
I've built step 1 for Z.ai GLM only and proposed it in #32173, with a working implementation linked. One constraint for the generic version: some registry agents front CLIs that T3 already reads natively ( |
|
TL;DR: Copilot CLI and Mistral Vibe both send Strong +1 for the generic reader. I have data from two more ACP Registry agents that the Usage page does not show today: GitHub Copilot CLI ( I connected to each agent with a small ACP client. The client sent
Both agents send 1. In these probes, both agents report running totals for the session, not per-turn numbers. Between the two prompts, the context in use (
One approach is a second flag in the agent flavor (the per-agent settings), for example This also answers the opt-in question: use an allowlist. Each agent on the list declares whether its usage is per-turn or cumulative. The list grows as agents are verified, which is what the flavor comment asks for. 2. No other blocker for these two. Neither agent wraps a CLI whose session logs T3 already reads, so the double-counting caveat above does not apply to them. The skip list for wrappers like If you prefer a reader that scans transcripts on disk, both agents also write per-call records there:
But the wire path (reading My answers to the three maintainer questions:
I can rerun this probe on new agent versions and test a branch on macOS. |
|
For Sinaps Studio at SinapsData, we snapshot dispatch identity before an attempt so later configuration edits cannot relabel historical usage. Would this reader likewise preserve the model value and its source at turn completion? The thread’s selected model is useful metadata, but when an ACP agent can route internally it should remain distinct from an agent-reported execution model, especially before attaching a price. |
|
#31776 Just jumping in here to include the work I have done on getting this working. Currently working on a second PR for usage limit reporting. |
Uh oh!
There was an error while loading. Please reload this page.
Problem
ACP agents never appear on the Usage page. That includes ACP Registry agents and agents added through Local ACP command. I use ZCode through
zcode-acp-serveras a local ACP command. It works well for chatting, but none of its turns show up in Usage. The same gap is behind two existing requests that name single agents:Instead of adding one transcript reader per agent, I'd like to propose one generic path for every ACP agent, based on data T3 already receives and stores.
What T3 already does (as of 6577c6b)
PromptResponse.usage(packages/provider-acp/src/server/adapter.ts:8376). It maps the value toturnTokenUsagethroughacpPerTurnPromptUsage, andturnTokenUsageis persisted with each provider turn inorchestration_v2_projection_provider_turns.payload_json.flavor.reportsPerTurnPromptUsage. Today the gate is on only forglm-acp-agent(packages/provider-acp-registry/src/server/adapter.ts:227) and Antigravity.usage_update(used,size,cost?) feeds only the context meter (contextUsageBySessionId).BUILT_IN_USAGE_DRIVERS(apps/server/src/provider/builtInDrivers.ts).AcpRegistryDriverhas nousagereader, andUsageProviderKindis a closed enum (packages/contracts/src/usage.ts).So per-turn token counts for ACP agents are already captured and stored, at least where the gate is on. The Usage page just never reads them.
Proposal (small, incremental)
Add a
scanusage reader toAcpRegistryDriver. It would emitUsageRecords from persisted provider turns that belong to ACP-registry or local-command instances:timestampMsfromcompleted_atdedupeKeyfromprovider_turn_idsessionIdfrom the threadtotalsfromturnTokenUsageThis needs no new storage and no agent-specific transcript parsing.
Widen
UsageProviderKindso these records can be labeled. This could be a single"acp"kind grouped by provider instance, or the instance's display name, whichever fits the Usage UI best.Let more agents opt into
reportsPerTurnPromptUsage. I understand why the gate exists: semantics differ between agents, for example whetherinputTokensincludes cache reads. Possible options:PromptResponse.usagepasses the existing clamping, and mark recordspartialwhen the subsets don't add up.Cost, optional and as a follow-up. Agents that send
usage_update.costin USD could setreportedCostUsd. Otherwise, records would show tokens with unknown cost when the model isn't in the rates table.Kiro's credit-based accounting (#27621) wouldn't be covered by step 1 as-is. It could extend the same reader later.
Questions for maintainers
"acp"provider kind, or labeling by provider instance?reportsPerTurnPromptUsagewould you be comfortable with?If the direction looks right, I'm happy to open a focused PR for steps 1–2, with tests next to the existing
apps/server/src/usage/*.test.ts.All reactions