Skip to content

No way to see what agents cost per module and task - #15

Merged
github-actions[bot] merged 1 commit into
mainfrom
improve/stats-tokens
Oct 29, 2026
Merged

github-actions[bot] merged 1 commit into
mainfrom
improve/stats-tokens

Conversation

@zolotarev

Copy link
Copy Markdown
Owner

Friction — token cost is the main complaint of the first project's owner, but dpcli could not say where tokens go: skills/harness-retro/cost.py measures sessions, not modules or tasks, so process changes (v18 per-wave coordinator, v20/v22 Edit instead of Bash edits) could not be verified by numbers.

Change — stats --tokens reads Claude Code's subagent logs of the project (~/.claude/projects/<project>/*/subagents/*.jsonl); an agent is attributed by its first message, which is a dpcli brief (<ID> <type>: …, Review <ID>: …, Module <m>: …; the Russian protocol of older journals too). Prints context tokens of executors / reviewers / coordinators per module and the per-task median. Other agents are not counted. Opt-in: reading the logs takes ~3 s on ~300 agent logs.

Real numbers from the first project (anonymized): per module, one coordinator processed 76–148M context tokens — about as much as all executors of the module together; per-task median 4–16M. That is the evidence behind v18's "one coordinator per wave".

Test: tests/test_stats_tokens.py (fake logs: executors, reviewer, coordinators in both languages, a non-dpcli agent).

… Claude Code subagent logs

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@zolotarev zolotarev added the feedback Friction reported from real use label Oct 29, 2026
@github-actions

Copy link
Copy Markdown

VERDICT: accept

  • Real friction with evidence in the PR body: process changes (v18, v20/v22) could not be checked by numbers. The feature is opt-in, stdlib-only and read-only.
  • Correct on reading: the brief regex covers the English and Russian protocols. A non-dpcli agent is skipped. A missing log directory gives empty output. Journals are untouched. I did not run the tests myself, so CI must be green.
  • Test tests/test_stats_tokens.py covers executors, reviewer, coordinators in both languages and a foreign agent. It would fail without the change.
  • Process is complete: VERSION 24, CHANGELOG line, docs/guide.md row. No prompt or template growth, so tools/budgets.json is unaffected.
  • Nit: agents whose first message looks like a brief but whose task ID is not in the journal (deleted or renamed task) are dropped without a word. CLAUDE.md says never truncate silently, so consider printing unattributed: N agents as a one-line hint.
  • Nit: task_module maps ID to module with the last one winning. This is fine while task codes are unique per module, but note the assumption.
  • Nit: the "context tokens" figure sums input and cache reads per turn, so it counts re-reads. The label says "context", which is fine. Say in the docs row that output tokens are excluded.

@github-actions
github-actions Bot merged commit 3f91659 into main Oct 29, 2026
3 checks passed
@github-actions
github-actions Bot deleted the improve/stats-tokens branch October 29, 2026 19:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feedback Friction reported from real use

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant