audit:issues: name the phase checkpoints, scale the bundle, fix snapshot provenance

[mmcky]

Three defects found reviewing /audit:issues against what it promises. The review started from the suspicion that the plugin was over-complicated; that is not what it found, and none of these three is a consolidation. The packaging question is parked on #16 instead, where it can be answered from a run rather than from taste.

The defects

1. The checkpoints had no names. audit/README.md, the skill body and doctrine.md all promise a run that survives a lost session by resuming at the last completed phase — it is the plugin's headline property, and the reason the snapshot script exists. But only phase 1 named its artifacts. Phase 2 produced "per-item findings with evidence tags" and phase 3 "the cluster map", with no filename and no location, so resuming worked only if two independent sessions invented the same file. Phase 2 is also the phase a hundred-item run dies inside rather than between, and it was specified to write on completion — which is no checkpoint at all, precisely where one was needed.

The skill now has a working-directory table (snapshot/, findings.md, links.md, then the bundle), phase 2 appends one entry per item as it is verified in the catalog entry format from deliverables.md — so phase 4 assembles the catalog rather than re-deriving it — and the resume rule is stated: read findings.md, resume at the lowest number in issues.json with no entry, re-verify the last entry rather than trusting a possibly truncated write. Doctrine §4 gains the general form, since it applies to any long audit: a checkpoint owes a findable name and incremental writes, or it is a claim about resumability rather than the property.

2. The bundle did not scale. Four documents were unconditional. deliverables.md hedges the shape ("another audit may need two of these, or none") but the skill did not, so a 20-issue tracker got three files of padding. Padding makes a report less checkable, which is the one thing the four-document shape exists to protect. Below roughly 30 open issues the catalog and the link graph now fold into the report, with the shape used named in the coverage statement.

3. meta["authenticated"] could only ever be true. preflight() exits on every unauthenticated path and returned a bare True for the field to record. A provenance field that cannot vary carries no information — a poor look in the plugin whose doctrine is that every claim carries its evidence class. Replaced with fetched_by, the account the snapshot was taken as, which does vary and matters: visibility is per-account, so two snapshots of a private repo taken by different people can legitimately differ, and knowing who fetched is the only way to tell that from a truncation. Non-fatal — a failure records null rather than exiting, because it means gh's user endpoint was unavailable, not that the snapshot is unauthenticated.

What was deliberately not changed

Worth recording, because the review proposed each of these first and then withdrew them on merit:

Proposed	Why not
Delete audit/scripts/README.md	qe/scripts/README.md exists for a directory with no scripts yet — it is a convention, not an accident. It is never loaded at run time, so it costs no context, and it carries one command (gh api repos/OWNER/REPO/pulls/N/comments) that appears nowhere else.
Drop the SKILL.md status blockquote	benchmark's SKILL.md opens with an identical one. House pattern for recording provenance and the tracking issue.
Trim the script's docstrings	The agent runs fetch_tracker.py; it never reads it. That prose is maintainer documentation, and this repo explains its reasoning on purpose.
Fold deliverables.md into the skill	Neutral at best, and churn if /audit:prs ever needs it back. Parked on #16.
Collapse repeated facts to one home each	Selective, not wholesale. A specification wants one home; an instruction at point of use — the GraphQL gap stated where phase 1 runs — is better repeated than linked.

Validation

Ran against QuantEcon/skills: 8 issues, 7 PRs, numbers 1..15 fully accounted, fetched_by populated. python scripts/validate.py passes all three plugins. Version bumped to 0.1.1 in both plugin.json and the marketplace.json entry.

Follow-up

The larger finding is not addressed here and cannot be: the skill has been executed once, by hand, as a runbook, and never as a skill. Its abstractions are generalised from that one execution. #16 is the validation program — a claims-under-test table, a four-repo run matrix anchored on re-running action-translation and diffing against the hand run, and exit criteria that include capturing the worked example benchmark has and audit does not. The more-structure-or-less question from #12 is deferred to after the QuantEcon/meta run, the case furthest from the original execution and so the sharpest test of what actually generalises.

🤖 Generated with Claude Code

[Copilot]

Pull request overview

This PR updates the /audit:issues skill and its snapshot machinery to make long runs genuinely resumable, scale deliverables to the size of the tracker, and improve snapshot provenance by recording who fetched the data rather than a non-informative auth boolean.

Changes:

• Document and enforce named, findable phase checkpoints, with phase 2 writing incremental per-item findings to findings.md for reliable resume.
• Scale the delivered bundle shape for small trackers by folding the catalog and link graph into the report under a stated threshold.
• Replace meta.authenticated with meta.fetched_by in the snapshot provenance, recorded non-fatally from gh api user.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
audit/skills/issues/SKILL.md	Defines the working directory layout, names phase artifacts, specifies incremental phase-2 checkpointing, and documents bundle scaling rules.
audit/scripts/README.md	Updates snapshot output provenance documentation to match meta.json’s new fetched_by field.
audit/scripts/fetch_tracker.py	Removes the always-true authenticated provenance and records fetched_by (best-effort) in meta.json.
audit/references/doctrine.md	Strengthens doctrine §4 to require named artifacts and incremental writes for resumability; updates the phase table to reflect partial phase-2 output.
audit/.claude-plugin/plugin.json	Bumps the audit plugin version to 0.1.1.
.claude-plugin/marketplace.json	Bumps the marketplace audit entry version to 0.1.1.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.