Propose the audit plugin: bulk repository audits, starting with /audit:issues

.claude-plugin/marketplace.json

@@ -1,7 +1,7 @@
 {
   "name": "quantecon",
   "description": "QuantEcon's shared Claude Code skills and tools",
-  "version": "0.1.0",
+  "version": "0.2.0",
   "plugins": [
     {
       "name": "qe",
@@ -22,6 +22,16 @@
       },
       "version": "0.1.0",
       "description": "Benchmarking and acceleration-evaluation tools for QuantEcon lecture code"
+    },
+    {
+      "name": "audit",
+      "source": {
+        "source": "github",
+        "repo": "QuantEcon/skills",
+        "path": "audit"
+      },
+      "version": "0.1.0",
+      "description": "Bulk, read-only audits of a QuantEcon repository — issue triage, PR review, technical debt, translation parity — each producing an evidence-cited report bundle"
     }
   ]
 }

CATALOG.md

@@ -33,3 +33,15 @@ This delivers the planned `qestyle-linter` + `qestyle` pair as Claude Code skill
 ## 2. `/benchmark:review-acceleration` — finish
 
 Blocked on the eight evaluation scripts (@xuanguang-li agreed 2026-07-07 to package and send). **Unblocked now:** the full rubric (7 weighted dimensions, verdict bands, HIGH/LOW calibration cases) is specified in the lecture-python.myst#717 thread and can move into `references/rubric.md` ahead of the scripts. Validation cases: lecture-python.myst#717 and #654.
+
+## 3. `audit` — the bulk-audit family
+
+Portfolio-wide, read-only sweeps of one repository, each delivering a report bundle. Four skills: **`/audit:issues`** (every issue, open and closed — landed), then **`/audit:prs`** (every open PR: does it solve a real issue, is it mergeable, what should the review say), **`/audit:tech-debt`** (a codebase's debt plus a filing-ready issue catalog), **`/audit:translations`** (parity between a source series and its translation).
+
+**Evidence:** unlike the style family, the case here is not per-repo frequency — a tracker audit is a once-or-twice-a-year event for any one repo. It is *breadth*: the org has ~245 non-archived repos, and three of the four procedures have already been executed by hand — the issue triage this plugin ships, the `quantecon-py` technical-debt report, and the zh-cn translation work. Each hand run cost hours of re-derivation because the method lived in a pasted prompt.
+
+**Structure (proposed 2026-07-26):** four sibling skills, no umbrella — unlike `qe`'s categories, these are distinct procedures over distinct inputs, so the `qe` pattern to reuse is plugin-level sharing, not the umbrella. The method is authored once in `audit/references/` (`doctrine.md`, `quantecon-context.md`, `deliverables.md`) and each `SKILL.md` carries only its own subject matter. Membership test: bulk **and** read-only **and** report-bundle output — which keeps single-item review (`/benchmark:review-acceleration`) outside the family and stops the plugin becoming a general runbook dump.
+
+**Why read-only is structural, not cautious:** the boundary mirrors the org's own automation split, where the family line *is* the permission line. It makes the family safe to point at any repo and safe to run headlessly, and it keeps a report honest — an audit that half-applied its findings would describe a repo that no longer exists. Acting on a bundle (filing the catalog, posting drafted comments, `qe gh labels sync`) is a separate human-invoked step.
+
+**Long-run machinery:** every skill runs the same five phases (snapshot → verify → relate → write → self-audit), each checkpointed to disk so a lost session resumes rather than restarts, and all reading one frozen snapshot so the report describes a single point in time. Phase 1 is deterministic (`audit/scripts/fetch_tracker.py`), which also makes the coverage self-audit mechanical rather than narrated.

README.md

@@ -10,8 +10,9 @@ Each plugin bundles one area of work — a skill (the instructions Claude follow
 |---|---|---|---|
 | `qe` | `/qe:check-style` (+ `check-writing`, `check-math`, `check-code`, `check-figures`, `check-jax`, `check-refs`) | scaffolding | [CATALOG.md](CATALOG.md), work plan in `project-style-guide` |
 | `benchmark` | `/benchmark:review-acceleration` | under construction | [meta#335](https://github.com/QuantEcon/meta/issues/335) |
+| `audit` | `/audit:issues` (+ planned `prs`, `tech-debt`, `translations`) | first runbook landed | [CATALOG.md](CATALOG.md) §3 |
 
-The `qe` plugin is the author-facing surface: one memorable prefix for the skills authors use while editing lectures and preparing PRs. `check-style` is the umbrella (whole lecture, optional category filter, e.g. `/qe:check-style lectures/aiyagari.md figures math`); the per-category sub-skills run the same shared rules individually. `benchmark` is a specialist family for maintainers evaluating accelerated implementations. See [CATALOG.md](CATALOG.md) for the plan and [FUTURE-IDEAS.md](FUTURE-IDEAS.md) for parked candidates.
+The `qe` plugin is the author-facing surface: one memorable prefix for the skills authors use while editing lectures and preparing PRs. `check-style` is the umbrella (whole lecture, optional category filter, e.g. `/qe:check-style lectures/aiyagari.md figures math`); the per-category sub-skills run the same shared rules individually. `benchmark` is a specialist family for maintainers evaluating accelerated implementations. `audit` is the maintainer-facing family for bulk, read-only sweeps of a whole repository — every issue, every PR, a whole codebase, a whole translated series — each delivering a report bundle rather than a chat answer. See [CATALOG.md](CATALOG.md) for the plan and [FUTURE-IDEAS.md](FUTURE-IDEAS.md) for parked candidates.
 
 ## Installation
 
@@ -33,12 +34,15 @@ Lecture repositories opt in by checking the following into their `.claude/settin
 }
 ```
 
+`audit` is deliberately absent from the lecture-repo block: it is maintainer tooling, and the plugin is the enable unit, so auto-installing it would put org-wide audit skills in every author's command list. Maintainers install it themselves.
+
 ### Manual (any project)
 
 ```
 /plugin marketplace add QuantEcon/skills
 /plugin install qe@quantecon
 /plugin install benchmark@quantecon
+/plugin install audit@quantecon
 ```
 
 ### CI (GitHub Actions)
@@ -68,6 +72,11 @@ benchmark/                        # specialist plugin
   .claude-plugin/plugin.json
   skills/review-acceleration/SKILL.md
   scripts/                        # supporting Python scripts the skill drives
+audit/                            # maintainer-facing bulk-audit plugin
+  .claude-plugin/plugin.json
+  skills/issues/SKILL.md          # one skill per audit subject
+  references/                     # shared doctrine, org context, bundle contract
+  scripts/                        # deterministic snapshot + coverage machinery
 ```
 
 ## Contributing

audit/.claude-plugin/plugin.json

@@ -0,0 +1,6 @@
+{
+  "name": "audit",
+  "description": "Bulk, read-only audits of a QuantEcon repository — issue triage, PR review, technical debt, translation parity — each producing an evidence-cited report bundle",
+  "version": "0.1.0",
+  "author": { "name": "QuantEcon" }
+}

audit/README.md

@@ -0,0 +1,61 @@
+# audit
+
+Bulk audits of a QuantEcon repository. Where `qe` serves an author working on one lecture and `benchmark` evaluates one conversion, these skills sweep a whole portfolio — every issue, every PR, a whole codebase, a whole translated series — and deliver a report.
+
+## Skills
+
+| Skill | Audits | Status |
+|---|---|---|
+| [`/audit:issues`](skills/issues/SKILL.md) | Every issue, open and closed: status verified against the code, tiered into the repo's plan | runbook landed |
+| `/audit:prs` | Every open PR: does it solve a real issue, is it mergeable, what should the review say | planned |
+| `/audit:tech-debt` | A codebase's accumulated debt, with a filing-ready issue catalog | planned |
+| `/audit:translations` | Parity between a source series and a translation (`lecture-python.myst` ↔ `lecture-python.zh-cn`) | planned |
+
+Planned skills are tracked in [CATALOG.md](https://github.com/QuantEcon/skills/blob/main/CATALOG.md); this plugin ships with the first one so the shared doctrine is proven against a real procedure before three more are written on top of it.
+
+## What belongs here
+
+Three tests, all of which must pass:
+
+1. **Bulk** — it sweeps a portfolio, not an item. Reviewing one PR's technical quality is not an audit; reviewing all of them is.
+2. **Read-only** — it observes and reports. No skill here mutates a tracker, a branch, or a file in the audited repo ([doctrine §3](references/doctrine.md#3-read-only-boundary)).
+3. **Report bundle** — the output is the four-document bundle in [deliverables.md](references/deliverables.md), not a chat answer.
+
+The read/write line is deliberate and mirrors the org's own automation split, where the family boundary *is* the permission boundary. Anything that acts on findings — filing the catalog as issues, posting the drafted comments, applying labels — is a separate human-invoked step, which is what makes this family safe to point at any repo and safe to run headlessly.
+
+## Shared references
+
+Skills are thin; the method lives once at plugin level.
+
+| Document | Owns |
+|---|---|
+| [references/doctrine.md](references/doctrine.md) | Trust rules, evidence classes, read-only boundary, phases, coverage self-audit |
+| [references/quantecon-context.md](references/quantecon-context.md) | Repo types, label ownership, the cross-repo graph, notes-system discovery, access |
+| [references/deliverables.md](references/deliverables.md) | The report bundle contract and where bundles are allowed to land |
+| [scripts/](scripts/) | Deterministic phase-1 machinery |
+
+## Running one
+
+```
+/audit:issues QuantEcon/action-translation
+```
+
+Audits are long. They work from a frozen snapshot and checkpoint each phase to disk, so a run that loses its session resumes at the last completed phase rather than restarting — and every number in the report refers to one point in time. Expect hours, not minutes, on a repo with a hundred items.
+
+Headless runs work the same way:
+
+```yaml
+- uses: anthropics/claude-code-action@v1
+  with:
+    plugin_marketplaces: "https://github.com/QuantEcon/skills.git"
+    plugins: "audit@quantecon"
+    prompt: "/audit:issues QuantEcon/action-translation"
+```
+
+## A note on naming
+
+Skill names here are objects, because the plugin is the verb: `/audit:issues`, `/audit:translations`. The repo's rule is that an invocation reads as a command, and where the plugin name is a namespace or a domain (`qe`, `benchmark`) the verb has nowhere to live but the skill — hence `check-style`, `review-acceleration`. Where the plugin name is itself the verb, the skill is the object; `/audit:audit-issues` would stutter at every invocation.
+
+Keep the form consistent across the family. The failure mode is not the noun names but mixing them — an `/audit:compare-translations` beside `/audit:issues` breaks the parallelism that makes the four memorable.
+
+`audit` was chosen over `review` for the same reason the family excludes single-item work: `review` is already the per-item word here (`/benchmark:review-acceleration`, and PR review generally), so a `/review:prs` that sweeps every open PR would sit one keystroke from reviewing one. `audit` also matches QEP-3's `audit-` repo prefix and already connotes observe-and-report, which is the boundary this plugin enforces.

audit/references/deliverables.md

@@ -0,0 +1,37 @@
+# The report bundle
+
+Every audit in this plugin delivers the same four-document bundle. A shared shape means a reader who has read one audit can read any of them, and a later audit can diff against an earlier one mechanically (doctrine rule 7).
+
+## Where it goes
+
+**Never into `QuantEcon/skills`.** This repo holds the procedure; the findings belong with the thing audited. In order of preference:
+
+1. The audited repo's own notes system — `.dev/audits/<date>-<subject>/` or equivalent.
+2. The paired `project-*` repo, when the audit serves a program rather than a repo (`project-translation/reports/` for a translation audit).
+3. A dedicated `audit-*` repo, per [QEP-3](https://github.com/QuantEcon/qeps/pull/7), once an audit is recurring and its findings are worth publishing.
+
+Confirm the destination before phase 4 and name it in the report. Private inputs stay in private destinations: a `project-*` repo's contents must not be summarised into a public one.
+
+## The four documents
+
+**`01-<subject>-report.md`** — the argument. Method and evidence base; the snapshot timestamp; portfolio statistics; the findings that change a status, each with its evidence tag; the tiering, tied to the repo's existing plan; policy alignment; and a time-boxed execution order for whoever acts on it.
+
+**`02-<subject>-catalog.md`** — the enumeration. A legend, then a full summary table (item · tier · action · type · priority), then one entry per open item:
+
+> **Bold header line** — the item and its one-line characterisation
+> *Status:* what is actually true, with evidence and its tag
+> *Recommend:* the proposed action, and what it depends on
+> *Links:* related items, in and out of this repo
+
+Close with the verification of the closed set (explicitly thread-complete), and write up anything agreed-but-never-filed as a proposed new item, in a form that could be filed as-is.
+
+**`03-<subject>-links.md`** — the graph. Clusters and their anchor items; a concrete table of missing links worth adding; true orphans and over-dense hubs; and the external cross-link registry (sibling repos, program and meta issues, evidence PRs).
+
+**`README.md`** — the index. Headline numbers, what each document is for, and **the coverage statement**: exactly what was and was not inspected, per [doctrine §5](doctrine.md#5-coverage-self-audit). This is the first thing a reader should be able to check and the last thing the audit should write.
+
+## Rules that apply to all four
+
+- **Tag every claim** with its evidence class ([doctrine §2](doctrine.md#2-evidence-classes)). An untagged status claim is a defect.
+- **Recommendations are proposals.** Write them in the imperative for whoever executes ("close, citing PR #204"), but the audit does not execute them.
+- **Suggested comment text is a draft, in a fenced block, clearly marked as unsent** — and subject to the closing-keyword hazard in [quantecon-context.md](quantecon-context.md#the-cross-repo-graph-is-the-point).
+- **Date the bundle and name the snapshot.** Two audits of the same repo must be distinguishable at a glance, and every number in a bundle refers to one point in time.

audit/references/doctrine.md

@@ -0,0 +1,60 @@
+# Audit doctrine
+
+The method every skill in this plugin follows. Skills own their subject matter — what to verify, how to tier, what the link graph means — and inherit everything here. Authored once; a skill that needs a rule cites this file rather than restating it.
+
+## 1. What makes a bulk audit trustworthy
+
+1. **Never trust an artifact's claim about its own state.** Every "fixed in X", "scheduled into Y", "superseded by Z" is a hypothesis until checked against the current default branch, `CHANGELOG.md`, and merged PRs. Cite `file:line`, a PR, or a tag for each verified claim.
+2. **Check both directions.** Open items that are actually resolved, *and* closed items whose fix never really landed. An audit that only prunes is half an audit.
+3. **Read every thread, closed ones included.** Closed threads carry decisions, deferred remainders, and agreed-but-never-filed work. Skipping the closed side was the one gap in this runbook's first execution, and it is the default failure mode — the snapshot captures closed threads precisely so there is no cost excuse.
+4. **Distinguish guarded from fixed.** "Now fails loudly" is not "root cause resolved". Record which kind of remainder each item carries.
+5. **Respect explicit maintainer intent.** A "keeping this open because…" comment beats tidiness. Note it and move on; never recommend against a stated decision without new evidence.
+6. **Draft policies are conditional.** Where the governing policy is itself a draft or an open PR, recommend only what is already canonical, mark the rest *post-acceptance*, and never pre-empt the policy's own tooling.
+7. **Diff against prior audits.** If an earlier audit exists, reconcile with it: convergence is corroboration, divergence needs an explanation.
+8. **Match confidence to inspection.** Every claim's evidence class must be recoverable from the writeup.
+
+## 2. Evidence classes
+
+Findings are tagged so a reader can tell what was actually inspected. This is the machine-checkable form of rule 8, and it is what lets a reviewer trust a long report without re-running it.
+
+| Tag | Means | Minimum citation |
+|---|---|---|
+| `[verified]` | Checked against code, a diff, or a build artifact | `file:line`, a merged PR, or a tag |
+| `[stated]` | Asserted by a human in a thread or a notes file | Comment URL or `file:line` |
+| `[inferred]` | The audit's own reasoning across sources | The sources it reasons from |
+
+An unqualified claim is a defect. `[inferred]` is legitimate and often the most valuable class — but it must never be dressed as `[verified]`, and a status change recommended on `[inferred]` alone should say so in the recommendation itself.
+
+## 3. Read-only boundary
+
+**These skills observe and report. They never mutate the tracker** — no closing, labelling, milestoning, commenting, or editing, and no branch or file changes in the audited repo.
+
+That boundary is what makes the family safe to run headlessly and safe to point at any repo in the org. Every action an audit recommends lands as a proposal in the report, executed later by a human or by a separate, deliberately-invoked step. An audit that half-applied its own findings would also be an audit whose report no longer describes the repo it was run against.
+
+Corollary: an audit never writes into the plugin directory, and its outputs never live in `QuantEcon/skills`. See [deliverables.md](deliverables.md) for where they go.
+
+## 4. Phases, and surviving a long run
+
+Bulk audits outlive sessions. Context runs out, rate limits bite, machines sleep. Each skill runs the same five phases, and **each phase writes its output to the working directory before the next begins**, so a lost session resumes at the last completed phase instead of restarting.
+
+| Phase | Produces | Resumable from |
+|---|---|---|
+| 1. Snapshot | `meta.json`, `issues.json`, `prs.json`, `coverage.json` | — |
+| 2. Verify | per-item findings with evidence tags | the snapshot |
+| 3. Relate | the cross-link / parity graph | phase 2 |
+| 4. Write | the report bundle | phases 2–3 |
+| 5. Self-audit | the coverage statement, folded back in | all of the above |
+
+Phase 1 is deterministic and belongs in [`../scripts/`](../scripts/), not in model judgement. Fetching once also **freezes the audit's point in time**: every later claim refers to the snapshot, and "events after the snapshot" becomes a stated property of the report rather than an unnoticed gap. Record the snapshot timestamp in the report; never silently mix fresh API reads into a later phase.
+
+## 5. Coverage self-audit
+
+Before delivering, reconcile — and **close any gap found rather than merely disclosing it**. `coverage.json` from phase 1 does the mechanical half: items captured against the number sequence `1..max`, threads captured against items with comments, and a truncation flag when a stream returns exactly at the fetch limit.
+
+The judgement half is the skill's:
+
+- Explain every unaccounted number. Gaps are usually deleted or transferred items, or numbers burned by branches that never opened a PR — but an unexplained gap looks identical to a silent truncation.
+- State the residue explicitly: what was *not* inspected. PR review threads? Native sub-issue and Projects membership, both invisible to REST? Anything after the snapshot time?
+- If the self-audit changes a finding, fold the change back into the documents. Never append a correction section — a report whose conclusions and body disagree is worse than one that is simply late.
+
+**No silent caps.** If the audit bounded its own coverage anywhere — sampled, capped at top-N, skipped a class of item — say so in the coverage statement. Silent truncation reads as complete coverage, which is the one failure an auditor cannot recover from.