Release v6.1.2: 2026-07-06 evaluation fixes to main#11
Merged
Conversation
The protocol said "Run 10 independent finder angles" while defining 11 angle sections and an 11-value angle enum, leaving the parallel fan-out instruction ambiguous. The intro also named Claude Code's /code-review command in a protocol consumed on harnesses without that command. Both edits are factual alignment with no semantic change; recorded in evaluation.md's new Protocol revisions section. The verbatim-import header now discloses local revisions, and the sha256 pin plus wording assertions in the integration test track the edited file. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SYxaVSy1xS1PFzAAesybUJ
Scenario 10 expected triage output to include child issue drafts, which SKILL.md forbids outright (decomposition moved to decomposing-issues and drafting child bodies during triage is a listed Red Flag). The stale expectation would grade correct behavior as a regression. The scenario now expects a Decomposition Handoff and forbids drafted child bodies. The When To Use exclusion routed PR/diff review to requesting-code-review, which is the pre-review checklist for one's own work; review requests belong to superpowers:code-review. The test assertion pinning the old routing asserted the opposite direction and is updated to match. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SYxaVSy1xS1PFzAAesybUJ
…, clean Four operational gaps in scripts/run-skill-evidence.sh: - Sessions ran unbounded; a hung CLI blocked forever. They now run under with_timeout (SKILL_EVIDENCE_TIMEOUT, default 1800s) using timeout(1) when present, else the base-system perl alarm+exec. - Transcripts were written directly at their final path, so an interrupted session left a plausible-looking half transcript in the evidence tree. They are now built in a same-directory temp file and mv'd into place on completion, with an EXIT trap for the temp file. - Rerunning the same skill/slug/harness/phase on the same day silently truncated an existing (possibly committed) transcript. The run subcommand now refuses to overwrite, matching the append-only evidence policy. - Credential copies seeded into the scratch root persisted forever. Scratch dirs are now chmod 700 and a clean subcommand removes the scratch root; also added a command -v presence check for the harness CLI. Verified: shellcheck clean; overwrite guard and clean exercised directly; real claude red preflight passed end-to-end under the timeout wrapper (PREFLIGHT OK); with_timeout kills a 30s hang in 2s (rc=124). The plan document embedding the original runner is marked as a dated historical snapshot rather than kept in sync. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SYxaVSy1xS1PFzAAesybUJ
The fork's only divergence in this file was a deleted trailing space — zero value, and it kept a tuned upstream skill on the sync conflict surface for nothing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SYxaVSy1xS1PFzAAesybUJ
Only code-review has committed green transcripts (8 green plus a no-skill control red, verified by directory count); the four issue-workflow skills have RED baselines only, with green results existing solely as paraphrased evaluation rows. The evidence tree's own standard is transcript-or-it-didn't-happen, so the gap must be stated where the standard is stated. Backfill is tracked as follow-up work. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SYxaVSy1xS1PFzAAesybUJ
Blocking semantics (P0-P2 block, P3 non-blocking, human-only acceptance, rerun-after-fix) were previously stated only in the two workflow skills that route here, while code-review itself — where a gate consumer actually lands — defined none of it. The new section is declared the single authoritative definition. It also closes two gaps the evaluation record exposed: priority labels drift between runs on the same finding (evaluation.md documents P1/P2 swaps), so the latest run's labels are declared authoritative; and the fix-and-rerun loop had no termination condition, so a circuit breaker now stops the loop after two consecutive cycles with an undiminished blocking count and escalates with the findings list. Red Flags gains the matching entry, and the integration test pins the new section. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SYxaVSy1xS1PFzAAesybUJ
…ding
The two-step exact-draft gate had no escape hatch: even an explicit
session-level grant from the human ('for this session, apply labels
without asking') was overridden by the blanket-approval clause, forcing
per-mutation confirmation in owner-operator automation. A standing
pre-authorization now lifts per-mutation confirmation for the mutation
categories the human names in the session. The injection surface does
not widen: repository files still cannot grant it, it is never inferred
from task phrasing, each applied mutation is still shown, and anything
outside the named categories falls back to two-step approval.
The gate text had also drifted across copies: triaging lacked the
two-step exact-draft semantics entirely. Triaging gains it, the new
pre-authorization block is byte-identical in all four issue-workflow
skills, and the decomposing test now asserts that cross-file identity
so future drift fails a test instead of rotting silently. Mutation
Preview semantics extend to record applied mutations under a
pre-authorization. Two pressure scenarios cover the grant and the
repo-file-claims-authorization refusal; live eval runs for them are
tracked as follow-up.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SYxaVSy1xS1PFzAAesybUJ
The description listed bare 'bug report, feature request, support request' as triggers — words that need no tracker context — so a pasted stack trace or a feature idea competed with systematic-debugging and brainstorming and could pull a one-hop task into the three-hop triage pipeline. The trigger list now requires issue-tracker context (issue, issue URL/number, pasted issue report, Triage Result) and explicitly excludes tracker-free bug/feature/support asks. pressure-scenarios gains the previously missing negative cases (a local stack trace and a bare feature idea, both marked must-not-trigger), and the test pins the exclusion plus the negative scenarios. Live eval runs are tracked as follow-up. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SYxaVSy1xS1PFzAAesybUJ
The fork carries ~900 lines of structural tests — including the review-protocol sha256 pin, the evidence-link integrity loop, and the cross-file gate-block identity check — that only ran when someone remembered to run them. They now run on every push and PR. Upstream ships no .github/workflows, so the file adds zero sync conflict surface. Shellcheck runs at warning level to match the current clean baseline (one pre-existing SC2016 info note is deliberate literal-backtick usage). Tool dependencies (shasum, zip, unzip, python3) are all present in the ubuntu-latest image. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SYxaVSy1xS1PFzAAesybUJ
…ario, narrow CI triggers Independent branch review returned no blocking findings and four P3s; three are applied here: - reconciling's main output template hard-coded 'No GitHub mutation was performed' on the very path where a standing pre-authorization can now legitimately perform one, contradicting the gate prose and producing a false audit line; the template fields now carry both states. The Blocked template keeps the hard-coded line — blocked reconciliation performs no mutation by construction. - the gate's key guard 'never inferred from task phrasing' had no negative pressure scenario anywhere; decomposing (the gate's authoritative home) gains one where vague 'handle it as you see fit' phrasing must not be treated as pre-authorization. - CI push triggers narrow to main/dev so PR branches run once and tag pushes not at all. The fourth P3 (unpinned rolling shellcheck on ubuntu-latest) is accepted, not fixed: pinning adds upgrade chores that outweigh fixing a rare new warning; the baseline version is noted in the workflow. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SYxaVSy1xS1PFzAAesybUJ
GitHub issues are disabled on this fork, so the nine follow-ups from the 2026-07-06 evaluation (GREEN transcript backfill, control-baseline discrimination, skill slimming items, upstream PR tracking, live eval runs) land in a committed tracker instead of only in a session summary. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SYxaVSy1xS1PFzAAesybUJ
First CI run on ubuntu failed in the packaging test's tar.gz leg: scripts/package-codex-plugin.sh (upstream code, fork only added TZ=UTC) uses bsdtar-only flags (--uid/--gid/--uname/--gname) that GNU tar rejects, so the script has always been macOS-only and ubuntu merely exposed it. The job now runs on macos-latest, matching the script's real support surface; the GNU tar fact is recorded in followups.md next to the upstream packaging-PR tracking entry rather than patched locally while upstream obra#1901/obra#1910 are in flight on the same file. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SYxaVSy1xS1PFzAAesybUJ
macos-latest does not preship shellcheck (the first macos run passed both test steps and died at shellcheck with command not found); install it via brew when absent. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SYxaVSy1xS1PFzAAesybUJ
Fix 2026-07-06 evaluation findings: protocol count, gate semantics, runner hardening, trigger narrowing, CI
The updater compares manifest versions; PR #10 changed skill content without bumping, so installed copies would stay at the 6.1.1 tree. 6.1.2 is a fork-local version ahead of upstream's 6.1.1 by design — it re-converges at the next upstream release sync. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SYxaVSy1xS1PFzAAesybUJ
This was referenced Jul 8, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Who is submitting this PR? (required)
claude-fable-5)What problem are you trying to solve?
PR #10's fixes live on
dev, but installs followmain(the marketplace manifest points at the repo withsource: "./"on the default branch) and the updater compares manifest versions. Without a release tomainplus a version bump, installed copies stay at the 6.1.1 tree and none of the merged fixes reach users.What does this PR change?
Releases
devtomain: everything merged in PR #10 (2026-07-06 evaluation fixes — protocol count correction, gate semantics, mutation-gate pre-authorization, runner hardening, trigger narrowing, fork CI, follow-up tracker) plus a version bump to 6.1.2 in both plugin manifests.6.1.2 is a fork-local version, deliberately ahead of upstream's 6.1.1: the fork's 6.1.1 already diverged from upstream's 6.1.1 in content (the shadow-version confusion flagged in the 2026-07-06 evaluation), so staying on the same number was the worse option. It re-converges at the next upstream release sync.
Is this change appropriate for the core library?
Fork-internal release PR (
dev→mainon heyi-co/superpowers). Nothing here targets upstream.Existing PRs
Fork release convention follows PR #9 (sync/release to
main) and commit a52e289 (the 6.1.1 bump, same two files). No competing release PR is open.Verification
devcontent already reviewed, tested, and CI-verified in PR Fix 2026-07-06 evaluation findings: protocol count, gate semantics, runner hardening, trigger narrowing, CI #10.devafter the Fix 2026-07-06 evaluation findings: protocol count, gate semantics, runner hardening, trigger narrowing, CI #10 merge (run 28794104832, 22s) and runs again on this PR.Environment table
🤖 Generated with Claude Code
https://claude.ai/code/session_01SYxaVSy1xS1PFzAAesybUJ