Skip to content

refactor: strip social proof, self-selling, and recap detritus from 12 skills (eval-gated)#1934

Draft
obra wants to merge 12 commits into
devfrom
skill-detritus-cleanup
Draft

refactor: strip social proof, self-selling, and recap detritus from 12 skills (eval-gated)#1934
obra wants to merge 12 commits into
devfrom
skill-detritus-cleanup

Conversation

@obra

@obra obra commented Jul 5, 2026

Copy link
Copy Markdown
Owner

Note for @arittr: hold merge until this has been run through evals.
This branch exists to BE the eval treatment arm: one skill per commit,
so effects can be bisected commit-by-commit. Several cuts (TDD
narrative, verification-before-completion persuasion sections) are
deliberate bets that the rationalization tables alone hold under
pressure — the evals decide, not the diff.

Who is submitting this PR? (required)

Field Value
Your model + version Claude Fable 5 (claude-fable-5)
Harness + version Claude Code CLI 2.1.199
All plugins installed superpowers 6.1.0, claude-session-driver 4.0.0, episodic-memory, superpowers-chrome, context7, linear, plugin-dev, agent-sdk-dev
Human partner who reviewed this diff Jesse Vincent (@obra) — directed the audit and the one-branch structure; full-diff review pending (hence draft)

What problem are you trying to solve?

Skills carry content that burns context tokens on every load without
shaping behavior: social-proof sections ("Real-World Impact" with
15-30-minute/95% statistics), self-selling sections ("Advantages",
"Key Benefits") aimed at a reader who has already invoked the skill, and
end-of-file recaps ("Remember", "The Bottom Line", "Key Principles") that
restate rules already at point of use. A 14-agent audit (one auditor per
skill, shared brief distinguishing detritus from pressure-tested behavior
control) surveyed every skill; every finding on this branch was then
verified by hand against the claimed point-of-use duplicate before
cutting — the audit's numbers were not trusted blind (one was inflated
2.6x).

What does this PR change?

Twelve commits, one skill each, net −222/+11 lines:

  • Social proof/self-selling deleted: dispatching-parallel-agents
    (Real-World Impact, Key Benefits), systematic-debugging (Real-World
    Impact, Overview throat-clearing), subagent-driven-development
    (Advantages), executing-plans (quality claim), requesting-code-review
    (workflow index + mechanism-rationale sentence).
  • Recap sections deleted after verifying every line exists at point of
    use: brainstorming (Key Principles — with YAGNI, a sole carrier the
    audit missed, moved into Exploring approaches first), writing-plans
    (Remember), writing-skills and receiving-code-review (The Bottom Line).
  • using-git-worktrees: Common Mistakes + Red Flags converted to one
    Common Rationalizations table (house Excuse/Reality form), preserving
    the Add problem-solving skills from amplifier patterns #1-mistake emphasis on bypassing native worktree tools.
  • Eval-bet cuts: verification-before-completion loses Why This Matters,
    the dishonesty reframing, and The Bottom Line;
    test-driven-development loses Why Order Matters (five prose rebuttals,
    each mapped to an existing Common Rationalizations row and Red Flags
    entry before deletion).

Judgment calls that diverge from the raw audit, for reviewers to check:
systematic-debugging's "95% of no-root-cause cases" line STAYS (it guards
the bail-out point — rationalization control, not social proof);
receiving-code-review's Common Mistakes table STAYS (one compact guard
table per skill is the pattern this cleanup standardizes toward, not
zero).

Is this change appropriate for the core library?

Yes — deletions and one table conversion in core skills; no new
dependencies or integrations.

What alternatives did you consider?

  • Many small branches: rejected by the maintainer — one branch with
    per-skill commits is the unit the eval pipeline can bisect.
  • Skip the eval-gated cuts (TDD, verification): the safe default, but
    it leaves the actual question — does persuasion prose in discipline
    skills do anything? — unanswered. Landing them on the eval branch is
    how the question gets answered; they are trivially revertable
    per-commit.
  • Trusting the audit numbers: rejected after the first verification
    found a haiku auditor had inflated a section's size ~2.6x; every cut
    was re-verified against the file by hand.

Does this PR contain multiple unrelated changes?

Twelve skills, one theme: removing content that does not change agent
behavior, found by one audit, structured for one eval run. The commits
are separable on purpose — reverting any single commit restores that
skill exactly.

Existing PRs

Environment tested

Harness (e.g. Claude Code, Cursor) Harness version Model Model version/ID
Claude Code CLI 2.1.199 Fable 5 (controller/verifier), Haiku 4.5 (audit fan-out) claude-fable-5, claude-haiku-4-5

New harness support (required if this PR adds a new harness)

N/A — no new harness.

Evaluation

  • Initial prompt: "skills like dispatching-parallel-agents have detritus
    like social proof 'real world impact'... we should audit all the skills
    for that... fan out haiku subagents one per skill."
  • The audit itself: 14 parallel auditors, one per skill, shared brief
    with an explicit protected-content list (Iron Laws, rationalization
    tables, thought-tables, menus, digraphs) and the test "if this text
    vanished, would an agent act differently?"
  • Verification on this branch: every deletion was checked by hand against
    the point-of-use duplicate the auditor claimed (grep/read), which
    caught one inflated size claim, one misclassified guard line, and one
    sole-carrier (YAGNI) that required a move instead of a delete.
  • Behavioral before/after evals: NOT yet run — that is this branch's
    entire purpose and the pre-merge gate for @arittr. The discipline-skill
    cuts (TDD, verification-before-completion) should get the heaviest
    pressure scenarios.

Rigor

  • If this is a skills change: I used superpowers:writing-skills and
    completed adversarial pressure testing (paste results below)
  • This change was tested adversarially, not just on the happy path
  • I did not modify carefully-tuned content (Red Flags table,
    rationalizations, "human partner" language) without extensive evals
    showing the change is an improvement

The two unchecked boxes are honest: this branch modifies pressure-tested
discipline skills ahead of the evals that would justify it, by design —
the maintainer chose to structure the question as one eval-able branch.
No rationalization-table rows or "human partner" language were changed;
the cuts remove prose adjacent to those tables. Nothing here merges
without the eval pass.

Human review

  • A human has reviewed the COMPLETE proposed diff before submission

Draft PR: opened for @obra's full-diff review and @arittr's eval pass; box
intentionally unchecked until that review happens.

obra added 12 commits July 5, 2026 12:22
Real-World Impact restated the Real Example from Session as statistics;
Key Benefits and the time-saved line sold the skill to a reader already
executing it. Instructions unchanged.
Real-World Impact was statistics; the Overview opener restated the core
principle as motivation. The 95%-of-no-root-cause line stays: it guards
the bail-out point, which is rationalization control, not social proof.
Supporting Techniques/Related skills untouched (PR #1932 owns that).
…ompletion

Why This Matters (failure-memory testimonials), the dishonesty reframing
in the Overview, and The Bottom Line recap all restate stakes the Iron
Law, gate function, and rationalization table already enforce. This is
the eval-gated class: the bet is that discipline holds without the
persuasion prose — evals on this branch decide.
The tell-your-partner directive and the prefer-SDD instruction stay; the
significantly-higher-quality sentence restated them as a claim.
Integration section untouched (PR #1932 owns it).
…pment

Five blocks of benefits and cost/benefit selling aimed at a reader who
has already invoked the skill; the vs-Executing-Plans comparison also
duplicates the one under When to Use. Integration section untouched
(PR #1932 owns it).
…view

Integration with Workflows restated the When to Request Review triggers
grouped by caller (each-task/before-merge/when-stuck all appear at point
of use), and the intro's mechanism-rationale sentence sold a rule the
preceding sentence already states.
…nalization table

Common Mistakes and Red Flags restated Steps 0-3 wholesale; both fold
into one Common Rationalizations table (house Excuse/Reality form) whose
five rows carry the tempting-thought version of each rule, including the
#1-mistake emphasis on bypassing native tools. Quick Reference stays as
the compact decision aid.
Five of six principles restated the Checklist and Process sections
verbatim-in-spirit. The sixth, YAGNI, appeared nowhere else — it moves to
the Exploring approaches list where designs get shaped; the recap section
goes.
All four lines restate the Overview (DRY/YAGNI/TDD/frequent commits),
Task Structure (exact paths, commands with expected output), and No
Placeholders (complete code in every step).
Restates the Iron Law, the RED-GREEN-REFACTOR mapping, and the
TDD-for-docs framing, all stated in full earlier in the file.
Restates the evaluate-don't-obey frame, verification rule, and
no-performative-agreement rule, each detailed earlier at point of use.
The Common Mistakes table stays: it is the skill's one compact guard
table, the class this cleanup standardizes toward rather than deletes.
…evelopment

The section's five prose rebuttals each map to a Common Rationalizations
row (test-after, manual-tested, sunk-cost, dogmatic, spirit-not-ritual),
and every excuse phrasing also appears in the Red Flags list. This is the
highest-stakes cut on the branch: TDD is the most pressure-tested
discipline skill, and the bet that the table alone holds under pressure
is exactly what the eval pass must decide.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant