Skip to content

Disambiguate colliding skill descriptions: 32 intent clusters where specialists never fire #2244

Description

@laurigates

What

Several skills describe the same user intent in near-identical language, so the router resolves the collision by picking whichever skill already has momentum and the specialists never fire. A usage-grounded audit of 265 enabled skills found 32 such clusters.

This is a disambiguation problem, distinct from the length problem in #2119. The two axes interact and are worth sequencing deliberately — see Relationship to #2119 below.

Evidence base: 400 session transcripts, plus a full-transcript pass over 427 sessions / 29,071 tool calls with per-call AttributionPlugin / AttributionSkill. Counts below are lifetime uses / attributed tool calls in 30d.

Why

The clearest case: testing-plugin is 17 skills for 1 lifetime use. "Run the tests" fires constantly — it just never lands there. It is absorbed by git-plugin:git-issue (48/234), whose description reads "Process GitHub issues end-to-end with TDD and parallel work", and by blueprint-plugin:blueprint-prp-execute (4/88), "validation loop, TDD, quality gates".

The six runner skills give the router no discriminator:

Skill Description head Uses
testing-plugin:test-run "Universal test runner…" 0/0
testing-plugin:test-quick "Run fast unit tests only…" 0/0
testing-plugin:test-full "Run complete test suite…" 1/0
testing-plugin:test-focus "Run one test file fail-fast…" 0/0
testing-plugin:test-tier-selection "Auto-select test tiers…" 0/0
testing-plugin:test-report "Show cached test status…" 0/0

Six skills each claiming "run tests" means no single one carries a distinguishing phrase, so a generalist with adjacent vocabulary wins by default. The same shape recurs across the catalog.

How

Incremental and gated, one cluster per PR — not a bulk sweep. #1278 established that bulk description edits silently break auto-invocation, and #2119 carries the same caution. Every PR keeps the literal Use when and passes scripts/audit-skill-descriptions.py --strict-all plus the length bands in .claude/rules/skill-quality.md.

The general rewrite rule, in priority order:

  • Lead with the artifact or object, not the verb. "Audit", "configure", "run" are shared by dozens of skills and carry no routing signal. configure-github-pages should open with actions/deploy-pages, not "GitHub Pages deployment workflows".
  • Name the discriminator when siblings share a domain. Where a run-vs-configure split exists, state it in the head: "Run the project's linter" vs "Set up linting and wire it into CI".
  • Delete territory a skill does not own. A description that lists four capabilities claims four intents and starves four siblings.
  • Model the good case. comfyui-plugin:comfy-registry-lifecycle (5/237) wins its traffic by naming concrete symptoms — "lockfile drift traps", "the empty-web/dist publish bug". Symptom nouns route; category nouns collide.

Ranked clusters

1. Test running — testing-plugin
Collapse the six runners into one test-run whose description names modes as flags ("quick / full / focused / tiered; cached status"). Add the literal phrase "run the project's tests" so it can win against the workflow skills above. Retire the other five and project-plugin:project-test-loop.

2. Lint running vs lint setup
code-quality-plugin:code-lint (47/1521, the busiest skill in the catalog) claims "linting code, auto-fixing, formatting, or running pre-commit checks" — the territory of configure-plugin:configure-linting (1/0), configure-formatting (0/0), configure-pre-commit (3/0), configure-editor (0/0). Keep both sides; the run-vs-configure split is real. Narrow code-lint to running and auto-fixing, drop its pre-commit clause, and front-load "Set up / install / wire into CI" on the four configure-* siblings.

3. Rules authoring — session-plugin:session-distill
At 39/413 it claims "codifying workflow into .claude/rules" plus "skill improvements" plus "cross-repo promotions", absorbing blueprint-plugin:blueprint-curate-docs (3/0), blueprint-rules (2/0), and blueprint-claude-md (16 lifetime, 0 in 30d). Re-scope session-distill to the end-of-session routing decision, and re-cast the blueprint trio around their artifacts ("edit CLAUDE.md", "add a path-scoped rule file", "library gotchas").

4. Commit → PR pipeline (11 skills, one winner)
git-plugin:git-commit-push-pr (48/71) absorbed the atomic steps. Five convention references are at 0/0: git-commit-workflow, git-commit-trailers, github-pr-title, git-branch-naming, github-issue-autodetect. Merge those five into one git-plugin:git-conventions reference that the pipeline skill links to instead of competing with. Keep git-commit / git-push / git-pr only if each leads with the scope limit it already carries ("local commits only", "from pushed branches").

5. GitHub Actions (9 descriptions, 0 attributed calls between them)
configure-workflows (1/0), ci-workflows (2/0) and configure-reusable-workflows (3/0) are indistinguishable from text alone — merge into one. Keep configure-container, configure-github-pages, configure-argocd-automerge, configure-release-please, each rewritten to lead with its artifact, and remove the generic "GitHub Actions workflows" phrasing from all of them.

6. Tail — description-only fixes
Session resume (session-spinup 21/259 vs project-continue 6/0); issue triage (git-issue 48/234 vs git-triage 28/11, with git-issue-hierarchy / git-issue-manage / github-labels at 0/0); issue filing (git-plugin:github-issue-writing 4/12 vs communication-plugin:ticket-drafting-guidelines 5/13 — an even split across plugins, i.e. a per-session coin flip); repo onboarding (configure-select positions itself as "instead of running /configure:all", which no router can resolve); meta-audit (health-check 4/15 enumerates a superset of its eight siblings, and the purpose-built health-skill-audit has never run); release-please split across configure-plugin and git-plugin; red-CI ownership split four ways.

Clusters where every member is unused

Two clusters have no traffic on any member, which makes them catalog weight rather than a routing problem: test quality (code-test-quality, test-quality-analysis, test-strategy-review, mutation-testing, test-consult — five skills splitting one intent) and visual regression (playwright-testing, playwright-cli, odiff-image-diffing, configure-ux-testing). Worth a separate decision on whether they earn their catalog entries.

Relationship to #2119

#2119 proposes C5-shaped shortening toward a compressed capability head plus one Use when trigger, based on experiments/skill-catalog-routing/FINDINGS.md.

These are orthogonal axes — length and collision — and shortening interacts with disambiguation in both directions:

  • Risk: compressing toward a generic capability head can increase collision. "Run tests", "Set up tests" and "Analyze tests" are all valid C5 heads that collide harder than the fuller descriptions they replace.
  • Opportunity: C5 puts the capability phrase first, which is exactly where a discriminator does the most work. A rewrite pass that picks the distinguishing capability phrase satisfies both goals at once.

Suggested sequencing: choose the discriminator first, then compress it — rather than compressing first and disambiguating the survivors. If #2119's trial batch is still being selected, the clusters above are a natural candidate set, since they are the cases where a wrong compression is measurable rather than theoretical.

The existing harness can measure this directly: a before/after routing spot-check on one cluster (six test-runner skills → one) would show whether disambiguation moves routing independently of length.

Caveat on the source data

The inventory backing this audit extracted description: as a single line, so 21 of 265 entries whose frontmatter uses a YAML block scalar (>- / |) were captured as the marker instead of the text. Affected: 17 comfyui-plugin skills, 2 foundryvtt-plugin, plus git-plugin:github-issue-writing and git-plugin:github-pr-title.

Only the last two appear in any recommendation above, and both sit in clusters whose verdict rests on usage counts (0/0) rather than description text. The comfyui-plugin and foundryvtt-plugin slices were not meaningfully audited and should be re-derived from a block-scalar-aware inventory before any action is taken on them.

References

  • Related: #2119 (description length / C5 shortening), #1278 (bulk description edits broke auto-invocation)
  • Gates: scripts/audit-skill-descriptions.py --strict-all, .claude/rules/skill-quality.md
  • Harness: experiments/skill-catalog-routing/ (FINDINGS.md)

Metadata

Metadata

Assignees

Labels

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions