Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
c9ecde9
docs(runtime): replace listing-collapse model with per-skill recency-…
yaniv-golan Aug 5, 2026
6de7cbc
fix(portability): rename listing-collapse-risk to listing-desc-drop-r…
yaniv-golan Aug 5, 2026
22d82b7
fix(portability): stop flagging Cowork's preinstalled stack; advisori…
yaniv-golan Aug 5, 2026
fcde8cc
fix(runtime-docs): allowed-tools grants permission, it does not reque…
yaniv-golan Aug 5, 2026
7cfaeef
docs(cowork): document the two skill listings and account-synced skil…
yaniv-golan Aug 5, 2026
67cc1d8
docs(cowork): retract the cwd-scratchpad rationale, split surface-vs-…
yaniv-golan Aug 5, 2026
1c6cb50
fix(skill): persist the viewer PID to a file; refresh stale model IDs…
yaniv-golan Aug 5, 2026
e7ba19c
docs(runtime): add compaction budget, skill content lifecycle, backgr…
yaniv-golan Aug 5, 2026
ce6e9ba
chore: bump version to 0.9.0
yaniv-golan Aug 5, 2026
72e88eb
docs: state corrected rules in the present tense, not as retraction n…
yaniv-golan Aug 5, 2026
631cef6
docs(dev): document check_portability's rules, gating and baseline
yaniv-golan Aug 6, 2026
8b0f5f4
docs(dev): gate releases behind explicit maintainer approval; tag by …
yaniv-golan Aug 5, 2026
d352759
docs: document the compaction budget as a character limit
yaniv-golan Aug 6, 2026
0a1c99b
fix(skill): workspace must not be a sibling of the skill directory
yaniv-golan Aug 5, 2026
60141b2
feat(portability): add compaction-truncation-risk — an exact SKILL.md…
yaniv-golan Aug 6, 2026
cd2229c
chore(harness): migrate to cowork-harness 1.19.0
yaniv-golan Aug 6, 2026
632c095
test(harness): verify the negative control survives the proactive-sug…
yaniv-golan Aug 6, 2026
57b6cee
docs(changelog): tighten the 0.9.0 entry to release-note form
yaniv-golan Aug 6, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 11 additions & 6 deletions .github/workflows/harness.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,14 +22,14 @@ jobs:

- name: Install cowork-harness (pinned)
run: |
npm i --prefix "$RUNNER_TEMP/ch" cowork-harness@1.16.0
npm i --prefix "$RUNNER_TEMP/ch" cowork-harness@1.19.0
echo "$RUNNER_TEMP/ch/node_modules/.bin" >> "$GITHUB_PATH"

- name: Verify cowork-harness version
run: |
V=$(cowork-harness --version | tail -1)
echo "cowork-harness $V"
case "$V" in 1.16.*) ;; *) echo "expected 1.16.x, got $V"; exit 1;; esac
case "$V" in 1.19.*) ;; *) echo "expected 1.19.x, got $V"; exit 1;; esac

- name: Lint skill (strict)
run: |
Expand All @@ -41,7 +41,10 @@ jobs:

- name: Lint harness scenarios
run: |
cowork-harness lint harness/scenarios/
# --strict --min-severity WARN per cowork-harness 1.18.0: without --strict the step
# CANNOT fail on a WARN-class rule (it prints and exits 0), and bare --strict also
# fails on advisory INFO. Both flags are required for this gate to gate.
cowork-harness lint --strict --min-severity WARN harness/scenarios/

- name: Load-check scenarios (token-free)
run: |
Expand All @@ -52,10 +55,12 @@ jobs:

- name: Verify + replay cassettes (if any)
run: |
# --allow-empty (1.17.0) greens an EXISTING but cassette-free dir; a MISSING path still
# exits 2, so it cannot green a typo. `replay` still needs the guard, so the branch stays.
# No allowlist flags — cowork-harness >=1.16.0 no longer emits the benign claude.com
# MCP-handshake false positive that once required --allow-domain. Any finding is real.
cowork-harness verify-cassettes harness/cassettes --allow-empty
if ls harness/cassettes/*.cassette.json >/dev/null 2>&1; then
# No allowlist flags — cowork-harness >=1.16.0 no longer emits the benign claude.com
# MCP-handshake false positive that once required --allow-domain. Any finding is real.
cowork-harness verify-cassettes harness/cassettes
cowork-harness replay harness/cassettes
else
echo "No committed cassettes by design (self-cassettes of an actively-developed skill go stale every edit; see harness/README.md). The static checks above are the CI gate; cassettes are recorded on-demand / in the nightly lane."
Expand Down
30 changes: 28 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,32 @@

All notable changes to this project will be documented in this file.

## [0.9.0] - 2026-08-06

Every Claude-runtime claim in this repo was re-verified against the shipping Claude Code binary. Four were wrong and are corrected below.

### Added
- **`compaction-truncation-risk` lint rule.** Flags a `SKILL.md` over the 19,900-character limit that survives auto-compaction, and names the line where the cut falls. The cap is a fixed character count in the runtime, so the check is exact rather than a heuristic. Advisory severity.
- **Compaction-budget guidance in `references/official-guide-patterns.md`.** The per-skill cap is documented as "5,000 tokens" but enforced as **19,900 characters** (100,000 combined across skills), because the runtime sizes skill content by character count rather than tokenizing it. Measure with `wc -m`. Also documents two behaviours that are easy to lose content to: truncation is written back, so a second compaction cannot recover the tail, and combined-cap overflow drops a skill's content entirely for the rest of the session.
- **Cowork shows a skill listing twice, and only one copy carries `when_to_use`** — a second reason never to put trigger-critical content there. Cowork and cloud sessions also load skills from your claude.ai account rather than `~/.claude/skills/`, so local edits are not what runs.
- **Skill content lifecycle**: an invoked `SKILL.md` enters the conversation once and persists for the session, while the `allowed-tools` grant clears on the next user message. Plus `background: false` with `context: fork` (Claude Code 2.1.218+).

### Changed
- **Advisory findings no longer gate `check_portability.py --strict`.** Warnings and errors gate exactly as before; `--strict-advisories` restores the old behaviour. Without this, the `thirdparty-import` change below could not take effect.
- **`thirdparty-import` no longer fires on Cowork's preinstalled Python stack.** numpy, pandas, requests, PyYAML, bs4, openpyxl, Pillow, matplotlib, python-docx and python-pptx are present in the image and installing from PyPI works, so the rule was a false positive on the most common case. Those ten import roots are now silent; anything outside them is advisory, noting per-run install cost and that egress is org-configurable.
- **`device_commit_files` is documented rather than banned.** Surfacing a file in the conversation and writing it onto the user's device are different outcomes, and only the first is what "present it" covers. Skill text should still describe the outcome and name no tool.
- **Runtime documentation re-pinned to Claude Code 2.1.222**, and stale model IDs in three `--help` epilogs, `references/schemas.md` and `harness/sessions/skill.yaml` refreshed to `claude-opus-5`.
- **[cowork-harness](https://github.com/yaniv-golan/cowork-harness) pin 1.16.0 → 1.19.0**, with the CI lint step now running `--strict --min-severity WARN` (without both flags it cannot fail on a WARN-class rule).

### Fixed
- **The generated skill was written where the user cannot see it, and on remote Cowork destroyed.** `SKILL.md` placed the working directory beside the skill directory, which is a read-only cache on plugin and marketplace installs; the agent then fell back to a session scratchpad that a remote session discards on exit, taking the new skill, its scripts and every eval result with it. It now requires a user-visible, writable location.
- **The eval viewer was never shut down.** `VIEWER_PID=$!` and `kill $VIEWER_PID` sat in separate shell invocations, so the variable was unset by the time it was read and the server kept running. The PID is now written to a file.
- **Listing overflow degrades per skill, not all at once.** There is no ~20-character collapse threshold: Claude ranks skills by recency-weighted usage and drops whole descriptions starting with the least recently used, so full and name-only entries coexist. A large description mostly costs itself. The `listing-collapse-risk` rule is renamed **`listing-desc-drop-risk`** — **update any tooling that keys on the old name.**
- **The listing budget is not a fixed ~8 KB.** It is `contextWindow × 4 × skillListingBudgetFraction` (default `0.01`), so a 1 M-context model gets ~40,000 characters. The 1,536-character per-entry cap is the `skillListingMaxDescChars` default rather than a constant. Also documents `skillOverrides` and the `/doctor`, `/skills` and `/context` diagnostics.
- **`allowed-tools` grants permission; it does not request it.** No permission prompt is triggered by populating `allowed-tools`, `hooks` or `shell`. It pre-approves tools for the invoking turn, clearing on the next user message, and the real gate is workspace trust, accepted once per folder. `disallowed-tools` is the denylist and is now documented as a skill field. Adds the MCP-sourced and shared-memory carve-outs where this frontmatter is ignored.
- **"In Cowork, cwd is a scratchpad the user can't see" is withdrawn.** cwd is the outputs mount. The instruction it justified — write to a stated path, then present it — is unchanged and still correct.
- **The Cowork eval-viewer step no longer contradicts the delivery rule** by suggesting a bare link, which cannot work on the remote lane.

## [0.8.0] - 2026-08-01

### Added
Expand All @@ -11,7 +37,7 @@ All notable changes to this project will be documented in this file.
- **A token-free scenario load gate in CI** (`cowork-harness record harness/scenarios/ --dry-run --quiet`). `lint` only warns on an unknown key, so a scenario that lints clean can still fail to load — a green lint is not evidence the suite runs.

### Changed
- **cowork-harness pin 1.2.0 → 1.16.0** across CI, `docs/DEVELOPMENT.md`, `harness/README.md`, and the shipped `references/environments.md` (which still advertised a 1.1.0 floor to every installed agent). `lane:` requires ≥ 1.14.0.
- **[cowork-harness](https://github.com/yaniv-golan/cowork-harness) pin 1.2.0 → 1.16.0** across CI, `docs/DEVELOPMENT.md`, `harness/README.md`, and the shipped `references/environments.md` (which still advertised a 1.1.0 floor to every installed agent). `lane:` requires ≥ 1.14.0.

### Fixed
- **The packaging step no longer names a file-delivery tool, or attributes one to the wrong surface.** `SKILL.md` told authors to present the packaged `.skill` via `present_files` "(Claude.ai)" — a hardcoded tool name plus an unsupported attribution; no public evidence indicates Claude.ai chat serves `present_files`, and the only confirmed server of it is Cowork's desktop-local lane, where it is MCP-namespaced as `mcp__cowork__present_files`. The step now follows the delivery guidance above and names no tool. The `[0.5.0]` entry below repeats the same misattribution and is left as shipped, since released entries are a historical record.
Expand All @@ -23,7 +49,7 @@ All notable changes to this project will be documented in this file.

### Added
- **`scripts/check_portability.py`** — a stdlib-only cross-runtime portability linter. Flags constructs that break on a skill's target runtime: over-cap `description`, subagent use (absent on Claude.ai), `claude` CLI use (absent on Claude.ai), browser/server assumptions (no display in Cowork/Claude.ai), and third-party Python imports in bundled scripts (Cowork's sandbox lacks them and can't `pip install`). `--target claude-code|claude-ai|cowork|all`, `--json`, `--strict`.
- **`harness/`** — a cowork-harness dogfood suite that regression-tests this skill under Claude Cowork's runtime, plus a token-free CI lane (`.github/workflows/harness.yml`).
- **`harness/`** — a [cowork-harness](https://github.com/yaniv-golan/cowork-harness) dogfood suite that regression-tests this skill under Claude Cowork's runtime, plus a token-free CI lane (`.github/workflows/harness.yml`).

## [0.6.0] - 2026-07-08

Expand Down
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ The skill that builds skills. Write a draft, run evals against a baseline, revie
| Best practices guide | — | 620+ line Anthropic patterns reference |
| Script vs. Instruct guidance | — | Decision framework for when to bundle scripts vs. use instructions |
| Cross-host portability | — | Validates full [agentskills.io](https://agentskills.io/specification) spec + Claude-specific fields |
| Claude Code runtime docs | — | v2.1.116 listing budget, collapse mode, truncation thresholds, live-reload behavior |
| Claude Code runtime docs | — | 2.1.222 listing budget + per-skill degradation order, permission semantics, truncation caps, live-reload behavior |
| Eval viewer in Cowork | Silent fail on submit | Copyable JSON textarea (fixed) |
| Description optimizer | Requires separate `ANTHROPIC_API_KEY`; drifts toward 1,024-char bloat | Uses your existing `claude` session; length-aware selection + plateau early-stop |
| Benchmarking script | Silent empty results | Fixed directory handling |
Expand All @@ -29,9 +29,9 @@ Anthropic ships a `skill-creator` plugin. It's good, but several parts are broke

- **Best practices guide included** — 620+ lines of patterns, structural templates, troubleshooting guide, and checklists extracted from Anthropic's [Complete Guide to Building Skills for Claude](https://resources.anthropic.com/hubfs/The-Complete-Guide-to-Building-Skill-for-Claude.pdf) and Thariq's [Lessons from Building Claude Code Skills](https://x.com/trq212/status/2024574133011673516). Includes a "Script vs. Instruct" decision framework for when to bundle pre-made scripts vs. keep logic as instructions — covering context window efficiency, reliability, and auditability. The built-in doesn't ship any of this.
- **Cross-host portability** — skills produced here validate against the full [agentskills.io](https://agentskills.io/specification) cross-host spec so they run on Claude, Gemini CLI, Cursor, OpenCode, and other hosts. Claude-specific fields (`when_to_use`, `model`, `paths`, `hooks`, etc.) are supported but flagged as optional extensions, never load-bearing.
- **Claude Code runtime docs** — documents v2.1.116 mechanics most skill authors hit the hard way: the ~8 KB total listing budget, the ~20-char collapse threshold that silently breaks *every* skill when any one gets too long, per-entry 1,536-char truncation, gitignore-syntax `paths:` matching (the docs say "glob" — they're wrong), and chokidar depth-2 live-reload.
- **Claude Code runtime docs** — documents the 2.1.222 mechanics most skill authors hit the hard way, each one verified against the shipping binary rather than inherited from a blog post: the shared listing budget (`contextWindow × 4 × skillListingBudgetFraction` — ~8 KB at 200 K, ~40 KB at 1 M) and the fact that overflow drops descriptions **per skill, least-recently-used first**, so full and name-only entries coexist; the 1,536-char per-entry default; that `allowed-tools` **grants** permission rather than requesting it, gated once per folder by workspace trust; the compaction budget that re-attaches only a skill's first 5,000 tokens; gitignore-syntax `paths:` matching (the docs say "glob" — they're wrong); and chokidar depth-2 live-reload. Where the public docs and the binary disagree, this says so and shows which one shipped.
- **Eval viewer actually works in Cowork** — the built-in silently fails: you write feedback, click "Submit All Reviews", it says "saved" — but nothing reaches Claude. This version reliably shows copyable JSON you can paste back.
- **Description optimizer doesn't bloat or crash** — the built-in calls the Anthropic SDK directly, requiring a separate `ANTHROPIC_API_KEY` most users don't have, and drifts toward 1,024-char descriptions that trigger the listing-collapse failure mode. This version uses `claude -p` (just works with your existing session), plus length-aware tie-break-shortest selection, plateau early-stop, and a tunable target length so descriptions stay tight.
- **Description optimizer doesn't bloat or crash** — the built-in calls the Anthropic SDK directly, requiring a separate `ANTHROPIC_API_KEY` most users don't have, and drifts toward 1,024-char descriptions that make a skill a top contributor to Claude's shared listing budget — and the first to lose its description when that budget overflows. This version uses `claude -p` (just works with your existing session), plus length-aware tie-break-shortest selection, plateau early-stop, and a tunable target length so descriptions stay tight.
- **Benchmarking script fixed** — the built-in's aggregation script silently produces empty results due to undocumented directory structure requirements. Fixed and tested.

See the [CHANGELOG](CHANGELOG.md) for the full list of fixes.
Expand Down
2 changes: 1 addition & 1 deletion VERSION
Original file line number Diff line number Diff line change
@@ -1 +1 @@
0.8.0
0.9.0
38 changes: 31 additions & 7 deletions docs/DEVELOPMENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,19 @@
python skill-creator-plus/skills/skill-creator-plus/scripts/quick_validate.py skill-creator-plus/skills/skill-creator-plus
# JSON output for tooling: add --json. Exit codes: 0 valid, 1 invalid, 2 reserved (stdlib-only now), 3 path not found.

# Cross-runtime portability lint (stdlib-only; --target claude-code|claude-ai|cowork|all; --strict to gate)
# Cross-runtime portability lint (stdlib-only; --target claude-code|claude-ai|cowork|all)
cd skill-creator-plus/skills/skill-creator-plus && python -m scripts.check_portability . --target all
# Rules: desc-over-hard-cap, listing-entry-truncation, listing-desc-drop-risk, subagent-dependency,
# claude-cli-dependency, browser-display-dependency, thirdparty-import (allowlisted against Cowork's
# preinstalled stack), delivery-tool-single-lane, delivery-conditional-deliverable,
# compaction-truncation-risk (SKILL.md over the 19,900-CHARACTER post-compaction cap — exact, not a
# heuristic: the runtime's "5,000 token" limit is Math.round(chars/4), binary-verified in 2.1.222).
# --strict gates on warnings/errors only; advisories report but never gate (add --strict-advisories
# to gate on those too). Exit codes: 0 no gating findings, 1 gated, 2 usage error, 3 path not found.
# Note this skill's own baseline: 4 findings (subagent / claude-CLI / browser deps, all Claude-Code-
# first by design, plus its own compaction-truncation-risk at ~2.0x cap) — so `--target all` is exit 0,
# and `--target cowork --strict` is exit 1 on the browser-display warning. A NEW rule id is the
# regression signal, not a non-empty finding list.

# Syntax-check all scripts
for f in skill-creator-plus/skills/skill-creator-plus/scripts/*.py; do python -c "import py_compile; py_compile.compile('$f', doraise=True)"; done
Expand All @@ -30,10 +41,11 @@ cd skill-creator-plus/skills/skill-creator-plus && python -m unittest discover -
# Merge analyst notes into a benchmark result
python -m scripts.aggregate_benchmark <dir> --notes notes.json # merge analyst notes

# cowork-harness static checks on the shipped skill (token-free, no Docker; needs cowork-harness >= 1.16.0)
# cowork-harness static checks on the shipped skill (token-free, no Docker; needs cowork-harness
# >= 1.19.0 — see `harness/README.md`)
cowork-harness lint-skill --strict skill-creator-plus/skills/skill-creator-plus
cowork-harness analyze-skill --strict skill-creator-plus/skills/skill-creator-plus
cowork-harness lint harness/scenarios/
cowork-harness lint --strict --min-severity WARN harness/scenarios/
# `lint` only WARNS on an unknown key, so a scenario that lints clean can still be unloadable;
# this runs the real loader (no token, no Docker, writes nothing) to prove the suite actually loads.
cowork-harness record harness/scenarios/ --dry-run --quiet
Expand All @@ -52,8 +64,8 @@ maintainer CI, not part of the user-facing skill workflow. Full instructions: `h
Desktop agent binary + a token — a maintainer step, not CI. Run `cowork-harness doctor --tier
container` first.
- **Install caveat:** `npx cowork-harness@<ver>` can silently serve a stale cached CLI. Verify
`cowork-harness --version` reports **1.16.x** (write-back detector landed in 1.1.0; the
`verify-cassettes` claude.com handshake fix landed in 1.2.0); the CI job pins `cowork-harness@1.16.0`
`cowork-harness --version` reports **1.19.x** (write-back detector landed in 1.1.0; the
`verify-cassettes` claude.com handshake fix landed in 1.2.0); the CI job pins `cowork-harness@1.19.0`
in an isolated prefix and asserts the version.

### Cassette privacy policy (public repo — BLOCKING)
Expand Down Expand Up @@ -93,11 +105,23 @@ Never edit version fields manually in plugin.json or SKILL.md.

## Release Process

`main` is protected — changes land via pull request — and **cutting a release is a maintainer step,
not self-serve.** Don't push to `main` or push a tag without the maintainer's explicit go-ahead for
that release. (Admin bypass is enabled, so a direct push from a maintainer account *succeeds* and
merely prints `Bypassed rule violations` — the protection won't stop an accidental release, so the
approval is the real gate. Automated agents: this includes you; committing locally is not approval
to push.)

```bash
./tools/bump-version.sh X.Y.Z
git commit -am "chore: bump version to X.Y.Z"
# open a PR and get it merged, then tag the merge commit on main:
git tag vX.Y.Z
git push origin main --tags
git push origin vX.Y.Z
```

CI creates a GitHub Release with a zip artifact automatically.
Push the **tag by name**, not `git push --tags`: `.github/workflows/release.yml` triggers on
`push: tags: 'v*'`, so the tag alone is what cuts the release — `main` itself is already updated by
the merge.

CI creates a GitHub Release with a zip artifact automatically once the tag arrives.
Loading
Loading