Skip to content

23 commands document flags they reject: the CLI reference describes invocations that exit 2 #3465

Description

@chernistry

docs/reference/cli-reference.md documents flags that the commands do not accept. A documented flag the parser rejects exits 2 with No such option before the command body runs, which reads to a user exactly like the command being missing.

Measured against main by resolving every #### \bernstein `section through the realcliobject and comparing its flag table to the command's actualparams`: 23 commands carry at least one documented flag that does not exist.

Command Documented, non-existent
chaos --compare --dataset --instance --out --reliability --sample --save --scheduler --stub-signer --subset --suite --workdir
scaffold --endpoint --endpoint-api-key-env --endpoint-engine --endpoint-model --endpoint-timeout --json --role
evolve --budget --github --interval --max-cycles --yes
debug-bundle --include-logs --include-secrets --out --workdir
dep-impact --from --package --to
pr --body --head --repo
graph --device --engine --language
man-pages --out --section
run --approval --merge
explain --format
mcp --server
plan generate --out
cost policy preflight --workdir

The remaining rows are in the same shape; regenerate the list from the probe rather than trusting this table after any fix lands.

dep-impact deserves a note: its documented --package is presented as required, so the documented invocation is not merely missing an option — the only invocation the reference describes cannot run at all.

Why this is a gate, not a docs chore

Fixing the tables once leaves nothing preventing the next drift. tests/unit/test_cli_command_registration.py already asserts that every documented command resolves, and (as of the change that filed this) that the two commands registered there accept every flag their table documents. The mechanism exists; what is missing is the reach.

Widening it in one step turns the build red on 23 pre-existing rows, which is why that PR scoped its assertion to the two commands it made reachable and filed this instead.

Acceptance criteria

  • For every command in the reference, either the flag table matches the command's accepted options, or the command is listed in an explicit, named exemption set with a reason.
  • The exemption set only shrinks: adding an entry requires touching a file whose whole purpose is to record what is still broken, so growth is visible in review.
  • The gate covers all documented commands, not a hand-picked subset, and runs in the Repo hygiene job.
  • A flag added to a command without a doc row, and a doc row for a flag that does not exist, both fail — the check is bidirectional or its one-directional limit is stated.
  • Tests fail before and pass after for at least one command in each direction — demonstrate, do not assert.
  • Where the docs describe a flag that should exist (dep-impact --package is the candidate), the issue is resolved by deciding which side is wrong and saying so in the PR, not by silently deleting the row.

Effort

M–L. Each table is mechanical; the judgement is per-command — a ghost flag can mean the docs invented it, or that a real capability was removed and the docs are the only surviving record of it. That distinction is why this is not a find-and-replace.

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions