test: pin CLAUDE.md and AGENTS.md identical - #906
Merged
Conversation
AGENTS.md was missing the whole mypy iteration notes subsection and nothing caught the drift. Define the shared part as every CLAUDE.md line not ending in a claude-only marker, copy the missing subsection over, and guard the invariant the way the other project-structure guards do. Both files leave the CI docs-only fast path: a test now asserts on their content, so a change to either has to run the gate.
Review found the guard could stay green while the drift it exists to catch recurred. The mypy notes anchor pinned only the heading, so emptying the subsection passed, and any line could leave the shared set just by carrying the marker, with no second signal anywhere. Pin the tagged lines to an explicit prefix allowlist (widening it is now a visible diff), require the subsection to carry a body, and pin both files in the paths-ignore protected set so a refactor of the string literals cannot silently reopen the docs-only fast path. Read bytes rather than text so a lost trailing newline or a CRLF rewrite is not invisible, and say in the docs that only content meaningless outside Claude Code may stay out of AGENTS.md.
The marker, the prefix allowlist and the subsection anchor all existed to keep one bullet out of AGENTS.md, and they cost about a hundred lines of guard to do it. The bullet is not worth that: AGENTS.md already points at .claude/agents/red-agent.md elsewhere, so a line saying when those files auto-load is consistent with what it already carries. Both files are now byte-identical and the guard is one assert. Reading bytes stays, so a CRLF rewrite or a lost trailing newline cannot compare equal, and so does the existence assert.
TKaltofen
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #902.
AGENTS.mdwas missing the entire### mypy iteration notessubsection, so an agent reading it got worse advice and could burn a tox cycle on a straybuild/directory or on amypy mloda/run that misses test-tree errors. Nothing checked the two files agreed.The two files are now identical, and one assert says so
tests/test_agent_docs_sync.pycompares them and prints a unified diff on failure. That is the whole guard.The issue asked for a defined shared part rather than a byte-for-byte match, to keep the
**Session root**bullet about.claude/agents/auto-loading out ofAGENTS.md. That carve-out was built first (aclaude-onlymarker, a prefix allowlist bounding it, an anchor pinning that the mypy subsection could not be tagged away) and then dropped: about a hundred lines of guard to exempt a single line.AGENTS.mdalready points at.claude/agents/red-agent.mdin its own Self-Improvement section, so a line saying when those files auto-load is consistent with what it already carries. Identity is easier to hold in your head, and there is nothing left to defeat.Two details are kept because each is one line and load-bearing:
read_text()applies universal-newline translation and would let a CRLF rewrite or a lost trailing newline compare equal, both of whichdiff CLAUDE.md AGENTS.mdreportsFileNotFoundErrorBoth files leave the docs-only CI fast path
A test now asserts on their content, so a change to either has to run the gate. Leaving them in
paths-ignorewould let drift land with the guard never executing, surfacing later as a red gate on an unrelated contributor's PR, which is the worst failure mode for a drift guard.test_protected_set_is_not_stalegained both files, because the coupling rests on the string literals in the new test surviving verbatim and a harmless-looking refactor of them would otherwise reopen the fast path silently.Cost, accepted deliberately: GitHub path filters are workflow-level, so a typo fix in either file now also triggers the notebooks job (30s in practice) on top of the test matrix. The remaining fast-path entries (
CODE_OF_CONDUCT.md,NOTICE.md,README.md,SECURITY.md) are unchanged.Also fixes a stale path in the
ci.yamlheader comment: the guard is attests/test_ci_paths_ignore.py, not undertests/test_documentation/.toxgreen: 7536 passed, 170 skipped, plus format, lint, licenses, strict mypy and bandit.