Skip to content

Latest commit

 

History

History
170 lines (130 loc) · 10.4 KB

documentation-grooming.md

File metadata and controls

170 lines (130 loc) · 10.4 KB

Documentation grooming

Grooming keeps documentation accurate and understandable. Automated checks run in continuous integration (CI). A human or independent agent also reviews whether a reader can understand the page. Neither part replaces the other.

Use the Documentation standards for audiences, plain language, task structure, evidence, and review authority. Apply them to changed sections and improve older pages incrementally; a grooming pass does not certify the whole documentation set.

Why this exists

On 2026-08-07 one writer read the docset cold and found twelve defects. None required running code or special access. Several were two statements contradicting each other on the same page. One sent every new user to http://localhost:18790 on getting-started-release.md — the highest-traffic page in the docset — while GatewayBindAddress.LoopbackListenUrl declares http://localhost:5005.

Every one of them survived grooming, because grooming was entirely a human read. A human read reliably catches tone, structure and readability, and reliably glides straight past a wrong port number in the third code fence of a long page. Issue #2865 added the mechanical half.

What is automated

scripts/repo/docs-lint.ps1. One command, runnable locally and in CI:

pwsh -NoProfile -File scripts/repo/docs-lint.ps1

Exit code 0 is clean, 1 is findings, 2 is a usage error (including the lint's own anti-vacuity floor — it refuses to certify a docset it barely read). Add -AsJson for a machine-readable result on pure stdout, or -Rule <name> to run one rule.

CI runs it from .github/workflows/docs-lint.yml on any change to docs/** or to the lint's own inputs. That second trigger matters: a change to the fact registry or the allow-list can silently disarm the gate, so it is gated too.

Rule What it asserts Defect it exists to prevent
literal-drift A port or config key instructed inside a fenced code block in docs/** must appear somewhere in src/**. A literal that lives only in docs is stale by definition. localhost:18790 on getting-started-release.md; the BotNexus.Cron.Jobs config key the binder never read.
intra-page-contradiction For a registry of high-value facts, one page states at most one distinct value. Hard failure, not a warning — a reader cannot tell which half is true, so both halves are worthless. cron-and-scheduling.md giving tickIntervalSeconds as 60 in a table and 10 in a diagram.
legacy-marker A "legacy" / "deprecated" / "non-functional" / "do not copy" disclosure inside a how-to section must appear above the code sample, not below it. Historical: the LlmProviderBase sample in extension-development.md, disclosed as non-functional only after the fence (removed in #2862).
docs-vs-source trigger A PR touching an extension manifest, a provider interface or a controller route must change a docs/ page or state no-docs-impact in the body. Extension layout drift; IApiProvider vs LlmProviderBase; /api/exchanges/budget.

The first three rules check page content. The fourth is implemented by the docs-impact job in its own workflow, .github/workflows/docs-impact.yml, and the Documentation impact item in .github/pull_request_template.md.

It is a separate workflow because the two want opposite triggers. Docs lint runs on paths: docs/**, deliberately, so a docs-only PR that starts no test workflow is still gated. The docs-vs-source trigger needs the complement of that — the source PRs that change no documentation at all — so it carries no paths filter and exits early when nothing documentation-sensitive was touched.

While the two shared a file the fourth rule could only run on PRs that had already changed documentation, where its first satisfying branch is "this PR also changes documentation". It was green from the day it landed without once being able to fail, which is indistinguishable from a working gate.

DocsImpactWorkflowArchitectureTests pins the repaired contract: the workflow remains unfiltered at the pull-request level, reruns when the PR body is edited, retains all three sensitive-source predicates, and preserves the docs-change, explicit-justification, and failure outcomes. A synthetic copy of the original docs-only trigger proves the regression detector rejects the broken shape. These checks protect the workflow mechanics; they do not decide whether the resulting documentation is accurate or understandable. That still requires the review described below.

Documentation-site PR check

The docs-site job in .github/workflows/docs-site.yml runs on every pull request, without path filters. It checks the candidate merge checkout using read-only repository access and does not configure, upload, or deploy GitHub Pages. The separate deploy-docs.yml workflow still publishes the site after relevant changes merge to main.

The classifier, scripts/repo/Get-DocsSiteImpact.ps1, compares the PR head with its merge base. It includes additions, deletions, and both sides of a rename. Changes only on the base branch do not count as PR changes. Relevant inputs are docs/** (including VitePress configuration and assets), package.json, package-lock.json, the site and deployment workflows, and the classifier and its tests. Missing commit inputs, missing Git objects, or failed merge-base or diff operations fail the check; they do not become a successful skip.

For relevant changes, the job uses Node.js 20 and runs the same commands as deployment:

npm ci
npm run docs:build

For unrelated or empty changes, the job records an explicit successful no-operation result (noop). The stable check name is docs-site, so unrelated PRs still receive a completed check. Repository administrators must decide whether to make it required after reviewing live CI results. Adding this workflow does not change branch protection or grant merge authority.

Keep the three checks distinct:

Check What it proves What it does not prove
docs-site Relevant site inputs install and build; VitePress rejects dead relative links. Documentation is accurate, understandable, or sufficient for a source change.
docs-impact A sensitive source change includes documentation or an explicit no-docs-impact justification. The site builds or the justification is correct.
Documentation lint The configured literal-drift, contradiction, and legacy-marker rules pass. All factual claims are correct or a reader can complete the task.

DocsSiteWorkflowArchitectureTests runs the standalone contract and disposable Git fixtures in the remote core suite. For a focused classifier/workflow check that starts no gateway, run:

pwsh -NoProfile -File tests/architecture/BotNexus.Architecture.Tests/DocsSiteFocusedTests.ps1

The fixtures cover relevant and unrelated changes, deletion, rename, base drift, empty diffs, and fail-closed input and Git errors. This focused check is not a substitute for the repository's required validation of code changes.

Tuning the rules

  • Fact registry — scripts/repo/docs-lint-facts.json. Each entry carries an id, the defect it prevents, and a regex with a named value group. Add a fact when a page states a documented default, a port, an enum value list or a config path that a future edit could contradict.
  • Allow-list — scripts/repo/docs-lint-allow.json. For literals that are legitimately absent from src/: Jaeger's 16686, Ollama's 11434, an OTLP collector's 4317. Every entry must carry a written justification. An unexplained suppression is how a gate gets hollowed out one entry at a time.
  • A line that sets a value (botnexus config set gateway.listenUrl http://localhost:8080, export BotNexus__...=) is a demonstration, not an assertion about the system, and is excluded from both rule 1 and rule 2. The motivating defect was a browse instruction with no assignment, so it is still caught.

Proving the lint still works

A lint that cannot fail on the defect that motivated it is decoration. DocsLintScriptTests (in BotNexus.Architecture.Tests) pins each rule against a fixture reproducing its motivating defect and against the corrected form, so the gate can neither pass vacuously nor flag everything. If you change a detector, re-prove it by mutation: inject the known-bad literal and the known contradiction, confirm the gate goes red naming that rule, then restore and confirm the baseline is green again.

What needs understanding review

The lint does not decide whether a page is useful. A human or independent agent must read the changed page as its intended audience, using the author and reviewer checklist. Check whether the reader can identify the goal, understand the prerequisites, follow the steps, recognize the result, and find safe help when something fails. Also check whether the page belongs in the chosen guide and whether it contradicts its linked references.

Trace factual claims to source and record that evidence separately from steps actually executed. Use the existing PR Validation section and identify the reviewed revision, reviewer, findings, and remaining gaps. Follow the standard's review and authority boundaries for higher-risk instructions, protected pages, and decisions that require a human.

Link checking is separate: npm run docs:build (VitePress) fails on a dead relative link. The docs-site PR check runs it before merge for relevant inputs; deploy-docs.yml runs it again before publishing. Neither a build pass nor source inspection proves an end-to-end installation works.

The release walk

Every release, walk getting-started-release.md end to end on a clean VM.

Not a read — a walk. Download what it says to download, run what it says to run, open what it says to open. It is the page where an error costs a user permanently, and it is exactly where this batch's worst defect lived. No lint catches "the instructions are individually true but do not add up to a working install".

Related