promote: doc-surface truth-in-docs refresh + drift guards (#93)

docs/EVIDENCE.md

@@ -151,15 +151,17 @@ where the claim is derived:
                      ▼
        ┌─────────────┼─────────────┐
        ▼             ▼             ▼
-   ┌───────┐   ┌──────────┐   ┌─────────┐
-   │ diff  │   │ inspect  │   │ archive │
-   └───────┘   └──────────┘   └─────────┘
+   ┌────────┐   ┌──────────┐   ┌──────┐
+   │ replay │   │ inspect  │   │ diff │
+   └────────┘   └──────────┘   └──────┘
 ```
 
 Generation flows forward only. Preservation is read-only after save.
-The arrow from artifact into `diff`/`inspect`/`archive` represents
+The arrows from the artifact into `replay`/`inspect`/`diff` represent
 consumers reading the preserved evidence — they do not re-derive the
-claim, they re-present it.
+claim, they re-present it. (These three are the read operations
+detailed in §9; the full consumer surface also includes `history` /
+`timeline` / `matrix` / `verify` / `export`.)
 
 ---
 

falsifyai/cli/render.py

@@ -1,19 +1,21 @@
-"""Plain-text terminal output for ``falsifyai run``, ``replay``, and ``diff``.
+"""Plain-text terminal output for the human-readable CLI surfaces.
 
-MVP scope: one row per case + a summary footer + the session id and store
-path so the user can find their saved artifact. No colors, no boxes, no
-JSON. Rich/colored output and ``--json`` land in Week 3 per
-[plan.md section 22.1](../../plan.md).
+Deliberately plain text -- no colors, no boxes, no JSON. The discipline is
+evidence density, not presentation: one row per case + a summary footer +
+the session id and store path so the user can find their saved artifact.
+This module renders ``run`` / ``replay`` (``render_session``), ``diff``
+(``render_diff``), ``verify`` (``render_verify`` / ``render_verify_all``),
+and ``export`` (``render_export`` / ``render_export_refusal``).
 
 The ``loaded_from`` parameter on ``render_session`` is what distinguishes
 the replay path: when set, an extra header line indicates the user is
 looking at a stored session rather than a fresh run. The detection of
-legacy artifacts (pre-PR-11, no CI evidence) lives in this module too --
+legacy artifacts (no CI evidence preserved) lives in this module too --
 the artifact shape, not the consumer, determines what's renderable.
 
-``render_diff`` is the diff CLI's render path (PR #14). It consumes a
-``DiffReport`` (consumer-side dataclass from cli/diff.py) and prints a
-compressed transition table: only rows where something changed are shown.
+``render_diff`` consumes a ``DiffReport`` (consumer-side dataclass from
+cli/diff.py) and prints a compressed transition table: only rows where
+something changed are shown.
 """
 
 import sys

falsifyai/replay/protocol.py

@@ -1,12 +1,15 @@
 """ReplayStore Protocol and its exception hierarchy.
 
 The Protocol is the contract every store implementation must satisfy. Two
-impls ship in PR #6: ``InMemoryStore`` (ephemeral, also useful as a test
-double) and ``SQLiteStore`` (default; file-backed at ``.falsifyai/replays.db``).
-
-MVP surface only — ``case_history`` and ``diff_sessions`` from plan.md
-section 18.1 are deferred to the Week 2 PRs that introduce ``falsifyai diff``
-and verdict-history queries.
+impls ship: ``InMemoryStore`` (ephemeral, also useful as a test double) and
+``SQLiteStore`` (default; file-backed at ``.falsifyai/replays.db``).
+
+The surface is deliberately small — ``save_session`` / ``load_session`` /
+``query_sessions`` / ``close``. Verdict-history and cross-session views
+(``history`` / ``timeline`` / ``matrix``) and ``diff`` are consumer-side
+operations layered on ``query_sessions`` and ``load_session``; they are not
+extra store methods. Keeping the store interface minimal is what lets new
+consumer commands ship without widening the persistence contract.
 """
 
 from collections.abc import Iterator

tests/meta/test_evidence_doc_freshness.py

@@ -5,7 +5,7 @@
 it. Stale claims there are higher-risk than stale claims anywhere else in the
 repo, because the document is the thing external readers trust.
 
-This is a **coarse** freshness guard, not a prose validator. It enforces two
+This is a **coarse** freshness guard, not a prose validator. It enforces three
 cheap, high-signal invariants and nothing semantic:
 
 1. Every ``Verdict`` enum member is named at least once in the document. This
@@ -18,10 +18,17 @@
 2. The document names the read-only preservation consumers at least once, so
    the producer/consumer boundary the artifact depends on stays documented as
    the command surface grows.
+3. Every leaf consumer box in the §3 lifecycle diagram names a real CLI
+   command. This catches the specific drift where the diagram keeps a node for
+   an operation that was never (or no longer) a command -- e.g. a stale
+   ``archive`` box surviving long after the consumer surface settled on
+   ``replay`` / ``inspect`` / ``diff``. The prose word "archive" is allowed to
+   appear elsewhere (retention discussion); only diagram *boxes* are checked.
 
 If this test fails, update ``docs/EVIDENCE.md`` -- do not weaken the assertion.
 """
 
+import re
 from pathlib import Path
 
 from falsifyai.verdict.models import Verdict
@@ -43,11 +50,54 @@
 )
 
 
+# Every CLI command FalsifyAI ships. A leaf box in the lifecycle diagram must
+# name one of these; "archive" (a past stale node) is intentionally absent.
+_ALL_COMMANDS = frozenset(
+    {
+        "run",
+        "replay",
+        "inspect",
+        "diff",
+        "history",
+        "timeline",
+        "matrix",
+        "minimize",
+        "export",
+        "verify",
+        "doctor",
+    }
+)
+
+# A box-drawing cell holding a single lowercase token: ``│ replay │``. The
+# inner whitespace is horizontal-only (``[^\S\n]`` not ``\s``) so a token never
+# pairs an opening ``│`` on one line with a closing ``│`` on the next -- that
+# cross-line greediness would otherwise capture a trailing label like the
+# ``│   evidence`` annotation beside the artifact box. Matched non-overlapping
+# so a one-line ``│ replay │  │ inspect │  │ diff │`` row yields all three.
+# Multi-word boxes ("the durable record") never match: ``[a-z_]+`` stops at the
+# first space and the following ``│`` is not adjacent.
+_DIAGRAM_BOX = re.compile(r"│[^\S\n]*([a-z_]+)[^\S\n]*│")
+
+
 def _doc_text() -> str:
     assert _EVIDENCE_DOC.exists(), f"evidence contract doc missing: {_EVIDENCE_DOC}"
     return _EVIDENCE_DOC.read_text(encoding="utf-8")
 
 
+def _lifecycle_diagram() -> str:
+    """The fenced ASCII block in §3 -- the one containing the artifact node."""
+    blocks = _doc_text().split("```")
+    # Fenced content sits at odd indices once split on the fence marker.
+    for block in blocks[1::2]:
+        if "REPLAY ARTIFACT" in block:
+            return block
+    raise AssertionError(
+        "lifecycle diagram not found in docs/EVIDENCE.md: no fenced block names "
+        "'REPLAY ARTIFACT'. The §3 diagram is the artifact contract's visual "
+        "anchor -- if it was removed or renamed, update this guard deliberately."
+    )
+
+
 def test_every_verdict_appears_in_evidence_doc() -> None:
     text = _doc_text()
     missing = [v.name for v in Verdict if v.name not in text]
@@ -67,3 +117,24 @@ def test_evidence_doc_names_readonly_consumers() -> None:
         f"The producer/consumer boundary is load-bearing protocol semantics; "
         f"keep the consumer surface documented."
     )
+
+
+def test_lifecycle_diagram_leaf_consumers_are_real_commands() -> None:
+    # Leaf consumer boxes fan out *below* the artifact node; restrict the scan
+    # to that region so upstream pipeline boxes ("spec", etc.) are not checked.
+    _, _, leaf_region = _lifecycle_diagram().partition("REPLAY ARTIFACT")
+    boxes = _DIAGRAM_BOX.findall(leaf_region)
+
+    assert boxes, (
+        "no leaf consumer boxes found below the REPLAY ARTIFACT node in the "
+        "docs/EVIDENCE.md lifecycle diagram. The diagram must show the consumers "
+        "reading the preserved artifact; if its shape changed, update this guard."
+    )
+
+    unknown = sorted({b for b in boxes if b not in _ALL_COMMANDS})
+    assert not unknown, (
+        f"lifecycle diagram in docs/EVIDENCE.md names non-command leaf nodes: "
+        f"{unknown}. Every consumer box below the artifact node must be a real "
+        f"CLI command -- this is the guard that locks out the stale 'archive' "
+        f"node. Known commands: {sorted(_ALL_COMMANDS)}."
+    )

tests/meta/test_readme_command_surface.py

@@ -0,0 +1,52 @@
+"""Stale-doc tripwire: every CLI subcommand is referenced in the README.
+
+The README is the public face of the CLI; its command reference is what a new
+user reads to learn the surface. The failure mode this locks out is silent:
+a new subcommand is wired into ``build_parser`` and ships, but the README is
+never updated to mention it, so the documented surface drifts behind the real
+one.
+
+This is a **coarse** guard, deliberately. It asserts only that the literal
+token ``falsifyai <command>`` appears somewhere in ``README.md`` for every
+registered subcommand. It does not validate that the surrounding prose is
+correct, complete, or current -- that is review's job. Catching the
+"command exists but is undocumented" drift cheaply is the whole ambition.
+
+The subcommand list is read from ``build_parser`` itself, not hardcoded, so
+the guard tracks the real surface automatically as commands are added.
+
+If this test fails, add the command to ``README.md`` -- do not weaken the
+assertion.
+"""
+
+import argparse
+from pathlib import Path
+
+from falsifyai.cli.main import build_parser
+
+_README = Path(__file__).resolve().parents[2] / "README.md"
+
+
+def _subcommand_names() -> list[str]:
+    parser = build_parser()
+    for action in parser._actions:
+        if isinstance(action, argparse._SubParsersAction):
+            return sorted(action.choices)
+    raise AssertionError(
+        "build_parser() exposes no subparsers; the CLI dispatch shape changed. "
+        "Update this guard to match how subcommands are now registered."
+    )
+
+
+def test_every_subcommand_is_referenced_in_readme() -> None:
+    readme = _README.read_text(encoding="utf-8")
+    commands = _subcommand_names()
+    assert commands, "no subcommands discovered from build_parser()"
+
+    missing = [c for c in commands if f"falsifyai {c}" not in readme]
+    assert not missing, (
+        f"README.md does not reference these CLI subcommands: {missing}. "
+        f"A command is registered in build_parser() but the README command "
+        f"reference never mentions ``falsifyai <command>`` for it. Document the "
+        f"command before shipping."
+    )

tests/meta/test_release_gate_coverage_sync.py

@@ -0,0 +1,62 @@
+"""Stale-config tripwire: the release gate's coverage floor matches CI's.
+
+``scripts/release_gate.py`` runs ``pytest --cov-fail-under=COVERAGE_FLOOR`` and
+``.github/workflows/ci.yml`` runs ``pytest ... --cov-fail-under=N``. These are
+two independently hardcoded copies of the same number, kept in step today only
+by a hand-maintained "keep in sync" comment in the CI workflow. That is exactly
+the kind of duplicated constant that drifts silently: bump one, forget the
+other, and the release tooling no longer enforces what CI enforces (or vice
+versa).
+
+This guard makes the drift fail loudly. It parses both numbers and asserts
+equality -- nothing more. It does not opine on what the floor *should* be; it
+only refuses to let the two copies disagree.
+
+If this test fails, reconcile ``COVERAGE_FLOOR`` in scripts/release_gate.py with
+``--cov-fail-under`` in .github/workflows/ci.yml so they are equal again.
+"""
+
+import re
+from pathlib import Path
+
+_ROOT = Path(__file__).resolve().parents[2]
+_RELEASE_GATE = _ROOT / "scripts" / "release_gate.py"
+_CI_WORKFLOW = _ROOT / ".github" / "workflows" / "ci.yml"
+
+
+def _release_gate_floor() -> int:
+    text = _RELEASE_GATE.read_text(encoding="utf-8")
+    match = re.search(r"^COVERAGE_FLOOR\s*=\s*(\d+)", text, re.MULTILINE)
+    assert match, (
+        "could not find a top-level ``COVERAGE_FLOOR = <int>`` in "
+        "scripts/release_gate.py; the release-gate coverage constant moved or "
+        "was renamed. Update this guard to read it from its new home."
+    )
+    return int(match.group(1))
+
+
+def _ci_floor() -> int:
+    text = _CI_WORKFLOW.read_text(encoding="utf-8")
+    matches = re.findall(r"--cov-fail-under=(\d+)", text)
+    assert matches, (
+        "could not find ``--cov-fail-under=<int>`` in .github/workflows/ci.yml; "
+        "the CI coverage gate moved or was renamed. Update this guard to read it "
+        "from its new home."
+    )
+    distinct = set(matches)
+    assert len(distinct) == 1, (
+        f".github/workflows/ci.yml uses multiple --cov-fail-under values "
+        f"{sorted(distinct)}; the CI coverage floor is no longer a single number, "
+        f"so 'matches release gate' is ambiguous. Reconcile CI first."
+    )
+    return int(matches[0])
+
+
+def test_release_gate_coverage_floor_matches_ci() -> None:
+    gate = _release_gate_floor()
+    ci = _ci_floor()
+    assert gate == ci, (
+        f"coverage floor drift: scripts/release_gate.py COVERAGE_FLOOR={gate} but "
+        f".github/workflows/ci.yml --cov-fail-under={ci}. Release tooling and CI "
+        f"must enforce the same coverage floor. Reconcile the two constants."
+    )

tests/unit/test_bundle_writer.py

@@ -1,8 +1,7 @@
 """Unit tests for ``falsifyai.bundle.writer`` (PR-32, Phase B).
 
-RED phase: imports ``BundleManifest``, ``BundleFileEntry``, ``write_bundle``,
-and the schema constants which do not exist yet. After GREEN, all tests
-should pass.
+Exercises ``BundleManifest``, ``BundleFileEntry``, ``write_bundle``, and the
+schema constants.
 
 Tests grouped by acceptance criterion from PR-32 plan §5:
 

tests/unit/test_cli_diff_sharpening.py

@@ -1,12 +1,8 @@
-"""RED-phase tests for PR-28 diff-sharpening: --strict, --show-timeline, exit 6.
+"""Tests for diff-sharpening: --strict, --show-timeline, exit 6.
 
 All tests in this file import ``STRICT_CONFIDENCE_DROP_THRESHOLD`` and
-``LOW_FALSIFIABILITY_THRESHOLD`` from ``falsifyai.cli.diff``.  Those constants
-do not exist yet, so the entire module fails to collect in RED phase.  Once
-GREEN adds the constants and the flag logic, every test here should pass.
-
-Tests that already document a true property (introspection, exit-code parity
-with no flags) will pass immediately after import is unblocked.
+``LOW_FALSIFIABILITY_THRESHOLD`` from ``falsifyai.cli.diff`` and exercise the
+flag logic those constants drive, alongside the exit-code parity with no flags.
 
 Covers acceptance criteria §5 of dev_notes/plans/PR-28-diff-sharpening.md:
   - Named constants present with correct values
@@ -24,8 +20,8 @@
 
 from falsifyai.cli import diff as diff_module
 from falsifyai.cli.diff import (
-    LOW_FALSIFIABILITY_THRESHOLD,  # RED: ImportError until GREEN adds this
-    STRICT_CONFIDENCE_DROP_THRESHOLD,  # RED: ImportError until GREEN adds this
+    LOW_FALSIFIABILITY_THRESHOLD,
+    STRICT_CONFIDENCE_DROP_THRESHOLD,
 )
 from falsifyai.replay.in_memory_store import InMemoryStore
 from falsifyai.replay.models import CaseResult, ReplayArtifact, SessionVerdict

tests/unit/test_cli_history.py

@@ -4,7 +4,7 @@
 Decisions A1/B1/D1/E1/F1/G1 + X1/Y1/Z1 from
 ``dev_notes/plans/PR-24-falsifyai-history-cli.md``.
 
-RED phase: these tests describe the public surface before it exists.
+These tests exercise the public surface of ``falsifyai.cli.history``.
 """
 
 import argparse

tests/unit/test_cli_inspect.py

@@ -5,8 +5,8 @@
 verdict. Tests below enforce that rule and the decisions A-F from
 `dev_notes/plans/PR-19-falsifyai-inspect-cli.md`.
 
-RED phase: these tests describe the public surface before it exists. They
-are expected to fail until cli/inspect.py is implemented.
+These tests exercise the public surface of ``cli/inspect.py`` and enforce the
+never-re-derive-a-verdict rule.
 """
 
 import argparse

tests/unit/test_cli_invocation_model.py

@@ -1,7 +1,7 @@
 """Unit tests for ``CliInvocation`` and the ``ReplayArtifact.cli_invocation`` field (PR-35).
 
-RED phase: imports ``CliInvocation`` from ``falsifyai.replay.models`` which
-does not exist yet. After GREEN, all tests should pass.
+Exercises ``CliInvocation`` from ``falsifyai.replay.models`` and the
+``ReplayArtifact.cli_invocation`` field.
 
 Capture contract (per PR-35 plan §1):
 - ``CliInvocation`` is a frozen dataclass with exactly two fields:

tests/unit/test_cli_run_captures_invocation.py

@@ -1,7 +1,7 @@
 """Unit tests for ``cmd_run``'s ``cli_invocation`` capture (PR-35, Phase C).
 
-RED phase: the capture helper and the field-attachment do not exist yet.
-After GREEN, all tests should pass.
+Exercises the ``cli_invocation`` capture helper and its field-attachment in
+``cmd_run``.
 
 Capture contract (per PR-35 plan §1):
 - Capture at entry into ``cmd_run`` (single capture point)

tests/unit/test_cli_store_lifecycle.py

@@ -7,11 +7,14 @@
 later unlink of the database file — is the failure this locks out.
 
 This is a single parametrized harness rather than a close assertion duplicated
-across eight command test files. ``run`` (and ``minimize``) are intentionally
-absent: they are *producers* that orchestrate generation -> interpretation ->
-preservation, not read-only consumers, and their store lifecycle is asserted in
-their own test file where the execution stack is already mocked. Folding them in
-here would blur the producer/consumer boundary the harness exists to protect.
+across eight command test files. ``run`` is intentionally absent: it is a
+*producer* that orchestrates generation -> interpretation -> preservation, not a
+read-only consumer, and its store lifecycle is asserted in its own test file
+where the execution stack is already mocked. ``minimize`` is also absent for a
+different reason -- it never opens a ``ReplayStore`` at all (it runs an
+in-memory search and persists nothing), so there is no store lifecycle to pin.
+Folding either in here would blur the producer/consumer boundary the harness
+exists to protect.
 
 Two situations matter, for every consumer: