sec: hash-pin every dependency install and add coverage-guided fuzzing

[cdeust]

Closes all 22 open code-scanning alerts: 21 Pinned-Dependencies + 1 Fuzzing. There were no Dependabot alerts and no known-CVE vulnerabilities open — every finding was OpenSSF Scorecard.

Closes #203, and goes past it: that issue covers the three Dockerfiles (11 alerts). The other 10 pinning alerts (ci.yml ×7, release.yml ×2, scripts/setup.sh) and the Fuzzing alert had no issue at all, and are fixed here rather than filed for later.

The core point

An exact version is not a pin. ruff==0.15.20 and torch==2.11.0 both counted as unpinned, and Scorecard is right: a version names a release, not the bytes the index serves for it today. Only a hash pins the artifact.

pip install --require-hashes is all-or-nothing — one hash means every requirement including transitive ones needs one — so it needs a resolved lock. uv.lock becomes the single source of truth.

Mechanism	What it does
scripts/generate_pip_constraints.py	Exports one hashed file per call site into requirements/. Refuses an export that is empty or carries an unhashed requirement. --check is a blocking Lint step, so a lock change that isn't re-exported fails there, not at install time.
[dependency-groups]	CI's own tools (ruff, pyright, build, hatchling, atheris) are locked instead of restated as bare version strings across two workflow files.
[[tool.uv.index]] + [tool.uv.sources]	Binds torch to the PyTorch CPU index on Linux.

The torch finding was worse than "unpinned"

The containers passed --index-url https://download.pytorch.org/whl/cpu at the call site. So uv.lock recorded PyPI's torch and its PyPI hashes, while the image installed a different artifact from a different index. No source of truth could produce a hash for what was actually installed — which is why --require-hashes wasn't reachable before.

The lock now carries torch 2.13.0+cpu with 22 hashes, and resolution drops 18 nvidia/cuda packages plus triton. torch is named in a container dependency-group only so the source can bind to it — PEP 735 groups aren't published, so nothing changes for anyone installing hypermnesia-mcp from PyPI.

Two findings couldn't be pinned — they had to stop being what they were

• curl https://deb.nodesource.com/setup_22.x | bash — an unreviewed remote script executed as root at build time, and there is no hash to check a pipe against. Replaced by what that script does: fetch the signing key, register the signed apt source, install the signed package. curl now feeds gpg --dearmor, which executes nothing.
• npm install -g @anthropic-ai/claude-code — unversioned, so the image tracked whatever the registry served that minute. Now npm ci against a committed lockfile, which is also the only form Scorecard accepts (an exact version on the command line is not enough) and which records a sha512 integrity hash per transitive package.

Accepted forms, from the checker's source

I read isUnpinnedPipInstall rather than inferring it. A bare pip install --no-deps . is unpinned — . is neither a flag nor a .whl, so it sets hasAdditionalArgs; isPinnedEditableSource is consulted only for -e. Hence: --require-hashes, or -e <local> --no-deps, or a .whl path. The root image therefore builds a wheel — an editable install there would leave a .pth pointing at /build, which the runtime stage never copies, and the venv would arrive broken.

Fuzzing — and what it found

Two harnesses over pure parsers that read untrusted text (§13.1 D2): the hand-rolled YAML frontmatter parser and the wiki source-path canonicaliser. ClusterFuzzLite runs a 120s batch on PRs (blocking) and a longer scheduled run (non-blocking — a fuzzer left running eventually finds something, and holding the merge queue hostage to an unrelated input makes the check ignored within a week).

Writing the path harness found a live bug. normalize_source_path stripped ./ in a loop, then / exactly once. Removing the slashes can expose a ./ the loop already walked past:

.//./x   ->  ./x     # still carries the prefix the function exists to remove
/./z     ->  ./z

The result was not idempotent, and extract_document_paths dedupes on it — so one document reachable by two spellings counted as two. Fixed by iterating to a fixed point. The four reproducers are committed as corpus inputs and fail on the pre-fix code (verified by stashing the fix and re-running).

fuzz/replay_corpus.py replays every corpus input with no atheris, so the properties run in the ordinary pytest suite on every platform. Atheris publishes manylinux x86_64 wheels for cpython 3.12–3.14 and nothing else — a property only one CI job can run is one that rots.

Also fixed (§14, seen en route)

• docker/Dockerfile could not build at all — it copied /usr/local/lib/python3.12/site-packages against a python:3.14 base, a path absent from both stages. Invisible because no CI job built this image. Now installs into a version-free venv (the rule the root Dockerfile already documents).
• scripts/setup.sh reported success over any failure — the install ended in 2>/dev/null, which is exactly where a resolution failure, hash mismatch and network error appear, and it printed "Python packages installed" regardless. Exit status is now checked.
• .gitattributes marks fuzz/corpus/** binary — EOL normalisation would have rewritten the CRLF seed on checkout and deleted the case it exists to cover.

---

Unblocking pass (2026-07-29)

The first push left Test (Python 3.10) red. Root-caused to the lock, not
to this PR's design, and fixed there (§6).

The 3.10 blocker

ERROR: Could not find a version that satisfies the requirement onnxruntime==1.24.3
       (from versions: ..., 1.23.0, 1.23.1, 1.23.2)
ERROR: No matching distribution found for onnxruntime==1.24.3

uv.lock recorded onnxruntime 1.24.3 for the python_full_version < '3.11'
fork, and onnxruntime 1.24.x publishes no cp310 artifact and no sdist —
1.24.3 ships 24 files whose lowest interpreter tag is cp311, and 1.24.0
declares no Requires-Python at all, which is why uv accepted it there. A lock
entry for the 3.10 fork with no 3.10 artifact is broken by inspection.

The defect is pre-existing on main (git show origin/main:uv.lock carries
the same entry). It stayed invisible because main installs from
pyproject.toml, where pip re-resolves onnxruntime down to 1.23.2 on 3.10 by
itself. A hash-pinned install cannot re-resolve — so making the lock the install
source is exactly what turned a latent lock defect into a visible failure. That
is the change working as intended, not a regression it introduced.

Fixed with a [tool.uv] constraint-dependencies entry rather than a project
dependency: constraints steer only our resolution and are never published in the
wheel metadata, so anyone installing hypermnesia-mcp on 3.11+ still resolves
the current onnxruntime. Declared before the [[tool.uv.index]] /
[tool.uv.sources] tables so it never re-opens a tool.uv super-table they
created implicitly.

Regenerated with uv 0.11.3 — the exact version ci.yml pins via
astral-sh/setup-uv, because the blocking Lint step byte-compares a uv export
result and any other uv version fails it.

Lock delta is minimal and forkwise: onnxruntime 1.24.3 -> 1.23.2 on the
<3.11 fork only (1.28.0 on >=3.11 untouched), plus its three transitive
deps (coloredlogs, humanfriendly, pyreadline3). 9 of 13 requirements files
changed.

onnxruntime was the only package with this defect — proven by resolving
every requirements file against every Python its consumers use, below.

§14 defects found and fixed in this pass

• .bestpractices.json was committed carrying four unresolved merge-conflict
  blocks (at c090278), leaving it invalid JSON — and it is transcribed into
  the OpenSSF Best Practices questionnaire, so an unparseable copy is a broken
  consumer, not a stale number. It had passed the doc-claim gate, CodeQL and 18
  green checks. Root cause (§6): .bestpractices.json is one of that gate's
  own SCANNED_FILES, but every check it runs is a claim regex — a regex matches
  the first side of a conflict and never looks at the file's structure. The gate
  read the broken file and printed doc claims OK. Both sides of all four blocks
  were byte-identical, so the repair is lossless (verified by comparing the
  sides, not by picking one). Two new checks close the class, both derived from
  SCANNED_FILES so a newly scanned file is enrolled with no further edit, and
  both failing closed on a file they cannot read.
• Every action was on the deprecated Node 20 runtime. The previous body
  reported this as "unrelated, pre-existing" with no issue number — precisely
  the deferral §14.3 forbids. These refs live in ci.yml, a file this diff
  modifies, so §14.1 puts them in touched material: fixed here, not filed.

---

Completion Ledger (§13.2)

Path enumeration

Path	Evidence
named_index_url — found	test_every_local_version_pin_names_a_serving_index
named_index_url — absent → ExportError	test_undeclared_index_is_refused
ConstraintSet.command() — extras / groups / only-groups	test_export_asserts_the_lock_matches_pyproject
header() — with and without index directive	test_every_local_version_pin_names_a_serving_index
render() — uv missing → ExportError	test_export_failure_exits_two_not_one
render() — uv non-zero exit → ExportError	test_export_failure_exits_two_not_one
render() — empty export refused	test_empty_export_is_refused
render() — unhashed requirement refused	test_unhashed_export_is_refused
render() — local pin absent from lock	test_local_pin_absent_from_the_lock_is_refused
write() — changed / unchanged	test_check_passes_on_the_committed_tree
stale() — missing file	test_check_fails_when_a_file_is_missing
stale() — content differs	test_check_fails_on_a_mutated_file
stale() — current → None	test_check_passes_on_the_committed_tree
main() — --check clean → 0 / drift → 1 / cannot run → 2	test_check_passes_on_the_committed_tree, test_check_fails_on_a_mutated_file, test_export_failure_exits_two_not_one
Drift message names the fix command	test_drift_message_names_the_regeneration_command
Every set has a file; no file orphaned; names unique	test_every_set_has_a_committed_file, test_no_committed_file_is_orphaned, test_filenames_are_unique
Every requirement version-pinned and hashed	test_every_requirement_is_version_pinned_and_hashed
No editable/directory requirements leak into a hashed file	test_no_editable_or_directory_requirements
normalize_source_path — canonical / ./ / / / backslash / blank / separators-only	TestNormalizeSourcePath (6 tests)
normalize_source_path — fixed-point regressions	TestNormalizeSourcePathReachesAFixedPoint (4 + idempotence)
normalize_source_path — .. left intact (negative)	test_parent_traversal_is_left_intact
extract_document_paths — two spellings collapse	test_two_spellings_of_one_document_collapse_to_one
Harness discovery non-empty (a fuzz setup finding nothing passes everything)	test_at_least_one_harness_exists
Every harness holds on its whole corpus; empty corpus refused	test_harness_holds_on_its_whole_corpus
YAML harness — type, dict-ness, lowercased keys, substring body	fuzz_yaml_frontmatter.consume over 10 corpus inputs
Path harness — no leading /, no leading ./, non-empty, idempotent	fuzz_source_path.consume over 10 corpus inputs
[tool.uv] constraint-dependencies — 3.10 fork resolves an installable onnxruntime	17/17 pip install --dry-run --require-hashes pairs on linux/amd64 (table below)
Lock regeneration is byte-identical to the CI gate's expectation	generate_pip_constraints.py --check → requirements OK (13 checked)
check_no_conflict_markers — labelled marker found	test_conflict_markers_are_reported_with_path_and_line
check_no_conflict_markers — clean file → no failure	test_a_clean_file_reports_nothing
check_no_conflict_markers — Markdown setext ======= NOT flagged (negative)	test_a_markdown_setext_underline_is_not_a_conflict_marker
check_no_conflict_markers — missing file fails closed	test_a_missing_scanned_file_fails_closed
check_scanned_json_parses — invalid JSON reported	test_unparseable_json_is_reported
check_scanned_json_parses — valid JSON → no failure	test_valid_json_reports_nothing
check_scanned_json_parses — non-JSON member skipped (negative)	test_markdown_is_not_json_checked
Both new checks are actually wired into collect_failures	test_both_checks_run_inside_collect_failures
The real repository tree is structurally clean	test_the_real_repository_tree_is_structurally_clean
Every pinned action runs on a supported Node runtime	repo-wide sweep of all 16 refs → 0 on node20

§13.1 checklist

A. Correctness & behavior

• A1 Full suite green locally on the merged tree: 6421 passed, 5 skipped, 117 subtests passed in 653.86s (0:10:53) (6421 + 5 = the 6426 collected). CI agrees on the exact pushed tree: collected 6426 tests → doc claims OK → badges OK (5 checked) (job 90574807372). --require-hashes install proven end-to-end (ruff 0.15.20 installed from requirements/lint.txt, exit 0).
• A2 Edge cases: missing/mutated/empty/unhashed export, absent index declaration, uv absent, uv non-zero, blank & separator-only paths, CRLF, lone surrogates, unicode, 200-char dash runs — each mapped above.
• A3 Every failure arm asserted on observable effect, including the exit-code distinction between "could not run" (2) and "found drift" (1) — a missing uv must not read as a clean gate.
• A4 Trust boundary is the index; --require-hashes is the validation. Adversarial case tested directly: every hash corrupted → pip exits 1 with "THESE PACKAGES DO NOT MATCH THE HASHES". A single corrupted hash is correctly not enough (pip accepts if any listed hash matches) — the first attempt at this test was invalid for that reason and was redone.
• A5 Invariants: --check never writes; generation idempotent; normalize_source_path idempotent (the property the bug violated).
• A6 Idempotent by construction — write() no-ops when content matches.

B. Concurrency — N/A: single-shot synchronous scripts, no shared mutable state, no async. CI jobs are independent.

C. Resources & performance

• C1 Loops are over a fixed 13-set registry, a fixed corpus, and file lines. No unbounded growth.
• C2 No handles held; corpus read via read_bytes. Image size falls — 18 nvidia/cuda packages + triton removed from every container.
• C3 PR fuzz batch capped at 120s/harness/sanitizer so it doesn't become the slowest required check.

D. Security — this is the point of the change

• D1 No SQL/shell built from data. subprocess.run uses a fixed argv, no shell, no user input (# noqa: S603 with that rationale).
• D2 Untrusted input is the whole subject: the two fuzz targets read LLM-written and user-supplied documents.
• D3 No secrets touched; permissions: read-all on the fuzz workflow. Net effect: one unreviewed remote script no longer executes as root at build time, and every installed artifact is hash-verified. The one --extra-index-url added is documented as safe only because the file is hash-pinned — an artifact from either index that isn't the locked one fails the hash check before unpacking.

E. Interfaces & compatibility

• E1 Additive: new script, new requirements/, new groups. No public API change; PEP 735 groups are not published, so PyPI consumers see identical metadata.
• E2 Downstream consumers identified by name and verified: ci.yml (all 7 sites), release.yml (2), scripts/setup.sh, Dockerfile, docker/Dockerfile, .devcontainer/Dockerfile, .clusterfuzzlite/build.sh. extract_document_paths and tests_py/handlers/test_wiki_page_sources.py re-verified against the changed normalisation (637 passed).
• E3 N/A: no persisted-data format change.
• E4 .gitattributes handles the cross-platform EOL case explicitly. The Windows CI leg installs ci-sqlite-min.txt, which resolves without a tree-sitter toolchain by design.

F. Observability & operations

• F1 Every failure names the file and the remedy (run scripts/generate_pip_constraints.py); asserted by test_drift_message_names_the_regeneration_command. Success prints requirements OK (13 checked). setup.sh no longer swallows pip's stderr.
• F2 No silent degraded mode: an unhashed or empty export raises rather than writing a weaker file.

G. Tests

• G1 Ledger above maps every diff path.
• G2 Both bugs carry regressions that fail on pre-fix code — verified for the path bug by git stash-ing the fix and re-running the replayer (it failed on repro-backslash-mixed, then passed once restored).
• G3 Deterministic and isolated: temp dirs, no network in unit tests, no sleeps, no ordering dependence.
• G4 Negative assertions present: test_no_editable_or_directory_requirements, test_no_committed_file_is_orphaned, test_parent_traversal_is_left_intact, corrupted-hash rejection.
• G5 Full suite quoted in A1; ruff check . → All checks passed!; ruff format --check . → 1064 files already formatted; check_doc_claims.py and generate_pip_constraints.py --check both green.

H. Code quality & delivery

• H1 SOLID/layering/sizes: the constraint table is separated from the engine that runs uv, so the mapping is testable without uv on PATH and both files stay under the cap. Adding a set is one registry entry; adding a harness is one file (discovered, not listed). The two lint waivers are per-site with stated reasons (S603 fixed argv; PLC0415 atheris has no wheel for macOS/aarch64).
• H2 No dead code, no debug leftovers. --upgrade pip build removed from the root image: it was itself an unpinned install and build was never invoked there.
• H3 Matches neighbouring script conventions (check_doc_claims.py, generate_repo_badges.py — same fail-closed --check idiom).
• H4 CHANGELOG has Security/Fixed entries, extended in this pass with the lock fix and the conflict-marker gate; suite size synced to the measured 6426 across 5 claim sites + the committed badge SVG.
• H5 Logic, formatting and docs kept in separate commits: merge+repair, lock fix, gate+tests, ruff format (formatting only), count refresh, CHANGELOG, action bumps.
• H6 CI green on the exact pushed tree — see the check list on this PR. The previously-failing Test (Python 3.10) now passes.
• H7 Boy-scout (§14) — every defect seen in touched material is fixed here, none deferred: the unbuildable docker/Dockerfile, setup.sh's swallowed stderr and drifted package list, the non-idempotent path canonicaliser, the CRLF-corruptible corpus, and hatchling being fetched unpinned during build isolation. No unfixed, un-issued seen defect remains.

Resolvability matrix — every (requirements file, python) pair its consumers use

pip install --dry-run --require-hashes, run on linux/amd64 (CI's platform;
this host is arm64, so a same-arch check would have validated a different wheel
set than CI resolves). Before the fix, the first row fails with
No matching distribution found for onnxruntime==1.24.3 — reproduced, then
re-run to green.

OK   ci-postgresql.txt on py3.10      <- the failing check, now green
OK   ci-postgresql.txt on py3.11
OK   ci-postgresql.txt on py3.12
OK   ci-postgresql.txt on py3.13
OK   setup.txt on py3.10
OK   setup.txt on py3.14
OK   ci-sqlite.txt on py3.12
OK   ci-sqlite-min.txt on py3.12
OK   lint.txt on py3.12
OK   packaging.txt on py3.12
OK   release.txt on py3.12
OK   ci-typecheck.txt on py3.13
OK   typecheck-tool.txt on py3.13
OK   runtime-postgresql.txt on py3.14
OK   devcontainer.txt on py3.14
OK   docker-runtime.txt on py3.14
OK   fuzz.txt on py3.12

17/17. fuzz.txt is verified on 3.12 only and by design: atheris publishes
manylinux x86_64 wheels for cpython 3.12–3.14 and nothing else, so that set does
not resolve elsewhere and is not expected to.

Scorecard — measured, not assumed

Real scanner (gcr.io/openssf/scorecard:stable) against this working tree,
re-run after the action bumps to prove they did not regress the pinning:

Aggregate score: 10.0 / 10
| 10 / 10 | Fuzzing             | project is fuzzed           | ClusterFuzzLite integration found;
|         |                     |                             | PythonAtherisFuzzer: fuzz/fuzz_source_path.py:62,
|         |                     |                             | fuzz/fuzz_yaml_frontmatter.py:73
| 10 / 10 | Pinned-Dependencies | all dependencies are pinned | 43/43 GH actions, 18/18 third-party,
|         |                     |                             | 6/6 containerImage, 22/22 pipCommand,
|         |                     |                             | 1/1 npmCommand

Both checks close only at max score, and both are at max. The PR's own added
lines were also swept against the checker's unpinned-install predicates: every
pip install added by this diff carries --require-hashes, or -e plus
--no-deps; no new curl | sh, no unversioned npm install.

Gates run on the merged tree

Gate	Result
ruff format --check .	1067 files already formatted
ruff check .	All checks passed!
python -m pyright mcp_server/	0 errors, 0 warnings, 0 informations
pytest	6421 passed, 5 skipped, 117 subtests passed in 653.86s
generate_pip_constraints.py --check	requirements OK (13 checked)
check_doc_claims.py --test-count 6426	doc claims OK
generate_repo_badges.py --check --test-count 6426	badges OK (5 checked)

The pyright run is quoted from a venv built exactly as CI builds it
(ci-typecheck.txt + -e . --no-deps + typecheck-tool.txt). A first attempt
using the test env reported 4 reportMissingImports errors on
mcp_server/infrastructure/otel_exporter.py — that env simply lacks the [otel]
extra, and an unresolved import makes a type-checker report fewer real errors,
not more. Quoted only after the environment could resolve the imports.

Still measured by CI rather than locally, and why

Stated plainly rather than implied green:

1. The amd64 container builds are CI's, not this machine's. Docker here is
   Colima on arm64 with no buildx, so --platform linux/amd64 is not a
   cross-build: the legacy builder installs aarch64 toolchains and then fails
   the platform assertion. The root Dockerfile builds clean natively
   (Successfully tagged cortex:pin-check), and the authoritative amd64
   evidence is this PR's own green Docker Build (runtime image), Docker
   Build (devcontainer image) and Docker Smoke jobs.
2. The fuzzers themselves do not run here. Atheris has no macOS/aarch64
   wheel. The harness properties run locally through the corpus replayer inside
   the ordinary pytest suite; the coverage-guided campaign is the green
   Fuzz (PR batch) jobs.
3. Alert closure is only provable on main after merge. Scorecard's alerts
   are written by the scorecard.yml run on the default branch, so the 22 → 0
   count must be read there, never off this PR ref. The 10/10 above is the
   scanner's verdict on this tree, which is the strongest pre-merge evidence
   available.

Nothing deferred

No un-issued deferral remains. The Node 20 deprecation — the one item the
previous body carried as "unrelated, pre-existing" without an issue number — is
fixed in this PR: all 16 pinned action refs now sweep to 0 on node20, each
still SHA-pinned and each verified by reading runs.using out of its
action.yml at the target SHA rather than trusting the tag. codeql-action
needed the v4 line, since v3.37.3 is still node20.

🤖 Generated with Claude Code

https://claude.ai/code/session_012Yu6EnWspTfqHoGkExyS6u

[cdeust]

Heads-up from #248: ci.yml's Node20 actions were bumped to their Node24 majors on main (checkout 3d3c42e5 v7.0.1, cache 55cc8345 v6.1.0, upload-artifact 043fb46d v7.0.1, build-push-action 53b7df96 v7.3.0). This branch's fuzz.yml still carries the Node20 pins and will emit the deprecation warning — please adopt the same four SHAs there when resolving this PR's conflicts. Tracked in #246.