Repository navigation
fix(mincut): deterministic, non-degenerate RuVectorGraphAnalyzer::partition() - #979
Conversation
Fixes the Rustfmt CI check on PR #979. Co-Authored-By: claude-flow <ruv@ruv.net> Claude-Session: https://claude.ai/code/session_01VFTtb2ZrTNkmpWVQfuiKkT
|
CI status: The job ran 3547 tests across the workspace; exactly one failed: Every other check on this head is green: Rustfmt, Clippy, Cargo check, and I don't have a fix to port — I did not touch Generated by Claude Code |
|
Second failure, also not this PR's:
I don't have a fix to port (no open PR found addressing it, and bumping Generated by Claude Code |
|
Re-run results:
I've used this session's one re-run for this workflow run already Generated by Claude Code |
RuVectorGraphAnalyzer::partition() returned an empty/degenerate result in ~50% of repeated calls against a byte-identical graph (first measured by the 2026-09-05 nightly run, root cause left unresolved). Two independent bugs: - DynamicGraph::vertices()/edges() iterated a DashMap directly. DashMap's default hasher is randomly seeded per instance, so iteration order isn't stable across process runs even for identical insertions. Downstream tie-breaking (BoundedInstance's seed selection and bitmask-to-vertex assignment) silently picked a different valid cut on every run. - WitnessHandle::materialize_partition() inferred the graph's vertex range from max(U) (the cut side's own membership) instead of the graph's real vertex count, so V \ U came back truncated or empty whenever U didn't happen to contain the graph's highest-numbered vertex — the common case. Fix: sort vertices()/edges() at the DynamicGraph read boundary, sort the HashSet-derived vertex lists BoundedInstance uses for tie-breaking, and have RuVectorGraphAnalyzer::partition() build both cut sides from the graph's real vertex list via witness.contains() instead of materialize_partition(). No public API signature changes. Verified against the exact reproduction script from the 2026-09-05 run: 0/60 empty results across two 30-trial runs (was 15/30, 50%), plus a new regression test asserting byte-identical partitions across repeated calls. Full ruvector-mincut suite: 644 passed, 0 failed, 5 pre-existing ignored. Co-Authored-By: claude-flow <ruv@ruv.net> Claude-Session: https://claude.ai/code/session_01VFTtb2ZrTNkmpWVQfuiKkT
Records the decision, evidence, alternatives considered, and open questions for the ruvector-mincut determinism fix. Regenerated docs/adr/INDEX.md via scripts/adr-index.mjs. Co-Authored-By: claude-flow <ruv@ruv.net> Claude-Session: https://claude.ai/code/session_01VFTtb2ZrTNkmpWVQfuiKkT
Full methodology, root-cause diagnostic narrative (including the intermediate finding that fixing only the DashMap-ordering bug made the symptom look worse before the second, independent witness-partition bug was found), benchmark results, limitations, and next research for the 2026-09-11 nightly run. Co-Authored-By: claude-flow <ruv@ruv.net> Claude-Session: https://claude.ai/code/session_01VFTtb2ZrTNkmpWVQfuiKkT
Fixes the Rustfmt CI check on PR #979. Co-Authored-By: claude-flow <ruv@ruv.net> Claude-Session: https://claude.ai/code/session_01VFTtb2ZrTNkmpWVQfuiKkT
Sorting every edges() call made it 2.2-2.8x slower (0.13->0.29 ms at 10k edges, 2.58->7.17 ms at 200k) and edges() sits in the LocalKCut::check_cut hot loop (#942). partition() determinism comes from the vertices() sort; without this sort the reviewer's probe still produced one vertex set per graph. Co-Authored-By: claude-flow <ruv@ruv.net>
Co-Authored-By: claude-flow <ruv@ruv.net>
Co-Authored-By: claude-flow <ruv@ruv.net>
f891db8 to
8b39ea9
Compare
…back anchor, SHA-256 spans Addresses the PR #989 security review: - HIGH coverage: verify_signed_chain now requires spans to tile the log exactly from seq 0 to the newest record (no gaps, sequence == index); an empty/partial span list no longer verifies a re-hashed forged log. - HIGH BatchTail tail: uncovered records fail closed with SignedChainError::UnsignedTail { signed, unsigned } (reported only after the signed prefix verified); flush() renamed seal(). - HIGH rollback: required SignedAnchor { record_count, head_digest } (signed analogue of the log head commitment); truncating log + spans below the anchor fails with AnchorMismatch. - HIGH digest: sign SHA-256 (rvf-types::sha256) over the full 64-byte records of each span, chained span-to-span via SHA-256 links, with v2 domain separation, instead of the 64-bit FNV-1a chain_hash. - LOW: hold an ed25519_dalek::SigningKey (same 2.2.x dalek rvf-types already uses), verify_strict locally, Result instead of panic on batch_size == 0, refuse non-contiguous sequences before the inner sink. - verify_signed_chain returns Result<SignedChainReport, SignedChainError>. - Regression tests for all three PoCs plus splice/reorder/positive cases. - ADR renumbered 346 -> 347 (346 is the mincut ADR, PR #979) and rewritten to state guarantees and limits precisely. Co-Authored-By: claude-flow <ruv@ruv.net>
…DR to 348 - Maintain per-token retained-twin counts incrementally so swaps are judged against the current retained set, not the input snapshot. A donor is only evicted if it still has a retained twin and no evicted token would lose its last retained twin; orphans already covered by an earlier admission are skipped. Fixes mutual-twin loss ([x,y,y,z,w], retained [0,1,2] previously returned [0,3,4]). Cost stays O(n K d); documented on the function. - Normalize keys after scaling by the max absolute component, so finite keys can no longer overflow or underflow the squared norm; ZeroNormKey is now reported only for all-zero keys. Drop the redundant self-candidate check. - Add regression tests: mutual twins, orphan already covered, evicted token stranded by donor eviction, extreme-magnitude normalization. - Renumber ADR-330 (clash with CAMA ADR-330) to ADR-348; ADR-346/347 are claimed by PRs #979/#989. Minimal INDEX.md update. Co-Authored-By: claude-flow <ruv@ruv.net>
Ships the deterministic partition()/witness fix from #979. Per-crate version override since workspace 2.3.0 is already on crates.io.
…ss chain (#989) * feat(agent-memory): add Ed25519 WitnessSigner for the TARL witness chain Closes the ADR-134 SS9 WitnessSigner gap named in ledger.rs/ops.rs's own tamper-evidence notes and the 2026-09-05 nightly's Next Research item 4: the FNV-1a witness chain is tamper-evident against accidental corruption only, not a log-writing adversary. Adds witness_signing::SignedWitnessSink<S: WitnessSink>, a decorator that Ed25519-signs records either per-record or via an amortized batch-tail strategy, reusing the crate's existing rvf-types Ed25519 primitive (ADR-320) -- no new dependency. verify_signed_chain combines the existing unsigned chain walk with signature checks cross-bound to each signed record's current content, which is what actually detects a diligent, fully self-consistent forgery that the unsigned walk alone cannot. TransactionalLedger gains into_witness_sink() to recover a wrapped sink's signed spans after a run. Co-Authored-By: claude-flow <ruv@ruv.net> Claude-Session: https://claude.ai/code/session_01Gpkdh8ATrH62owAUHQhRUQ * bench(agent-memory): add witness signing latency/throughput benchmark Compares unsigned baseline vs PerRecord vs BatchTail{16,64,256} over a 20,000-entry (40,000 witness record) synthetic workload, release build, real wall-clock timing, deterministic signing key. Also constructs a diligent-forgery scenario (edit one record, recompute the chain forward consistently) and confirms both signing strategies reject it while the unsigned chain walk alone does not. Run: cargo run --release -p ruvector-agent-memory --example witness_signing_bench Co-Authored-By: claude-flow <ruv@ruv.net> Claude-Session: https://claude.ai/code/session_01Gpkdh8ATrH62owAUHQhRUQ * docs(research): nightly report for the TARL witness-signer experiment Full hypothesis, methodology, raw benchmark evidence, acceptance gates, failure modes, explicit scope boundary on the FNV-1a second-preimage question, ecosystem-fit analysis, and next-research items for the 2026-09-16 nightly run (docs/research/nightly/2026-09-16-witness-signer-agent-memory). Co-Authored-By: claude-flow <ruv@ruv.net> Claude-Session: https://claude.ai/code/session_01Gpkdh8ATrH62owAUHQhRUQ * docs(adr): add ADR-346 for the TARL ledger WitnessSigner Records the accepted decision, evidence summary, alternatives considered, security scope, and rollback plan for witness_signing.rs. Updates the ADR index (highest allocated number, new row). Co-Authored-By: claude-flow <ruv@ruv.net> Claude-Session: https://claude.ai/code/session_01Gpkdh8ATrH62owAUHQhRUQ * fix(agent-memory): harden WitnessSigner — coverage, sealed tail, rollback anchor, SHA-256 spans Addresses the PR #989 security review: - HIGH coverage: verify_signed_chain now requires spans to tile the log exactly from seq 0 to the newest record (no gaps, sequence == index); an empty/partial span list no longer verifies a re-hashed forged log. - HIGH BatchTail tail: uncovered records fail closed with SignedChainError::UnsignedTail { signed, unsigned } (reported only after the signed prefix verified); flush() renamed seal(). - HIGH rollback: required SignedAnchor { record_count, head_digest } (signed analogue of the log head commitment); truncating log + spans below the anchor fails with AnchorMismatch. - HIGH digest: sign SHA-256 (rvf-types::sha256) over the full 64-byte records of each span, chained span-to-span via SHA-256 links, with v2 domain separation, instead of the 64-bit FNV-1a chain_hash. - LOW: hold an ed25519_dalek::SigningKey (same 2.2.x dalek rvf-types already uses), verify_strict locally, Result instead of panic on batch_size == 0, refuse non-contiguous sequences before the inner sink. - verify_signed_chain returns Result<SignedChainReport, SignedChainError>. - Regression tests for all three PoCs plus splice/reorder/positive cases. - ADR renumbered 346 -> 347 (346 is the mincut ADR, PR #979) and rewritten to state guarantees and limits precisely. Co-Authored-By: claude-flow <ruv@ruv.net> * docs(research): reconcile nightly README with post-review hardening The 01b9395 fix commit addressed PR #989's review (coverage gap, unsigned tail, rollback, FNV-1a->SHA-256 digest) but only touched the ADR and a one-line gist reference, leaving the research README's design description, raw benchmark numbers, and Next Research item 2 describing the pre-fix implementation. Adds an "Update (2026-09-26)" section summarizing each finding and fix, the re-run benchmark output against the hardened implementation, and marks Next Research item 2 (FNV-1a second-preimage cost) resolved by design rather than by measurement, per the review's own observation that FNV-1a's per-step invertibility makes the crate's "~2^32" estimate an unverified upper bound, not a floor. Co-Authored-By: claude-flow <ruv@ruv.net> Claude-Session: https://claude.ai/code/session_01Gpkdh8ATrH62owAUHQhRUQ --------- Co-authored-by: Claude <noreply@anthropic.com>
Summary
Nightly research run (autonomous). Root-causes and fixes the
non-determinism in
ruvector_mincut::RuVectorGraphAnalyzer::partition()that the 2026-09-05 nightly (
docs/research/nightly/2026-09-05-mincut-gated-forgetting,ADR-345)
measured (15/30 calls returning empty/degenerate on a byte-identical graph)
but did not locate.
Two independent bugs, both fixed, no public API signature changes:
DynamicGraph::vertices()/edges()iterated aDashMapdirectly.DashMap's default hasher is randomly seeded per instance, so itsiteration order is not stable across process runs even for identical
insertions — downstream tie-breaking (
BoundedInstance's seed selectionand bitmask-to-vertex assignment) silently picked a different valid cut
on every run.
WitnessHandle::materialize_partition()inferred the graph's vertexrange from
max(U)(the cut side's own membership) instead of thegraph's real vertex count, so the complement
V \ Ucame back truncatedor empty whenever
Udidn't happen to contain the graph'shighest-numbered vertex — the common case, not an edge case. This was
the direct, deterministic cause of most empty results; fixing bug 1
alone (see commit history) made the symptom measurably worse before
bug 2 was found, which is itself a notable finding about diagnosing
compound bugs from a single aggregate metric.
Fix: sort
vertices()/edges()at theDynamicGraphread boundary;sort the
HashSet-derived vertex listsBoundedInstanceuses fortie-breaking; have
RuVectorGraphAnalyzer::partition()build both cutsides from the graph's real vertex list via
witness.contains()instead ofmaterialize_partition().Eighteen crates depend on
ruvector-mincutdirectly (including WASM/Nodebindings,
ruvector-agent-memory,prime-radiant,cognitum-gate-kernel,mcp-brain-server); all get this correctness fix for free on next rebuildwith zero migration cost.
Explicitly out of scope: the separately-documented latency scaling
problem (77ms–11.4s across 50–400 vertices) is untouched — this is a
correctness fix, not a performance fix.
Evidence
Reused the prior nightly's own unmodified reproduction script
(
crates/ruvector-agent-memory/examples/mincut_determinism_probe.rs) foran honest before/after comparison:
New regression test (
crates/ruvector-mincut/tests/determinism_tests.rs)asserts a strictly stronger property — byte-identical partitions across 30
repeated calls, not just "non-empty":
Full pre-existing
ruvector-mincutsuite (lib + all 10 integration testfiles): 644 passed, 0 failed, 5 pre-existing ignored.
Downstream sanity check —
ruvector-agent-memory'sgraph_forgettests(the crate/feature that originally surfaced this bug,
mincut-forgetfeature): 3 passed, 0 failed.
cargo clippy --release -p ruvector-mincut --lib --tests -- -D warnings:clean (no new warnings).
Files changed
crates/ruvector-mincut/src/graph/mod.rs— sortvertices()/edges()crates/ruvector-mincut/src/instance/bounded.rs— sort seed/tie-breakvertex lists; deterministic witness seed
crates/ruvector-mincut/src/instance/witness.rs— doc comment only(documents
materialize_partition()'s scope limitation)crates/ruvector-mincut/src/integration/mod.rs—partition()usesgraph.vertices()+witness.contains()instead ofmaterialize_partition()crates/ruvector-mincut/tests/determinism_tests.rs— new regression testdocs/adr/ADR-346-deterministic-mincut-witness-partition.md— new ADRdocs/adr/INDEX.md— regenerated vianode scripts/adr-index.mjsdocs/research/nightly/2026-09-11-mincut-partition-determinism/README.md— full nightly research report
docs/research/nightly/2026-09-11-mincut-partition-determinism/gist.md— standalone technical write-up
Benchmark commands (reproducible)
cargo build --release -p ruvector-agent-memory \ --example mincut_determinism_probe --features mincut-forget TRIALS=30 ./target/release/examples/mincut_determinism_probe cargo test --release -p ruvector-mincut --lib --testsAcceptance result
ACCEPT — see ADR-346
and the nightly README
for the full evidence table, falsification criteria, and limitations
(only the
<20-vertex brute-force path was end-to-end re-benchmarked;concurrent-mutation and WASM/Node binding smoke tests are flagged as
follow-up, not done this run).
MetaHarness / Flywheel / Darwin capability discovery
Re-verified per the nightly process's own rule — unchanged from the prior
nightly:
npx metaharness --helpresolves to a generic project-scaffoldingCLI not wired into this repo;
npx ruvector harness doctor --jsonfails(no such CLI package installed). No Darwin evolutionary search was run —
this was a targeted root-cause diagnosis of an already-reported bug, not a
parameter search over a candidate population.
Security / governance
No security-relevant surface changed (no new I/O, no trust-boundary or
signing-path changes). No schema, wire-format, or persisted-data changes.
Limitations / follow-up (see ADR-346 "Open Questions")
search_for_cuts's seed-ordering fix (the>=20-vertexLocalKCutpath)was applied by the same reasoning but not independently re-benchmarked at
that size this session.
MincutGatedForgettingacceptancebenchmark (that rejection has a second, independent, unaddressed latency
cause).
🤖 Generated with claude-flow
https://claude.ai/code/session_01VFTtb2ZrTNkmpWVQfuiKkT
Generated by Claude Code