Skip to content

fix(testing): make finalized-safety head check scheme-independent - #916

Merged
tcoratger merged 1 commit into
leanEthereum:mainfrom
tcoratger:fix/finalized-safety-scheme-independent-head
Jun 10, 2026
Merged

fix(testing): make finalized-safety head check scheme-independent#916
tcoratger merged 1 commit into
leanEthereum:mainfrom
tcoratger:fix/finalized-safety-scheme-independent-head

Conversation

@tcoratger

Copy link
Copy Markdown
Collaborator

Problem

test_fork_above_finalized_wins_at_or_below_loses passed under the test scheme but failed under prod:

AssertionError: Step 5: head_slot = 6, expected 5

Root cause — a scheme-dependent fork-choice tie

At step 5 the test adds above_6 (slot 6) as a sibling of block_5 (slot 5) — both children of the justified block block_4, both empty (no attestations target either).

  • LMD-GHOST starts the head walk at the justified root block_4 and takes the heaviest child. Every chain vote (V0..V5) has latest-message head = block_4, so neither child gets distinguishing weight → a zero-weight tie.
  • The spec breaks weight ties by block root (standard LMD-GHOST), fork_choice.py:
    head = max(children, key=lambda child_root: (weights[child_root], child_root))
  • A block root is hash_tree_root(Block), which includes the state_root, which embeds the validator registry → each validator's XMSS public key. test and prod use different XMSS schemes, so the keys differ → state_root differs → the two block roots differ, and their ordering flips:
    • test keys: root(block_5) > root(above_6) → head block_5 (slot 5) ✅
    • prod keys: root(above_6) > root(block_5) → head above_6 (slot 6) ❌

So the test asserted the outcome of a hash-based tiebreak that happens to depend on key material. The spec is correct; the test expectation was scheme-fragile.

(The sibling test in the same file does not fail because its fork branches off block_1, below the justified root, so it is unreachable in the head walk — block_5 is the sole reachable child, no tie.)

Fix

At the tie step, stop asserting which sibling is head. Assert instead the scheme-independent facts that the empty fork block changes neither justification (block_4) nor finalization (block_3). The genuine "above fork wins" behavior is still asserted deterministically at the next step, where above_7 brings the 6 votes that justify above_6 and move the head onto above_7 (whose parent has a single child → no tie).

The test's Given/When/Then docstring describes the final state and reachability — both scheme-independent — and never asserted the step-5 head, so it stays accurate unchanged.

Also

Adds a repository rule to CLAUDE.md: every test change must update the associated documentation in the same change, per the documentation rules (with the tests/consensus/ step ↔ Given/When/Then mapping kept one-to-one).

Testing

  • just check passes.
  • Both tests in the file pass under the test scheme (no regression).
  • The fix removes the only scheme-dependent assertion, so it holds under prod (where the step-5 head is legitimately above_6).

🤖 Generated with Claude Code

A fork-choice vector asserted head_slot=5 at the step that adds an empty
sibling block above the canonical head.
At that step both children of the justified block carry zero distinguishing
fork-choice weight, so the head is a pure tie broken by the larger block
root.
A block root embeds the post-state root, which embeds the validator public
keys, so the tie resolves differently under the test and production
signature schemes.
The vector passed under the test scheme and failed under production, where
the sibling at slot 6 won the tie.

Stop asserting which sibling is head at the tie step.
Assert instead that the empty fork block changes neither justification nor
finalization, which holds under both schemes.
The genuine "above fork wins" property is still asserted at the next step,
where the fork gathers the votes that justify it and move the head onto it
with no tie.

Also add a repository rule: every test change must update the associated
documentation in the same change, per the documentation rules.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@tcoratger
tcoratger merged commit d8a2dd4 into leanEthereum:main Jun 10, 2026
13 checks passed
tcoratger pushed a commit that referenced this pull request Jul 11, 2026
…1189)

* fix(testing): make equivocation head assertions scheme-independent

The equivocation tiebreak vectors hardcoded the winning fork of #1181's
largest-attestation-data-root rule. That root embeds validator XMSS pubkeys,
so the winner flips between test and prod schemes; the vectors passed PR CI
(test scheme) and failed prod-vectors on main (prod scheme). Same class as #916.

Add a scheme-independent canonical_equivocation_head_among check that reads
each fork's attestation root from the store and asserts the head is the
largest-root fork, mirroring lexicographic_head_among. Rewrite the 3 vectors
to use it, note the scheme-dependence on the spec tiebreak, add a CLAUDE.md
rule, and gate fork-choice vectors under --scheme=prod in PR CI.

* ci: drop prod-scheme fork-choice smoke job

The fill-tests-prod-scheme job was cancelled at its 20-minute timeout on
the macOS runner (the fork-choice tree under --scheme=prod takes >20 min in
CI). Remove it; the scheme-independence rule in CLAUDE.md and the
canonical_equivocation_head_among check remain.

* docs: trim equivocation scheme-independence notes

Cut the root-to-pubkey derivations and test guidance an experienced reader
already knows. The CLAUDE.md rule keeps the actionable bullets, the check's
field docstring matches lexicographic_head_among's brevity, and the spec
docstring states only the behavioral property (no test instructions).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant