Skip to content

Commit e9fa1f8

Browse files
authored
fix: update spec-test runners for the latest leanSpec fixture schema (#429)
## Motivation #385 switched CI to download the latest leanSpec fixtures release instead of generating them from a pinned commit. The current release carries fixture-schema changes the test runners don't understand yet, so the Test job on `main` fails. Note: leanSpec's release is a single rolling `latest` tag — it was republished mid-review (2026-06-10 17:31 UTC), which is what the second commit addresses. ## Description Runner/harness-only changes — no consensus behavior is touched, and ethlambda already matches the spec in every affected case: - **`tickToSlot` (fork-choice block steps):** new flag, default `true`. The early-arrival tests set it to `false` to deliver a block ahead of the store clock and assert the clock doesn't move. The forkchoice spectest runner and the Hive test driver now only tick to the block's slot when the flag is set. - **`rejectionReason`:** leanSpec renamed `expectException`. The `verify_signatures` and `state_transition` fixture types accept both spellings via a serde alias. - **`ssz_test`:** the SSZ fixture format string was renamed from `ssz`; the runner accepts both. - **`proofSetting` / mocked proofs:** leanSpec now fills most vectors with placeholder aggregation proofs (`MOCKED-AGGREGATION-PROOF` sentinel) instead of paying for recursive SNARK merges, marking the regime in a top-level `proofSetting` field (0 = mocked, must not be verified; 1 = real and valid; 2 = real and invalid). The forkchoice runner routes mocked vectors through a new `on_gossip_aggregated_attestation_without_verification` (mirroring the `on_block`/`on_block_without_verification` split). Production paths always verify. Known follow-ups (out of scope): - The Hive test driver receives fork-choice steps one at a time with no fixture-level context, so it cannot honor `proofSetting` yet; Hive runs against mocked-proof fixtures will need a protocol addition. - Tracking the rolling `latest` release means any upstream republish can break `main`'s CI at any time; pinning the fixtures tarball SHA256 would restore reproducibility. ## How to Test ``` make test # downloads latest release fixtures, all suites green ``` Locally verified against the 2026-06-10 17:31 UTC fixtures drop: full `cargo test --workspace --release` passes (108 forkchoice, 118 SSZ, 67 STF, all remaining suites green).
1 parent ed2fea0 commit e9fa1f8

7 files changed

Lines changed: 75 additions & 14 deletions

File tree

crates/blockchain/src/store.rs

Lines changed: 22 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -392,6 +392,26 @@ pub fn on_gossip_attestation(
392392
pub fn on_gossip_aggregated_attestation(
393393
store: &mut Store,
394394
aggregated: SignedAggregatedAttestation,
395+
) -> Result<(), StoreError> {
396+
on_gossip_aggregated_attestation_core(store, aggregated, true)
397+
}
398+
399+
/// Process a gossiped aggregated attestation WITHOUT verifying its proof.
400+
///
401+
/// Only for spec tests whose fixtures carry mocked (placeholder) proofs
402+
/// (`proofSetting == 0`); production paths must use
403+
/// [`on_gossip_aggregated_attestation`].
404+
pub fn on_gossip_aggregated_attestation_without_verification(
405+
store: &mut Store,
406+
aggregated: SignedAggregatedAttestation,
407+
) -> Result<(), StoreError> {
408+
on_gossip_aggregated_attestation_core(store, aggregated, false)
409+
}
410+
411+
fn on_gossip_aggregated_attestation_core(
412+
store: &mut Store,
413+
aggregated: SignedAggregatedAttestation,
414+
verify: bool,
395415
) -> Result<(), StoreError> {
396416
validate_attestation_data(store, &aggregated.data)
397417
.inspect_err(|_| metrics::inc_attestations_invalid())?;
@@ -420,16 +440,16 @@ pub fn on_gossip_aggregated_attestation(
420440
let data_root = hashed.root();
421441
let slot: u32 = aggregated.data.slot.try_into().expect("slot exceeds u32");
422442

423-
{
443+
if verify {
424444
let _timing = metrics::time_pq_sig_aggregated_signatures_verification();
425445
ethlambda_crypto::verify_aggregated_signature(
426446
&aggregated.proof.proof,
427447
pubkeys,
428448
&data_root,
429449
slot,
430450
)
451+
.map_err(StoreError::AggregateVerificationFailed)?;
431452
}
432-
.map_err(StoreError::AggregateVerificationFailed)?;
433453

434454
// Read stats before moving the proof into the store.
435455
let num_participants = aggregated.proof.participants.count_ones();

crates/blockchain/tests/forkchoice_spectests.rs

Lines changed: 18 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -41,6 +41,10 @@ fn run(path: &Path) -> datatest_stable::Result<()> {
4141
}
4242
println!("Running test: {}", name);
4343

44+
// Mocked-proof vectors (`proofSetting == 0`) carry placeholder
45+
// aggregation proofs that must not be cryptographically verified.
46+
let proofs_are_mocked = test.proofs_are_mocked();
47+
4448
// Initialize store from anchor state/block.
4549
//
4650
// Fixtures whose `steps` is empty are "anchor rejection" cases (e.g.
@@ -93,11 +97,14 @@ fn run(path: &Path) -> datatest_stable::Result<()> {
9397

9498
let signed_block = block_data.to_blank_signed_block();
9599

96-
let block_time_ms =
97-
genesis_time * 1000 + signed_block.message.slot * MILLISECONDS_PER_SLOT;
98-
100+
// Advance time to the block's slot unless the test delivers
101+
// the block ahead of the store clock.
99102
// NOTE: the has_proposal argument is set to true, following the spec
100-
store::on_tick(&mut store, block_time_ms, true);
103+
if step.tick_to_slot {
104+
let block_time_ms =
105+
genesis_time * 1000 + signed_block.message.slot * MILLISECONDS_PER_SLOT;
106+
store::on_tick(&mut store, block_time_ms, true);
107+
}
101108
let result = store::on_block_without_verification(&mut store, signed_block);
102109
let import_ok = result.is_ok();
103110
assert_step_outcome(step_idx, step.valid, result)?;
@@ -184,7 +191,13 @@ fn run(path: &Path) -> datatest_stable::Result<()> {
184191
TypeOneMultiSignature::new(proof_fixture.participants.into(), proof_data);
185192
let aggregated = SignedAggregatedAttestation { data, proof };
186193

187-
let result = store::on_gossip_aggregated_attestation(&mut store, aggregated);
194+
let result = if proofs_are_mocked {
195+
store::on_gossip_aggregated_attestation_without_verification(
196+
&mut store, aggregated,
197+
)
198+
} else {
199+
store::on_gossip_aggregated_attestation(&mut store, aggregated)
200+
};
188201
assert_step_outcome(step_idx, step.valid, result)?;
189202
}
190203
other => {

crates/common/test-fixtures/src/fork_choice.rs

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -44,13 +44,30 @@ pub struct ForkChoiceTest {
4444
#[serde(rename = "anchorBlock")]
4545
pub anchor_block: Block,
4646
pub steps: Vec<ForkChoiceStep>,
47+
/// Aggregation proof regime: 0 = mocked (placeholder bytes, must not be
48+
/// verified), 1 = real and must verify, 2 = real and must fail
49+
/// verification. Older fixtures lack the field and carry real proofs.
50+
#[serde(rename = "proofSetting", default = "default_proof_setting")]
51+
pub proof_setting: u8,
4752
#[serde(rename = "maxSlot")]
4853
#[allow(dead_code)]
4954
pub max_slot: u64,
5055
#[serde(rename = "_info")]
5156
pub info: TestInfo,
5257
}
5358

59+
fn default_proof_setting() -> u8 {
60+
1
61+
}
62+
63+
impl ForkChoiceTest {
64+
/// Whether the vector's aggregation proofs are placeholders that must
65+
/// not be cryptographically verified (`proofSetting == 0`).
66+
pub fn proofs_are_mocked(&self) -> bool {
67+
self.proof_setting == 0
68+
}
69+
}
70+
5471
// ============================================================================
5572
// Step Types
5673
// ============================================================================
@@ -76,6 +93,11 @@ pub struct ForkChoiceStep {
7693
pub has_proposal: Option<bool>,
7794
#[serde(rename = "isAggregator")]
7895
pub is_aggregator: Option<bool>,
96+
/// Whether the harness must advance the store clock to the block's slot
97+
/// before delivering a `block` step. Early-arrival tests set this to
98+
/// `false` to deliver the block ahead of the store clock.
99+
#[serde(rename = "tickToSlot", default = "default_true")]
100+
pub tick_to_slot: bool,
79101
}
80102

81103
fn default_true() -> bool {

crates/common/test-fixtures/src/state_transition.rs

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,6 @@ use serde::Deserialize;
1919
pub struct StateTransitionRunRequest {
2020
pub pre: TestState,
2121
pub blocks: Vec<Block>,
22-
#[serde(default, rename = "expectException")]
22+
#[serde(default, rename = "expectException", alias = "rejectionReason")]
2323
pub expect_exception: Option<String>,
2424
}

crates/common/test-fixtures/src/verify_signatures.rs

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -44,7 +44,10 @@ pub struct VerifySignaturesTest {
4444
pub anchor_state: TestState,
4545
#[serde(rename = "signedBlock")]
4646
pub signed_block: TestSignedBlock,
47-
#[serde(rename = "expectException")]
47+
/// Expected rejection, when present. Newer fixtures name this field
48+
/// `rejectionReason` (leanSpec replaced `expectException`); both
49+
/// spellings are accepted.
50+
#[serde(default, rename = "expectException", alias = "rejectionReason")]
4851
pub expect_exception: Option<String>,
4952
#[serde(rename = "_info")]
5053
#[allow(dead_code)]

crates/common/types/tests/ssz_spectests.rs

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ use ethlambda_types::primitives::HashTreeRoot;
55
mod ssz_types;
66
use ssz_types::{SszTestCase, SszTestVector, decode_hex, decode_hex_h256};
77

8-
const SUPPORTED_FIXTURE_FORMAT: &str = "ssz";
8+
const SUPPORTED_FIXTURE_FORMAT: &str = "ssz_test";
99

1010
fn run(path: &Path) -> datatest_stable::Result<()> {
1111
let tests = SszTestVector::from_file(path)?;

crates/net/rpc/src/test_driver.rs

Lines changed: 7 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -356,10 +356,13 @@ fn apply_step(store: &mut Store, step: ForkChoiceStep) -> Result<(), String> {
356356
.ok_or_else(|| "block step missing block data".to_string())?;
357357
let signed_block = block_data.to_blank_signed_block();
358358
// Match the spec-test runner: advance time to the block's slot
359-
// before importing so the future-slot guard doesn't reject it.
360-
let block_time_ms = store.config().genesis_time * 1000
361-
+ signed_block.message.slot * MILLISECONDS_PER_SLOT;
362-
store::on_tick(store, block_time_ms, true);
359+
// before importing, unless the step delivers the block ahead of
360+
// the store clock.
361+
if step.tick_to_slot {
362+
let block_time_ms = store.config().genesis_time * 1000
363+
+ signed_block.message.slot * MILLISECONDS_PER_SLOT;
364+
store::on_tick(store, block_time_ms, true);
365+
}
363366
store::on_block_without_verification(store, signed_block).map_err(|e| e.to_string())
364367
}
365368
"attestation" => {

0 commit comments

Comments
 (0)