Skip to content

Commit 233cc94

Browse files
committed
feat(cli): add offline block-building benchmark sub-command
`ethlambda benchmark synthetic` measures block building exactly as executed when the node proposes, against a reproducible synthetic workload, with no devnet required. "Optimize block building" (#465) is the top roadmap item, but the only observability today is Prometheus histograms on a live devnet: noisy, not reproducible, and unable to compare an optimization against a baseline. The harness drives the production proposer entry point (`produce_block_with_signatures`) over a seeded in-memory chain, seeding the pending pool per slot and letting the proposal tick promote it, as on a live node. Per-phase timings come from the existing `lean_block_proposal_attestation_build_phase_seconds` histogram: the per-label sample sums are deltaed between iterations, so attribution is exact and the hot path is untouched. Each iteration reports its block root, so a baseline-vs-optimized diff proves an optimization changed only speed and not attestation selection. Dispatch goes through the `benchmark` token that command.rs already strips, parsed by a `clap::Parser` of its own, so the harness arguments never enter CliOptions and the node parser keeps the exact shape it has today. `main` becomes synchronous and only the node path enters the tokio runtime: the benchmark is synchronous CPU-bound work and would otherwise park a worker thread for its whole run. Its logs go to stderr so the report owns stdout. NEW_PAYLOAD_CAP becomes public so the harness can reject a --proofs-per-data batch the pending pool would evict whole. `make bench` runs it. Reports one row per measured iteration. Aggregate statistics, build provenance and machine-readable output follow separately, as does the real-crypto workload — see docs/plans/block-building-benchmark.md for the milestones.
1 parent e518c7b commit 233cc94

10 files changed

Lines changed: 727 additions & 27 deletions

File tree

Cargo.lock

Lines changed: 1 addition & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

Makefile

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
.PHONY: help fmt lint docker-build shadow-build shadow-docker-build run-devnet test docs docs-deps docs-serve
1+
.PHONY: help fmt lint bench docker-build shadow-build shadow-docker-build run-devnet test docs docs-deps docs-serve
22

33
help: ## 📚 Show help for each of the Makefile recipes
44
@grep -E '^[a-zA-Z0-9_-]+:.*?## .*$$' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-30s\033[0m %s\n", $$1, $$2}'
@@ -14,6 +14,11 @@ test: leanSpec/fixtures ## 🧪 Run all tests
1414
# signature verification/aggregation, without paying for LTO on every rebuild
1515
cargo test --workspace --profile release-fast
1616

17+
BENCH_ARGS ?= synthetic --mock-crypto
18+
19+
bench: ## 🏁 Benchmark block building offline (override BENCH_ARGS to customize)
20+
cargo run --release --bin ethlambda -- benchmark $(BENCH_ARGS)
21+
1722
GIT_COMMIT=$(shell git rev-parse HEAD)
1823
GIT_BRANCH=$(shell git rev-parse --abbrev-ref HEAD)
1924
DOCKER_TAG?=local

bin/ethlambda/Cargo.toml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,7 @@ shadow-integration = ["ethlambda-crypto/shadow-integration"]
2121
[dependencies]
2222
ethlambda-blockchain.workspace = true
2323
ethlambda-crypto.workspace = true
24+
ethlambda-metrics.workspace = true
2425
ethlambda-network-api.workspace = true
2526
ethlambda-p2p.workspace = true
2627
ethlambda-types.workspace = true
Lines changed: 145 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,145 @@
1+
//! Synthetic benchmark corpus: deterministic validators, a genesis store, and
2+
//! per-slot attestation-pool seeding.
3+
4+
use std::sync::Arc;
5+
6+
use ethlambda_blockchain::store::produce_attestation_data;
7+
use ethlambda_storage::{Store, backend::InMemoryBackend};
8+
use ethlambda_types::{
9+
attestation::{AggregationBits, HashedAttestationData},
10+
block::SingleMessageAggregate,
11+
state::{State, Validator, ValidatorPubkeyBytes},
12+
};
13+
14+
/// Fixed genesis time for synthetic runs. The harness derives every tick
15+
/// timestamp from slot numbers relative to this value and never reads the wall
16+
/// clock, so runs are reproducible at any time of day.
17+
const GENESIS_TIME: u64 = 1_700_000_000;
18+
19+
pub(crate) struct SyntheticCorpus {
20+
num_validators: u64,
21+
proofs_per_data: u64,
22+
}
23+
24+
impl SyntheticCorpus {
25+
pub(crate) fn new(num_validators: u64, proofs_per_data: u64) -> Self {
26+
Self {
27+
num_validators,
28+
proofs_per_data,
29+
}
30+
}
31+
32+
/// Build a genesis store over an in-memory backend with `num_validators`
33+
/// seed-derived validators.
34+
///
35+
/// Pubkeys are deterministic placeholder bytes: in mock-crypto mode no code
36+
/// path decodes them (signature verification is skipped and best-proof
37+
/// compaction never resolves pubkeys).
38+
pub(crate) fn genesis_store(&self, seed: u64) -> Store {
39+
let mut rng_state = seed;
40+
let validators = (0..self.num_validators)
41+
.map(|index| Validator {
42+
attestation_pubkey: synthetic_pubkey(&mut rng_state),
43+
proposal_pubkey: synthetic_pubkey(&mut rng_state),
44+
index,
45+
})
46+
.collect();
47+
let genesis_state = State::from_genesis(GENESIS_TIME, validators);
48+
Store::from_anchor_state(Arc::new(InMemoryBackend::new()), genesis_state)
49+
}
50+
51+
/// Seed the pending ("new") pool with the full validator set's attestations
52+
/// for `attestation_slot`, split into `proofs_per_data` disjoint aggregates.
53+
///
54+
/// Mirrors what committee aggregators gossip during a slot: several
55+
/// aggregates for the same `AttestationData`, each covering a validator
56+
/// subset. The proposal tick then promotes them to the known pool, exactly
57+
/// as on a live node. Entries are inserted in a fixed order because pool
58+
/// insertion order pins within-entry proof choice during selection.
59+
pub(crate) fn seed_pool(&self, store: &mut Store, attestation_slot: u64) {
60+
let data = produce_attestation_data(store, attestation_slot);
61+
let entries = participant_groups(self.num_validators, self.proofs_per_data)
62+
.into_iter()
63+
.map(|participants| {
64+
(
65+
HashedAttestationData::new(data.clone()),
66+
SingleMessageAggregate::empty(participants),
67+
)
68+
})
69+
.collect();
70+
store.insert_new_aggregated_payloads_batch(entries);
71+
}
72+
}
73+
74+
/// Partition validators 0..num_validators into `groups` disjoint bitfields,
75+
/// assigning validator `i` to group `i % groups`. Every group is non-empty
76+
/// (groups is capped at the validator count) and the union covers every
77+
/// validator exactly once.
78+
fn participant_groups(num_validators: u64, groups: u64) -> Vec<AggregationBits> {
79+
let groups = groups.clamp(1, num_validators);
80+
(0..groups)
81+
.map(|group| {
82+
let mut bits = AggregationBits::with_length(num_validators as usize)
83+
.expect("validator count is within the bitlist limit");
84+
for index in (group..num_validators).step_by(groups as usize) {
85+
bits.set(index as usize, true)
86+
.expect("index is within the bitlist length");
87+
}
88+
bits
89+
})
90+
.collect()
91+
}
92+
93+
/// splitmix64: tiny deterministic generator for placeholder pubkey bytes,
94+
/// avoiding a rand dependency.
95+
fn splitmix64(state: &mut u64) -> u64 {
96+
*state = state.wrapping_add(0x9e37_79b9_7f4a_7c15);
97+
let mut z = *state;
98+
z = (z ^ (z >> 30)).wrapping_mul(0xbf58_476d_1ce4_e5b9);
99+
z = (z ^ (z >> 27)).wrapping_mul(0x94d0_49bb_1331_11eb);
100+
z ^ (z >> 31)
101+
}
102+
103+
fn synthetic_pubkey(rng_state: &mut u64) -> ValidatorPubkeyBytes {
104+
let mut bytes = [0u8; 52];
105+
for chunk in bytes.chunks_mut(8) {
106+
let word = splitmix64(rng_state).to_le_bytes();
107+
chunk.copy_from_slice(&word[..chunk.len()]);
108+
}
109+
bytes
110+
}
111+
112+
#[cfg(test)]
113+
mod tests {
114+
use super::*;
115+
use ethlambda_types::attestation::validator_indices;
116+
117+
#[test]
118+
fn participant_groups_partition_all_validators() {
119+
for (validators, groups) in [(8u64, 2u64), (8, 3), (5, 8), (1, 1), (4096, 4)] {
120+
let partition = participant_groups(validators, groups);
121+
assert_eq!(partition.len() as u64, groups.min(validators));
122+
let mut seen = vec![0u32; validators as usize];
123+
for bits in &partition {
124+
let indices: Vec<u64> = validator_indices(bits).collect();
125+
assert!(!indices.is_empty(), "every group must be non-empty");
126+
for index in indices {
127+
seen[index as usize] += 1;
128+
}
129+
}
130+
assert!(
131+
seen.iter().all(|&count| count == 1),
132+
"every validator must appear in exactly one group: {seen:?}"
133+
);
134+
}
135+
}
136+
137+
#[test]
138+
fn synthetic_pubkeys_are_deterministic() {
139+
let mut a = 42u64;
140+
let mut b = 42u64;
141+
assert_eq!(synthetic_pubkey(&mut a), synthetic_pubkey(&mut b));
142+
let mut c = 43u64;
143+
assert_ne!(synthetic_pubkey(&mut a), synthetic_pubkey(&mut c));
144+
}
145+
}

0 commit comments

Comments
 (0)