Skip to content

Commit 2455ce8

Browse files
committed
feat(aggregation): rank subnet candidates by tier alone off the proposer path
A committee aggregator that does not propose the next slot no longer gives this slot's groups a queue jump over stale ones. Those groups are the committee's view-merge payload: the next slot's proposer folds them itself off the global heartbeat topic, and every other node already receives them raw there. Spending an aggregator's two scarce leanVM jobs on recency therefore bought the network nothing while starving the one thing only an aggregator can produce, justification and finalization progress. SlotOrdering::TierOnly drops the recency bucket for that role, leaving Finalize > Justify > Build to decide, so a current-slot group is aggregated only when it wins on consensus value. The proposer's subnet fallback keeps CurrentSlotFirst: the block it is about to build wants this slot's committee covered.
1 parent a375883 commit 2455ce8

3 files changed

Lines changed: 197 additions & 53 deletions

File tree

CLAUDE.md

Lines changed: 12 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -77,10 +77,10 @@ line up with other clients even while the interval grid inside the slot diverges
7777
One session per slot, started at interval 2 or earlier once a threshold is met.
7878
Two roles trigger it:
7979

80-
| role | early threshold | jobs |
81-
|---|---|---|
82-
| committee aggregator | 2/3 of signatures expected from subscribed subnets | up to `MAX_AGGREGATION_JOBS` from the subnet pool |
83-
| next slot's proposer | `ceil(3K'/4)` heartbeat votes (the safe-target threshold) | exactly 1, over the heartbeat committee votes |
80+
| role | early threshold | jobs | subnet ordering |
81+
|---|---|---|---|
82+
| committee aggregator | 2/3 of signatures expected from subscribed subnets | up to `MAX_AGGREGATION_JOBS` from the subnet pool | `SlotOrdering::TierOnly` |
83+
| next slot's proposer | `ceil(3K'/4)` heartbeat votes (the safe-target threshold) | exactly 1, over the heartbeat committee votes | `SlotOrdering::CurrentSlotFirst` (fallback only) |
8484

8585
The proposer's job is built by `heartbeat_fold::heartbeat_aggregation_snapshot`,
8686
which picks the `AttestationData` with the most buffered committee signers and
@@ -90,6 +90,14 @@ nothing is foldable. The result flows through the ordinary `AggregateProduced` p
9090
into `new_payloads`, is promoted at interval 3, and reaches the builder as one
9191
candidate among many; `Tier::Heartbeat` is what makes it win.
9292

93+
Heartbeat signatures are the proposer's alone: `heartbeat_aggregation_snapshot` is
94+
the only reader of that buffer. An aggregator that does not propose the next slot
95+
sees the same committee votes only where they duplicate into its subnet pool, and
96+
`SlotOrdering::TierOnly` denies them the recency bucket there, so they are
97+
aggregated only when they win on consensus value (Finalize > Justify > Build).
98+
Recency is worth a queue jump only to the proposer, which is the one node that has
99+
to pack those votes; everyone else already has them raw off the global topic.
100+
93101
`K` is `HEARTBEAT_COMMITTEE_SIZE` from the genesis config (default 16); `K' = min(K, n)`
94102
and every threshold is denominated in `K'`, never the raw `K`. `N` is
95103
`RLMD_LOOKBACK_LIMIT` (8). Heartbeat votes ride in `body.attestations` — there is no

crates/blockchain/src/aggregation.rs

Lines changed: 161 additions & 44 deletions
Original file line numberDiff line numberDiff line change
@@ -25,8 +25,8 @@
2525
//! up-front store pass resolves every candidate `AttestationData`'s
2626
//! aggregation material once (raw-first + trim, see [`resolve_job`]), then a
2727
//! pure in-memory loop scores and orders candidates by consensus value
28-
//! (current-slot before stale, then Finalize > Justify > Build), emitting at
29-
//! most `max_jobs` jobs.
28+
//! (Finalize > Justify > Build, with this slot's groups jumping the queue only
29+
//! for the proposer — see [`SlotOrdering`]), emitting at most `max_jobs` jobs.
3030
3131
use std::collections::{HashMap, HashSet};
3232
use std::time::{Duration, Instant, SystemTime};
@@ -94,6 +94,32 @@ pub enum JobSource {
9494
Heartbeat,
9595
}
9696

97+
/// How a session ranks this slot's candidate groups against stale ones.
98+
///
99+
/// This slot's groups are the committee's view-merge payload: the next slot's
100+
/// proposer needs them, and every other node already receives them raw on the
101+
/// global heartbeat topic. So recency is worth a queue jump only to the proposer,
102+
/// which is why the two roles order candidates differently.
103+
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
104+
pub enum SlotOrdering {
105+
/// Current-slot groups precede stale ones; tier decides within a bucket.
106+
///
107+
/// The next slot's proposer's ordering. It reaches the subnet pool only as a
108+
/// fallback (nothing foldable from the heartbeat topic), and the one job it
109+
/// spends there is still meant to cover this slot's committee, since that is
110+
/// what `Tier::Heartbeat` packs into the block it is about to build.
111+
CurrentSlotFirst,
112+
/// Tier alone decides: Finalize > Justify > Build, no recency bucket.
113+
///
114+
/// A committee aggregator that does not propose the next slot. Its scarce
115+
/// resource is leanVM time, and its contribution to the network is pushing
116+
/// targets over 2/3 — not re-publishing votes the heartbeat topic already
117+
/// delivered. Under this ordering a current-slot group is aggregated only when
118+
/// it wins on consensus value: it finalizes, it justifies, or it adds coverage
119+
/// no other candidate does.
120+
TierOnly,
121+
}
122+
97123
/// A single pre-prepared aggregation group.
98124
///
99125
/// Built on the actor thread from a store snapshot; consumed by an off-thread
@@ -226,21 +252,24 @@ pub(crate) const MAX_AGGREGATION_JOBS: usize = 2;
226252
/// at least two existing proofs to merge).
227253
/// 2. **Greedy loop**, at most `max_jobs` rounds: each round
228254
/// scores every unselected candidate against the projected state and
229-
/// keeps the lowest ordering key (current-slot before stale, then
230-
/// Finalize > Justify > Build, mirroring the block builder). The winning
231-
/// [`AggregationJob`] is emitted as-is; the projection is updated with its
232-
/// realized coverage.
255+
/// keeps the lowest ordering key (Finalize > Justify > Build, mirroring the
256+
/// block builder, behind whatever recency bucket `slot_ordering` asks for).
257+
/// The winning [`AggregationJob`] is emitted as-is; the projection is updated
258+
/// with its realized coverage.
233259
///
234260
/// Stops early when no remaining candidate scores (converged).
235261
///
236-
/// `max_jobs` is [`MAX_AGGREGATION_JOBS`] for an ordinary session. A session run
237-
/// because we propose the next slot prefers a single heartbeat job instead
262+
/// `max_jobs` is [`MAX_AGGREGATION_JOBS`] for an ordinary session, run with
263+
/// [`SlotOrdering::TierOnly`]. A session run because we propose the next slot
264+
/// prefers a single heartbeat job instead
238265
/// ([`crate::heartbeat_fold::heartbeat_aggregation_snapshot`]) and only falls back
239-
/// here, with `max_jobs = 1`, when nothing is foldable.
266+
/// here, with `max_jobs = 1` and [`SlotOrdering::CurrentSlotFirst`], when nothing
267+
/// is foldable.
240268
pub fn snapshot_aggregation_inputs(
241269
store: &Store,
242270
current_slot: u64,
243271
max_jobs: usize,
272+
slot_ordering: SlotOrdering,
244273
) -> Option<AggregationSnapshot> {
245274
let gossip_groups = store.iter_gossip_signatures();
246275
let new_payload_keys = store.new_payload_keys();
@@ -318,6 +347,7 @@ pub fn snapshot_aggregation_inputs(
318347
&extended_historical_block_hashes,
319348
current_slot,
320349
validator_count,
350+
slot_ordering,
321351
) else {
322352
trace!(
323353
jobs_selected = jobs.len(),
@@ -368,15 +398,16 @@ pub fn snapshot_aggregation_inputs(
368398
/// voters (relative to the candidate's realized [`AggregationJob::coverage`],
369399
/// not the full proof union — see [`resolve_job`]). Among the rest, returns
370400
/// `(data_root, score)` for the entry with the lowest composite key:
371-
/// current-slot groups precede stale ones, then `EntryScore::ordering_key`
372-
/// (tier, then tier-dependent dims, then `data_root`) decides.
401+
/// `slot_ordering`'s recency bucket, then `EntryScore::ordering_key` (tier, then
402+
/// tier-dependent dims, then `data_root`).
373403
fn pick_best_candidate(
374404
candidates: &HashMap<H256, AggregationJob>,
375405
projected: &block_builder::ProjectedState,
376406
known_block_roots: &HashSet<H256>,
377407
extended_historical_block_hashes: &[H256],
378408
current_slot: u64,
379409
validator_count: usize,
410+
slot_ordering: SlotOrdering,
380411
) -> Option<(H256, EntryScore)> {
381412
let mut best: Option<(H256, EntryScore)> = None;
382413
let mut best_key: Option<(u8, block_builder::OrderingKey)> = None;
@@ -399,11 +430,11 @@ fn pick_best_candidate(
399430
continue;
400431
};
401432

402-
// Current-slot groups always precede stale ones (goal: consider
403-
// current-slot signatures first); within a bucket, `EntryScore`
404-
// decides.
405-
let slot_bucket: u8 = if att_data.slot == current_slot { 0 } else { 1 };
406-
let candidate_key = candidate_ordering_key(slot_bucket, &score, *data_root);
433+
let candidate_key = candidate_ordering_key(
434+
slot_bucket(slot_ordering, att_data.slot, current_slot),
435+
&score,
436+
*data_root,
437+
);
407438
if best_key.as_ref().is_none_or(|k| candidate_key < *k) {
408439
best = Some((*data_root, score));
409440
best_key = Some(candidate_key);
@@ -413,9 +444,21 @@ fn pick_best_candidate(
413444
best
414445
}
415446

416-
/// Composite ordering key (lower is better): current-slot groups (`0`)
417-
/// precede stale ones (`1`); within a bucket, `EntryScore::ordering_key`
418-
/// (tier, then tier-dependent dims, then `data_root`) decides.
447+
/// Recency bucket for a candidate (lower is better), per [`SlotOrdering`].
448+
///
449+
/// [`SlotOrdering::TierOnly`] collapses every candidate into bucket `0`, so the
450+
/// composite key degenerates to `EntryScore::ordering_key` alone and a current-slot
451+
/// group has to out-tier a stale one to be picked.
452+
fn slot_bucket(slot_ordering: SlotOrdering, att_slot: u64, current_slot: u64) -> u8 {
453+
match slot_ordering {
454+
SlotOrdering::CurrentSlotFirst => u8::from(att_slot != current_slot),
455+
SlotOrdering::TierOnly => 0,
456+
}
457+
}
458+
459+
/// Composite ordering key (lower is better): the [`slot_bucket`] leads; within a
460+
/// bucket, `EntryScore::ordering_key` (tier, then tier-dependent dims, then
461+
/// `data_root`) decides.
419462
fn candidate_ordering_key(
420463
slot_bucket: u8,
421464
score: &EntryScore,
@@ -1011,16 +1054,27 @@ mod tests {
10111054

10121055
// ---- ordering ----
10131056

1014-
/// The slot bucket dominates the within-bucket score: a current-slot
1015-
/// candidate is picked ahead of a stale candidate that has *more* new
1016-
/// voters (which, absent the bucket, would win the Build-tier
1017-
/// `new_voters` dimension). Exercises `candidate_ordering_key` through the
1018-
/// real `pick_best_candidate` path rather than constructing an
1019-
/// `EntryScore` directly.
1020-
#[test]
1021-
fn pick_best_candidate_prefers_current_slot_over_higher_stale_score() {
1022-
const NUM_VALIDATORS: usize = 100;
1023-
const CURRENT_SLOT: u64 = 3;
1057+
const ORDERING_NUM_VALIDATORS: usize = 100;
1058+
const ORDERING_CURRENT_SLOT: u64 = 3;
1059+
1060+
/// Everything `pick_best_candidate` needs for the two ordering tests.
1061+
struct OrderingFixture {
1062+
candidates: HashMap<H256, AggregationJob>,
1063+
projected: block_builder::ProjectedState,
1064+
known_block_roots: HashSet<H256>,
1065+
historical_block_hashes: Vec<H256>,
1066+
root_current: H256,
1067+
root_stale: H256,
1068+
}
1069+
1070+
/// One current-slot candidate with a single new voter against one stale
1071+
/// candidate with five, on independent target roots so their voter buckets
1072+
/// never interact. Both score `Tier::Build`, so the stale one wins the
1073+
/// within-tier `new_voters` dimension and only the recency bucket can flip
1074+
/// the outcome — which is exactly what the two [`SlotOrdering`] variants
1075+
/// disagree about.
1076+
fn ordering_fixture() -> OrderingFixture {
1077+
const CURRENT_SLOT: u64 = ORDERING_CURRENT_SLOT;
10241078

10251079
let genesis_root = H256([1u8; 32]);
10261080
let target_root = H256([7u8; 32]);
@@ -1095,23 +1149,72 @@ mod tests {
10951149
current_votes: HashMap::new(),
10961150
};
10971151

1152+
OrderingFixture {
1153+
candidates,
1154+
projected,
1155+
known_block_roots,
1156+
historical_block_hashes,
1157+
root_current,
1158+
root_stale,
1159+
}
1160+
}
1161+
1162+
/// Under [`SlotOrdering::CurrentSlotFirst`] the recency bucket dominates the
1163+
/// within-bucket score: the current-slot candidate is picked ahead of a stale
1164+
/// candidate that has *more* new voters. Exercises `candidate_ordering_key`
1165+
/// through the real `pick_best_candidate` path rather than constructing an
1166+
/// `EntryScore` directly.
1167+
#[test]
1168+
fn pick_best_candidate_prefers_current_slot_over_higher_stale_score() {
1169+
let fixture = ordering_fixture();
1170+
10981171
let (picked_root, score) = pick_best_candidate(
1099-
&candidates,
1100-
&projected,
1101-
&known_block_roots,
1102-
&historical_block_hashes,
1103-
CURRENT_SLOT,
1104-
NUM_VALIDATORS,
1172+
&fixture.candidates,
1173+
&fixture.projected,
1174+
&fixture.known_block_roots,
1175+
&fixture.historical_block_hashes,
1176+
ORDERING_CURRENT_SLOT,
1177+
ORDERING_NUM_VALIDATORS,
1178+
SlotOrdering::CurrentSlotFirst,
11051179
)
11061180
.expect("both candidates are viable Build-tier entries");
11071181

11081182
assert_eq!(score.tier, block_builder::Tier::Build);
11091183
assert_eq!(
1110-
picked_root, root_current,
1184+
picked_root, fixture.root_current,
11111185
"the current-slot group must be picked ahead of a stale group with more new voters"
11121186
);
11131187
}
11141188

1189+
/// Under [`SlotOrdering::TierOnly`] recency buys nothing: the same pool picks
1190+
/// the stale candidate, because it wins Build tier's `new_voters` dimension.
1191+
///
1192+
/// This is what keeps a non-proposing aggregator's jobs on consensus value
1193+
/// rather than on the committee's view-merge payload — those votes reach every
1194+
/// peer raw on the global heartbeat topic, so re-proving them buys the network
1195+
/// nothing an aggregator alone could provide.
1196+
#[test]
1197+
fn pick_best_candidate_tier_only_ignores_slot_recency() {
1198+
let fixture = ordering_fixture();
1199+
1200+
let (picked_root, score) = pick_best_candidate(
1201+
&fixture.candidates,
1202+
&fixture.projected,
1203+
&fixture.known_block_roots,
1204+
&fixture.historical_block_hashes,
1205+
ORDERING_CURRENT_SLOT,
1206+
ORDERING_NUM_VALIDATORS,
1207+
SlotOrdering::TierOnly,
1208+
)
1209+
.expect("both candidates are viable Build-tier entries");
1210+
1211+
assert_eq!(score.tier, block_builder::Tier::Build);
1212+
assert_eq!(
1213+
picked_root, fixture.root_stale,
1214+
"with no recency bucket the higher-coverage group wins, current-slot or not"
1215+
);
1216+
}
1217+
11151218
// ---- projection ----
11161219

11171220
/// Two candidates targeting the same root accumulate coverage: the
@@ -1196,6 +1299,7 @@ mod tests {
11961299
&historical_block_hashes,
11971300
999,
11981301
NUM_VALIDATORS,
1302+
SlotOrdering::TierOnly,
11991303
)
12001304
.expect("round 1 should find a candidate");
12011305
assert_eq!(picked_root, root_a);
@@ -1219,6 +1323,7 @@ mod tests {
12191323
&historical_block_hashes,
12201324
999,
12211325
NUM_VALIDATORS,
1326+
SlotOrdering::TierOnly,
12221327
)
12231328
.expect("round 2 should find B");
12241329
assert_eq!(picked_root, root_b);
@@ -1237,7 +1342,9 @@ mod tests {
12371342
fn snapshot_returns_none_for_empty_store() {
12381343
let hashes = vec![H256([1u8; 32])];
12391344
let store = new_test_store(make_head_state(0, 4, &hashes));
1240-
assert!(snapshot_aggregation_inputs(&store, 0, MAX_AGGREGATION_JOBS).is_none());
1345+
let snapshot =
1346+
snapshot_aggregation_inputs(&store, 0, MAX_AGGREGATION_JOBS, SlotOrdering::TierOnly);
1347+
assert!(snapshot.is_none());
12411348
}
12421349

12431350
/// A single gossip signature with no other material to merge is dropped
@@ -1266,7 +1373,9 @@ mod tests {
12661373
let hashed = HashedAttestationData::new(att_data);
12671374
store.insert_gossip_signature(hashed, 0, dummy_sig());
12681375

1269-
assert!(snapshot_aggregation_inputs(&store, 0, MAX_AGGREGATION_JOBS).is_none());
1376+
let snapshot =
1377+
snapshot_aggregation_inputs(&store, 0, MAX_AGGREGATION_JOBS, SlotOrdering::TierOnly);
1378+
assert!(snapshot.is_none());
12701379
}
12711380

12721381
/// A group whose target is already justified (here: at or behind the
@@ -1309,7 +1418,8 @@ mod tests {
13091418
store.insert_gossip_signature(hashed, 1, dummy_sig());
13101419

13111420
assert!(
1312-
snapshot_aggregation_inputs(&store, 999, MAX_AGGREGATION_JOBS).is_none(),
1421+
snapshot_aggregation_inputs(&store, 999, MAX_AGGREGATION_JOBS, SlotOrdering::TierOnly)
1422+
.is_none(),
13131423
"a group targeting an already-justified slot must never become a job"
13141424
);
13151425
}
@@ -1365,8 +1475,13 @@ mod tests {
13651475
store.insert_gossip_signature(hashed.clone(), 0, dummy_sig());
13661476
store.insert_gossip_signature(hashed, 1, dummy_sig());
13671477

1368-
let snapshot = snapshot_aggregation_inputs(&store, HEAD_SLOT, MAX_AGGREGATION_JOBS)
1369-
.expect("a vote for the current head must produce a job (chain view covers the tip)");
1478+
let snapshot = snapshot_aggregation_inputs(
1479+
&store,
1480+
HEAD_SLOT,
1481+
MAX_AGGREGATION_JOBS,
1482+
SlotOrdering::TierOnly,
1483+
)
1484+
.expect("a vote for the current head must produce a job (chain view covers the tip)");
13701485
assert_eq!(snapshot.jobs.len(), 1);
13711486
assert_eq!(
13721487
snapshot.jobs[0].hashed.data().target.slot,
@@ -1426,8 +1541,9 @@ mod tests {
14261541
fn snapshot_caps_jobs_at_max_aggregation_jobs() {
14271542
let store = store_with_competing_build_tier_groups();
14281543

1429-
let snapshot = snapshot_aggregation_inputs(&store, 999, MAX_AGGREGATION_JOBS)
1430-
.expect("should produce jobs");
1544+
let snapshot =
1545+
snapshot_aggregation_inputs(&store, 999, MAX_AGGREGATION_JOBS, SlotOrdering::TierOnly)
1546+
.expect("should produce jobs");
14311547
assert_eq!(snapshot.groups_considered, NUM_GROUPS);
14321548
assert_eq!(snapshot.jobs.len(), MAX_AGGREGATION_JOBS);
14331549

@@ -1452,7 +1568,8 @@ mod tests {
14521568
fn snapshot_caps_jobs_at_one_for_proposer() {
14531569
let store = store_with_competing_build_tier_groups();
14541570

1455-
let snapshot = snapshot_aggregation_inputs(&store, 999, 1).expect("should produce a job");
1571+
let snapshot = snapshot_aggregation_inputs(&store, 999, 1, SlotOrdering::CurrentSlotFirst)
1572+
.expect("should produce a job");
14561573
assert_eq!(snapshot.groups_considered, NUM_GROUPS);
14571574
assert_eq!(snapshot.jobs.len(), 1);
14581575
assert_eq!(

0 commit comments

Comments
 (0)