Skip to content

Commit 8ee2c79

Browse files
MegaRedHandpablodeymodicethedev
authored
feat: add devnet 5 support (#378)
## 🗒️ Description / Motivation Adds support for the `pq-devnet-5` spec, whose headline feature is the new two-tier signature aggregation scheme from [leanSpec PR #717](leanEthereum/leanSpec#717): per-attestation **Type-1** multi-signatures that get merged into a single block-level **Type-2** proof. Instead of a block carrying one aggregated proof per attestation plus a proposer signature, a `SignedBlock` now carries a single merged proof binding every signature it depends on. Closes #285. ## What Changed **Wire format / types** (`crates/common/types`) - `SignedBlock.signature: BlockSignatures` → `SignedBlock.proof: ByteList512KiB`, an SSZ-encoded `TypeTwoMultiSignature` envelope. Helpers `merged_proof_bytes()` / `wrap_merged_proof()` handle the 4-byte SSZ offset header. - `AggregatedSignatureProof` is replaced by `TypeOneMultiSignature` (flat container per leanSpec PR #717). `BlockSignatures` and `AttestationSignatures` are gone. **Crypto** (`crates/common/crypto`) - New leanVM operations: `merge_type_1s_into_type_2` (block building), `verify_type_2_signature` (block import), and `split_type_2_by_message` (recovering per-attestation Type-1 proofs from a merged proof). **Blockchain** - Block builder produces the merged Type-2 proof from the pooled Type-1 proofs; block import verifies it. - New `reaggregate.rs` module ports leanSpec's `SyncService._deconstruct_block_into_store`: after importing a block, it SNARK-splits the merged proof back into per-attestation Type-1 proofs and folds them into the local aggregated-payload pool, so block-borne votes can be republished on gossip. Splits are bounded (only when in sync, skip already-justified targets, skip participant subsets, max `MAX_REAGGREGATIONS_PER_BLOCK` splits per block) since each split runs a fresh SNARK. - `process_block` now reports whether the block was newly imported, so reaggregation only runs once per block. **Storage** - `BlockSignatures` table now stores the raw proof envelope; the `AggregatedPayloads` table stores `TypeOneMultiSignature`s. Genesis blocks simply have no proof entry (the placeholder `empty_block_signatures` is gone). **Dependencies / CI** - leanSpec pin bumped to `30ffb6c` (2026-06-03), leanVM to `e2592df`, `leansig` pinned to its `devnet4` branch. - CI fixture generation runs with `-n 1`: the devnet5 prover peaks at ~12 GiB per proof, so parallel provers OOM the 16 GiB runner. The fixtures cache is now only saved when generation actually succeeds, preventing a cancelled run from poisoning the cache with an empty fixture set. - Removed the curl+tar key-download workaround (the new pin is past leanSpec PR #745). ## Correctness / Behavior Guarantees - The on-chain block root is unchanged by proof serialization: consumers always hash the inner `Block`, never the `SignedBlock` envelope. - Block import rejects blocks whose merged Type-2 proof does not verify against the block's attestations and proposer key. - Reaggregation never floods gossip: it is skipped while syncing, and the per-block SNARK-split budget is capped. ## Tests Added / Run - Fork choice, signature, STF, and SSZ spec tests adapted to the devnet5 fixture format (fixtures regenerated from the new leanSpec pin). - New unit tests for the proof envelope helpers and reaggregation bounds. - `make fmt` / `make lint` clean (CI Lint is green). The CI Test job regenerates devnet5 fixtures (~2.5 h single-prover) before running the suite. ## Related Issues / PRs - Closes #285 - Spec reference: [leanSpec PR #717](leanEthereum/leanSpec#717) (Type-1/Type-2 aggregation wire format) ## ✅ Verification Checklist - [x] Ran `make fmt` — clean - [x] Ran `make lint` (clippy with `-D warnings`) — clean - [ ] Ran `cargo test --workspace --release` — CI Test job in progress (devnet5 fixture generation takes ~2.5 h) --------- Co-authored-by: Pablo Deymonnaz <pdeymon@fi.uba.ar> Co-authored-by: Blessing Samuel <dicethedev@gmail.com>
1 parent df8020b commit 8ee2c79

29 files changed

Lines changed: 1502 additions & 920 deletions

.github/workflows/ci.yml

Lines changed: 21 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -98,21 +98,10 @@ jobs:
9898
path: leanSpec/packages/testing/src/consensus_testing/test_keys/prod_scheme
9999
key: prod-keys-${{ steps.prod-keys-url.outputs.hash }}
100100

101-
# Download + extract the keys ourselves rather than via
102-
# `consensus_testing.keys --download`. The pinned leanSpec commit predates
103-
# leanSpec PR #745, whose `download_keys` reads the still-open (unflushed)
104-
# download tempfile, intermittently truncating the gzip tail and aborting
105-
# with EOFError. curl+tar fully writes the archive before reading it.
106-
# Remove once the pin moves past PR #745.
107101
- name: Download production keys
108102
if: steps.cache-fixtures.outputs.cache-hit != 'true' && steps.cache-prod-keys.outputs.cache-hit != 'true'
109103
working-directory: leanSpec
110-
run: |
111-
KEYS_URL=$(uv run python -c "from consensus_testing.keys import KEY_DOWNLOAD_URLS; print(KEY_DOWNLOAD_URLS['prod'])")
112-
KEYS_DIR=packages/testing/src/consensus_testing/test_keys
113-
mkdir -p "$KEYS_DIR"
114-
curl -sSL "$KEYS_URL" -o /tmp/prod_scheme.tar.gz
115-
tar -xzf /tmp/prod_scheme.tar.gz -C "$KEYS_DIR"
104+
run: uv run python -m consensus_testing.keys --download --scheme prod
116105

117106
# Save production keys even if a later step fails, so a re-run does
118107
# not have to re-download. See: https://github.com/actions/cache/tree/main/save#always-save-cache
@@ -128,15 +117,30 @@ jobs:
128117
path: leanSpec/packages/testing/src/consensus_testing/test_keys/prod_scheme
129118
key: ${{ steps.cache-prod-keys.outputs.cache-primary-key }}
130119

120+
# `-n 1` (not `-n auto`) runs a single leanVM prover at a time. The
121+
# devnet5 prover peaks at ~12 GiB per proof (measured), so more workers
122+
# blow past the 4-vCPU/16 GiB runner's RAM: `-n auto` (4 provers) hard
123+
# OOM-killed the runner, `-n 2` thrashed and lost its heartbeat so GitHub
124+
# cancelled the job. One prover peaks ~12.4 GiB with 0 swap and completes
125+
# in ~2h36m. The Makefile keeps `-n auto` for local machines with more RAM.
131126
- name: Generate test fixtures
127+
id: generate-fixtures
132128
if: steps.cache-fixtures.outputs.cache-hit != 'true'
133129
working-directory: leanSpec
134-
run: uv run fill --fork=Lstar --scheme prod -o fixtures -n auto
135-
136-
# Save fixtures even if a later step fails, so a re-run does not
137-
# have to regenerate them. See: https://github.com/actions/cache/tree/main/save#always-save-cache
130+
run: uv run fill --fork Lstar -n 1 --scheme prod -o fixtures
131+
132+
# Save fixtures only when generation actually SUCCEEDED. A bare
133+
# `always()` here previously saved the (empty) fixtures dir when
134+
# generation was cancelled or OOM-killed mid-run, poisoning the cache:
135+
# later runs hit the empty cache, skipped generation, and the Rust tests
136+
# failed with no fixtures. Gating on the generate step's outcome keeps
137+
# the "save even if the later Rust test step fails" intent without ever
138+
# persisting a partial fixture set.
138139
- name: Save test fixtures cache
139-
if: always() && steps.cache-fixtures.outputs.cache-hit != 'true'
140+
if: >-
141+
always()
142+
&& steps.cache-fixtures.outputs.cache-hit != 'true'
143+
&& steps.generate-fixtures.outcome == 'success'
140144
uses: actions/cache/save@v5
141145
with:
142146
path: leanSpec/fixtures

Cargo.lock

Lines changed: 37 additions & 37 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)