perf(erigon): 4.4-4.9× faster state generation — Direct-Drive Fold + M1 pipeline stack#110
Merged
Merged
Conversation
… at scale)
Generating a large trie (e.g. a 100 GB account-heavy alloc, ~613 M EOAs)
OOM-killed the Erigon snapshot writer. The slot stream already stayed
bounded; the OOM lived in the unlogged post-stream phases, which held
O(total-keys) in RAM. Two distinct causes, the second only visible at scale:
1. Commitment held every branch/key in RAM: an in-memory mergedBranches map
(Θ(total-keys)) plus upstream's Updates.keys dedup map (~30-65 GB at
100 GB) plus an etl working set on /tmp (tmpfs = RAM).
2. The snapshot writer os.ReadFile'd each .kv into anonymous heap; Phase 5b
builds domains concurrently, so the 28 GB accounts + 44 GB commitment .kv
loaded at once (~72 GB) → a second OOM at ~120 GiB anon-rss.
Fixes:
- commitment: replace mergedBranches with a live read-write Pebble branchStore
(branchstore.go); chunked first-serial-then-concurrent Process bounds
Updates.keys per chunk; etl spills to cfg.DBPath (real disk) not tmpfs;
ComputeGenesisRoot streams branches into a write-once .kv.
- seg.Decompressor: mmap the .kv read-only (munmap on Close) instead of
os.ReadFile, so the bytes are reclaimable file-backed pages (anon O(1)).
- Phase-5b index builders: btindex offsets → bufio spill; recsplit bucket
collector → streamsort (seq-suffix preserves collision detection);
WriteDomain defer-Close leak fix.
- add --autofill-profile=accounts (streamed pure EOAs) so an account-dominated
large trie is reachable without materializing a sequential_eoas spec.
- env tunables: STATE_ACTOR_{COMMITMENT_CHUNK_KEYS,ERIGON_WORKERS,
COMMITMENT_CACHE_GB,BRANCH_CACHE_GB}.
Consolidates the 10-commit fix branch. Validated: cgo commitment tests (H4
erigon HPH root == geth MPT root; ChunkedVsSingle root==single) + a 100 GB
account-heavy bench (exit 0, 65.9 GiB peak, root 0x31e0bfb1…).
… dedupe, madvise Root-preserving perf follow-up (no RNG/content change; cross-client root pinned by TestH4 + ChunkedVsSingle). None raises RAM; several lower it. - commitment: cap the SERIAL first chunk at min(firstChunkKeys=128K, commitmentChunkKeys). Chunk 0 runs single-core to seed the root branch; at the full chunk width it was ~12% of the whole fold (serial is ~16x/key). 128K keys already span all 16 first nibbles. Root unchanged (chunk boundaries don't affect the fold) — new ChunkedVsSingle assertion with firstChunkKeys=2 < chunk=5. - snapshot_cgo: Close commitmentInputStore immediately after ComputeGenesisRoot (last use) instead of at function scope — frees its block cache + memtable + ~tens-of-GB spill dir BEFORE the mmap-heavy Phase-5b write. Idempotent vs the deferred Close. - generation: hash account code ONCE in encodeEntity and thread it into the commitment Update via new EncodeAccountUpdateCodeHash — was keccak'd twice (snapshot CodeHash + EncodeAccountUpdate) for every code-bearing entity. - seg.Decompressor: MADV_SEQUENTIAL after mmap — aggressive read-ahead for the front-to-back Pass-2 scan and lets the kernel drop read pages sooner, lowering file-RSS on a 44 GiB .kv. Advisory; byte output unchanged.
The commitment domain is the LARGEST (~44 GB .kv at 100 GB, plus the only recsplit MPHF over ~nBranches keys) yet ran SERIALLY after the accounts/storage/ code fan-out, leaving the machine ~1-core-busy during its build. It depends only on branchesStore + keyStateValue (both ready from the fold), not on the other domains, so run it as a 4th goroutine in the same fan-out — the Writer is immutable and each domain writes its own files. Phase-5b wall time drops toward max(domains) instead of sum. Byte output unchanged.
…old (item 2) The three account/storage/code domain writes depend only on the finalized stores + counts, NOT on the commitment fold's result — yet they ran AFTER the 2-4h fold in Phase 5b, on cores the 16-way fold left idle. Start their fan-out BEFORE ComputeGenesisRoot so ~sequential compression overlaps the fold; only WriteCommitment (needs branches + keyStateValue) stays queued after. The Writer is immutable and each domain writes its own files, so no races; output bytes are unchanged (same inputs, reordered concurrency only).
…(item 6b) seg.Compressor.Compress rescanned the entire .idt (28-44 GiB per domain at 100 GB) solely to build the word-length histogram (posMap) + emptyWordsCount. Both depend only on each word's length, which AddWord already sees — accumulate them incrementally there and drop the Pass-A ForEach. One fewer full pass over the intermediate per domain. Byte output unchanged (the histogram fully determines the Huffman tree); seg goldens green.
… 1 foundation) The commitment-input store is being partitioned into 16 nibble sub-stores to kill the shared-Pebble-cache random-Get contention (58% of a 5GB run). The Touch must read those 16 sub-stores ROUND-ROBIN so chunk 0 spans all 16 first-nibbles (the first-serial-then-concurrent invariant) — which the callback-only Iterate can't do. Cursor is a caller-driven forward pull-iterator (NewCursor/Next/Key/ Value/Close) mirroring Iterate's finalize-gate + reader-ref lifecycle. Test: round-robin over 2 stores interleaves + preserves per-store sort order.
…C Phase 1) Kills the shared-Pebble-cache random-Get contention (58% of a 5GB run: 16 ParallelHashSort workers ping-pong the same block-cache atomic refcount + shard lock). Split the commit-input store into 16 sub-stores by keccak first-nibble — exactly the worker sharding — so each worker reads its OWN store → disjoint caches → no cross-worker contention. Keeps ModeDirect + 16-way; root unaffected (partition only changes which file holds a value, not the keccak order or fold). - commitment.go: ComputeGenesisRoot takes []*streamsort.Store (len 16); subtreeCtx.Account/Storage route by InputPart(plainKey); the Touch ROUND-ROBINS 16 Cursors so chunk 0 still spans all 16 first-nibbles (else the concurrent unfold reads an empty branch). Exported NumInputParts/InputPart. - snapshot_cgo.go: 16 sub-stores (cache budget /16); commitIn writes tagged with their part on the parallel encode workers (keccak stays off the single writer); runCommitInWriter routes by part; finalize/close all 16. Phase 2 (re-key by hashed key + reused per-worker SeekGE cursor to also kill the per-Get newIters ~17%) is a follow-up. Unit gate: H4/ChunkedVsSingle/FirstChunkShrink.
…tores (Tier-C Phase 2) Phase 1 killed the shared-cache contention; Phase 2 kills the per-Get newIters (~17%). Re-key the 16 commit-input sub-stores by the HASHED key (value = EncodeInputRow(plainKey, update) so the Touch can still recover the plain key), so each worker reads its sub-store in hashed-sorted order. Each subtreeCtx then serves Account/Storage from ONE lazily-opened reused streamsort.Getter (SeekGE on a long-lived pebble.Iterator) instead of db.Get-per-call — forward SeekGE stays in-file, skipping the per-Get iterator build. The root/serial ctx spans nibbles, so it retires the getter and falls back to db.Get (low volume). - streamsort.Getter (reused SeekGE iterator) + test. - commitment.go: HashedKey/EncodeInputRow/DecodeInputRow; subtreeCtx.lookup routes to the reused getter (or db.Get for the root); Touch recovers plainKey from the row value; factory closes the getter; helper keys by HashedKey. - snapshot_cgo.go: commitIn writes keyed by HashedKey(plainKey), value wrapped. Root must stay byte-identical: H4/ChunkedVsSingle/FirstChunkShrink.
…nch at scale This reverts 88b492d. Phase 2 re-keyed the 16 commit-input sub-stores by the HASHED key so a reused SeekGE Getter could kill per-Get newIters (~17% of commitment). But hashed keying makes each round-robin Touch chunk a NARROW consecutive hashed slice per nibble: the serial first chunk builds only a deep narrow skeleton, a later CONCURRENT chunk descends into an intermediate branch never established -> 'empty branch data read during unfold' (hex_patricia_hashed.go:1749). Bites once slices are dense (100GB and 5GB@chunk=250K reproduce; unit tests + 5GB@chunk=5M do not). Phase 1 (plain-key sub-stores, kept) makes every chunk a BROAD hashed sample = pre-Tier-C regime that passes at 100GB. Confirmed: 5GB@chunk=250K FAILS on Phase 2, PASSES (0x7acbe35d) on Phase 1. Phase 1 keeps the partition win (~29%); the Getter (~17%) needs hashed stores -> restoring it needs a broad Touch over hashed stores, deferred. Root unaffected.
… goroutine The single-threaded draw loop keccaks AddrHash for every account and CodeHash for every ~30% delegated EOA, but the erigon writer never reads either (it keys stores by plain address and re-derives the code hash on its parallel encode workers — encodeEntity). Neither keccak consumes RNG draws, so eliding them cannot change the drawn byte-sequence or any cross-client root. - entitygen.GenerateEOALean + autofill.GenerateEOAFlavoredLean: same draws, derived hashes left zero; existing functions unchanged (other clients keep AddrHash). - Plan.SkipDerivedHashes routes DrawEOA; erigon opts in at its draw loop. - TestFlavoredDrawRNGSequenceInvariant pins the safety argument: 2000 draws byte-identical content + post-draw RNG state aligned, delegation branch exercised. - Refresh the stale commitmentChunkKeys comment (chunked runs chunk 0 serial + rest CONCURRENT, not 'serial'; single-shot RAM math + GOMEMLIMIT=20GiB guidance instead of the obsolete '240 GB bench' note).
…ranch store The branch bytes used to be sorted TWICE: the walk's live branchStore (pebble, already ascending) was iterated and re-Put row-by-row into a second streamsort (branchesOut), finalized, then iterated again by WriteDomain — a full extra pebble write+compaction+read of the ~44 GB branch set at 100 GB scale. - commitment.Result now RETAINS the branch store (ownership transfers on success): BranchIterate streams it (repeatably), CloseBranches releases it (idempotent). ComputeGenesisRoot drops the branchesOut param; the count pass is a pure sequential scan (chunked mode re-folds prefixes, so the count must come from a scan, not a set-counter). - snap.WriteCommitment takes a BranchStream and splices the KeyCommitmentState row at its binary sort position (1-key 2-way merge). Producer errors surface as the ROOT CAUSE ahead of WriteDomain's downstream symptoms (no silently truncated .kv). A branch prefix colliding with 'state' now FAILS LOUDLY — the old last-write-wins Put would have silently dropped the branch row. - snapshot_cgo: branchesStore deleted; WriteCommitment consumes Result.BranchIterate via method-value conversion; retained store closed after emitWg.Wait via defer. Tests: TestWriteCommitment_StateRowSplice (mid-stream/last/only + ascending), TestWriteCommitment_StreamErrorSurfaces, TestWriteCommitment_StateCollisionErrors, TestResultBranchLifetime (cgo). Output byte-identity pinned by the existing seg/btindex/recsplit goldens + ChunkedVsSingle branch-bytes oracle.
…s/send) The M1a-0 profile (5GB, single-shot) showed generation is NOT writer-Put-bound (pebble arenaskl ~4%) — the cost is the per-item channel machinery itself: sendDomainWrite 10.15% cum, runtime.selectgo 30% cum, futex/lock2/steal ~35% scheduler churn across 48 producers. 613M entities × 2-3 sends each ≈ 1.5-2B channel operations. Batch them: each producer (encode worker; the PreAlloc drain on main) owns per-domain batchedSenders accumulating 128 domainWrites per channel send; writers consume []domainWrite batches. ~99% fewer channel ops, same in-flight row count (256 batches ≈ old 4096-item buffer). Root-neutral: pebble sorts by key regardless of arrival order/batching (partition-invariance argument unchanged). Producers flushAll before exit so no buffered rows are dropped (worker exit + PreAlloc drain, both before the channels close).
…e-shot-GATED Re-applies the Phase-2 machinery (88b492d) with the guard that makes its bug class structurally impossible. KeyingHashed keys the 16 commit-input sub-stores by KeyToHexNibbleHash(plainKey) (value = EncodeInputRow carrying the plain key), so each 16-way fold worker reads its nibble in exactly stored order and serves Account/Storage from ONE reused SeekGE streamsort.Getter — killing the profiled per-Get newIters/SeekGE chain (16.7% of the whole 5GB run, ~60% of the fold). THE GUARD: hashed keying is valid ONLY single-shot. Under chunking, hashed order makes each round-robin Touch chunk a narrow consecutive hashed slice whose thin serial-chunk skeleton breaks later concurrent chunks ('empty branch data read during unfold' — the 209af4a revert). HashedInput() (== chunkKeys==0) is the single source of truth for BOTH the gen-side keying and the fold's keying parameter, and ComputeGenesisRoot rejects KeyingHashed+chunking at entry. At 100 GB under the 20 GiB heap budget (chunked), gen stays plain-keyed — this ships as M2's input layer, active at single-shot scales now. - streamsort: Getter (reused SeekGE iterator) + TestGetter restored. - commitment: InputKeying/HashedInput/HashedKey/Encode+DecodeInputRow; subtreeCtx.lookup routes plain (db.Get) vs hashed (bound Getter; the nibble-spanning root ctx retires to db.Get); keying-aware Touch; ComputeGenesisRootFromAccountsKeyed for dual-layout tests. - snapshot_cgo: commitIn rows keyed per HashedInput() on the encode workers. - Tests: TestComputeGenesisRoot_HashedVsPlainSingleShot (byte-equal root + branch bytes over a 2048-acct all-nibble fixture w/ storage), TestComputeGenesisRoot_HashedRejectsChunking (negative guard).
Mechanical vendor of execution/commitment into internal/erigon/hph — the same
maintenance model as internal/erigon/{seg,btindex,recsplit}: pinned provenance
headers, byte-golden-tested against the module (Golden A), bench erigon daemon
stays stock upstream.
- Copied verbatim: hex_patricia_hashed.go, hex_concurrent_patricia_hashed.go,
commitment.go, keys_nibbles.go, metrics.go, warmup_cache.go, warmuper.go,
nibbles/nibbles.go. Package commitment -> hph; nibbles import rewritten;
cgo_erigon_commitment build tag (module deps resolve on the bench only).
- Stripped: the witness/trie surface (GenerateWitness, toWitnessTrie,
witnessCreateAccountNode, witnessComputeCellHashWithStorage + the trie/
witnesstypes imports — no kept callers) and the untouched-by-us files
(bin_patricia_hashed, verify, warmuper... trie_reader, recording_context,
trie_trace, tests).
- Golden A (golden_parity_test.go): drives the vendored AND module engines
through the identical 16-way concurrent Process over a 2048-account
all-nibble fixture (half with storage) and asserts root + EVERY branch row
byte-identical (upstream only ever asserts root equality). Also pins the
Update wire-encoding equality across the two packages.
Why: the DDF driver (next commit) calls the engine's own followAndUpdate
directly from sorted cursors — that requires the unexported fold surface,
hence the vendor.
…DDF step 2) The DDF driver feeds the vendored engine's followAndUpdate straight from the 16 hashed-keyed sub-store cursors, from empty, in one pass: - hph.DirectFold (state-actor addition inside the vendored package — needs the unexported mounts/unfoldRoot/foldNibble): replicates ParallelHashSort's engine choreography verbatim; the ONLY change is the feed — a KeyStream per nibble instead of an etl collector, with the Update passed inline so the ctx.Account/Storage re-fetch branch is never taken. - commitment.ComputeGenesisRootDirect: cursorStream adapters (row key IS the nibblized hashed key the engine sorts by; value carries plainKey + Update), per-worker WRITE-ONCE branch sinks (lazy — unfoldRoot's provisional ctxs never write; rows counted as emitted → exact BranchCount, no count scan), root row flushed via ApplyAndClearInlineDeferredUpdates as on the engine path. Branch reads return nil (from-empty single-shot never re-reads a written prefix; every prev-merge read is nil-correct); Account/Storage FAIL LOUDLY (any re-fetch = broken assumption). - Result gains branchSinks mode: BranchIterate k-way heap-merges the sorted sinks (nibble-disjoint prefixes + unique root row) into the ascending stream snap.WriteCommitment already consumes; CloseBranches dual-mode. - Gated: STATE_ACTOR_COMMITMENT_DIRECT (default ON) AND KeyingHashed (single-shot) — the Updates/etl engine path stays as fallback. Deleted relative to the engine path: the never-spilled Touch dedup map (~50-67 GB at 613 M keys — the reason chunking existed), the etl re-sort spill, all cross-chunk branch prev-reads, all per-key input Gets, and the 44 GB BranchCount scan. Golden B (directdrive_test.go): engine/plain == engine/hashed == DIRECT on root + BranchCount + every branch row byte + HPHState, across span-2048 (w/ storage), single, one-slot, multi-slot, one-nibble-cluster, and empty fixtures.
…omment rot
Five-agent review of the full stack found zero correctness defects; this lands
the fact-checked remainder, holding to a minimum-additions rule (guards that
replace tests; deletions over rewrites).
Hardening (error paths only — no produced byte changes):
- directdrive: register the root sink BEFORE acting on flush errors (was
leakable), and record each sink's Finalize error in the registry (firstErr,
surfaced before handoff) instead of swallowing it.
- snap.WriteCommitment: assert the branch stream is strictly ascending — the
one seam WriteDomain trusts blindly ('behaviour is undefined'); replaces the
proposed fixture merge-order test with a check every run performs.
- snap.WriteDomain: remove the final-named .kv when pass-2 (accessor build)
fails, so an aborted build can't pass for a complete domain.
- commitment.ComputeGenesisRoot: probe the input key length (plain 20/52 vs
hashed 64/128 — disjoint) against the declared keying; a layout skew now
fails loud instead of folding a silently wrong root.
- snapshot_cgo: set SkipDerivedHashes on a LOCAL copy of the Plan (the shared
config is no longer mutated).
Oracle gap (the one genuine coverage hole): code-bearing accounts never
crossed the byte-identity gates. Golden A's fixture now marks every 5th
account with delegation code (both engines' encoders gained CodeUpdate);
Golden B adds 'delegated' + 'codeAndStorage' fixtures and bumps multiSlot to
64 slots (multi-level storage subtree).
Comment rot (net-negative lines): provenance header now lists the real strip
(5 funcs incl. PrintGrid + 4 imports); deleted the ~45-line orphaned block
documenting the removed branchesOut parameter; the CloseBranches defer comment
states the true safety reason (reader spawns after the early returns); the
chunk-keys/pipeline/streamsort docs no longer describe the engine fallback as
the default path; batch-buffer doc states 8x in-flight rows, not 'same'.
…y-default
Tail (the ~26.5-min snapshot/commitment phase after the DDF fold):
- seg: CompressFromSource writes the .kv straight from a repeatable word
source (pass A counts, pass B encodes) — byte-identical by construction
(the .kv is a pure function of the word sequence; the huffman table
derives from the incremental posMap; the bit writer flushes to byte
alignment per word). WriteDomain now drives its entries closure twice
through it, eliminating the .idt scratch: ~90 GB of writes across the 4
domains and a 44 GB transient disk peak at 100 GB scale. The golden test
runs BOTH paths against the same upstream bytes.
- snap.WriteCommitment: state-row splice + ascending assert moved to
per-invocation scope (load-bearing for the twice-driven closure — hoisted
state would silently drop the state row from pass 2).
- recsplit: parallel Build (Args.Workers) mirroring upstream's
buildWithWorkers — producer groups buckets in ascending order with dense
seq numbers, N workers run the unchanged recsplitRecurse on private
scratches (own golombParamCache each; the footer harvests the longest
table), a min-heap consumer writes strictly in dispatch order, so output
is byte-identical at any worker count. flushCurrentBucket refactored into
computeBucket (pure) + writeResult (serial) shared by both paths. The
spike golden now runs Workers∈{0,4}. Client default min(NumCPU,8) via
STATE_ACTOR_RECSPLIT_WORKERS.
- Instrumentation under Verbose: per-domain WriteDomain, fold, commitIn
close, WriteCommitment timings; STATE_ACTOR_SNAP_TIMING adds pass-1/
pass-2/Build splits — the first true decomposition of the tail.
RAM under control BY DEFAULT (in-binary, zero-config): pebble memtable
arenas and block caches are C-malloc'd under cgo — off the Go heap,
invisible to GOMEMLIMIT — so the fix is code defaults, not env discipline:
- streamsort.Options.MemTableBytes (new knob) + right-sized defaults:
64 MiB commitIn/branch-sink/recsplit-spill arenas, 128 MiB value stores
(was 256 MiB each; up to 2 live arenas per store).
- commitIn block cache defaults to 8 MiB/store under hashed keying (DDF
cursors and the engine Getter read sequentially — the 4 GiB cache only
helps the chunked plain path's random Gets, which keeps it);
STATE_ACTOR_COMMITMENT_CACHE_GB set explicitly still wins.
- debug.SetMemoryLimit(8 GiB) iff GOMEMLIMIT unset — heap ceiling for
unconfigured runs; explicit env always wins.
- Verbose prints the go_heap/go_sys split so RSS readers aren't misled by
reclaimable mmap pages.
Expected committed off-heap during the fold: ~8-12 GiB -> ~1.5-3 GiB.
All seg/recsplit/snap/streamsort suites green locally, including the
two-path seg golden and the workers-parity spike golden.
… (R1) The Direct-Drive Fold obsoleted chunking as the low-RAM option; the engine path survives single-shot-only as the Golden-B oracle and the STATE_ACTOR_COMMITMENT_DIRECT=0 fallback. ComputeGenesisRoot's fallback collapses to one Updates + concurrent Process; the chunk knobs (STATE_ACTOR_COMMITMENT_CHUNK_KEYS, firstChunkKeys), the hashed+chunked guard, HashedInput() (constant-true once chunking is gone), and the cursor round-robin walker all go. chunked_test.go dies with the path; hashed_single_shot_test.go was a strict subset of Golden B (its keccakSpanFixture relocates to directdrive_test.go). Result.BranchNodes was test-only plumbing on a production struct — tests now collect via BranchIterate (collectBranches helper) and own the CloseBranches lifecycle. Dead setErigonWorkers removed. Net −406 lines; roots re-proven by Golden B + H4 + the 5 GB gate.
Reachability closure from the live roots (DirectFold + the Golden A driver): none of these had an inbound edge — deleted with Golden A left UNCHANGED as the proof. hex_patricia_hashed: SetState/state.Decode/ cell.Decode (state-restore, unused — the chunked consumer used the upstream module), the HexTrie* string readers, feedBranchHashesToKeccak (superseded by hashRow), Grid, the DomainPutter/CommitmentWrite aliases (+stateifs import), and the never-armed collapse-tracer cluster (type, field, setter, both detectors, two nil-guarded call sites — the only live-function edits, both provably never taken since the setter had no callers). hph/commitment.go: ReplacePlainKeys/MergeHexBranches/ Validate+helpers/BranchStat+Collect+DecodeBranchAndCollectStat/ ParseTrieVariant/PendingCommitmentUpdate/the non-Direct Touch* family/ NewEmpty/SetMode/Mode/PlainKeys/keyHasherNoop (+ crypto/accounts/keccak/ slices/nibbles imports). keys_nibbles + nibbles: dead helpers (incl. the CompactToHex cascade once validatePlainKeys died). metrics.go: the CSV read-back cluster (zero callers anywhere). Provenance headers updated. Net ≈ −1,290 lines; byte-identity re-proven by the unchanged Golden A + Golden B + full cgo suites.
…ed engine half (R3) Golden A now pins what production actually runs: vendored DirectFold vs the UPSTREAM module engine, byte-for-byte on root + every branch row + HPHState (new), fed by an in-memory sliceStream (16 first-nibble shards, ascending hashed order — the cursorStream analogue). The old driver exercised Updates/Process/ParallelHashSort — a choreography production never executes; it was the only thing keeping that half of the vendored package alive. Purged with it: the Updates core (NewUpdates/TouchPlainKeyDirect/ HashSort/arena/Mode/KeyUpdate/keyHasher), both Process variants, ParallelHashSort, CanDoConcurrentNext, the dead Trie interface + TrieVariant + InitializeTrieAndUpdates + CommitProgress + the Tier-2 methods they compile-required, the never-enabled warmup subsystem (warmuper.go + warmup_cache.go deleted; TrieContextFactory relocated to directdrive.go; cacheOrDB fast-paths collapsed to plain ctx delegates), the hphPool/Release/resetForReuse cluster, Process-confined fields (traceDomain/capture/leaveDeferredForCaller — trace stays, 54 live reads), and metrics.go reduced to the live counter subset (790 → 188). hex_concurrent is now exactly the DirectFold core: mountTo, NewConcurrentPatriciaHashed, foldNibble, unfoldRoot. Vendored package: ~7,400 → 3,677 lines. Provenance headers list the full strip. Byte-identity: the retargeted Golden A (independent upstream oracle — a stream-order bug fails loud, it cannot false-green) + Golden B + the 5 GB root gate.
Shrinks the heavyweight comment blocks to their load-bearing content: fullRange/fat-genesis (constraint kept, history cut), Step-5b/overlap narration, domainBatchSize/commitIn-close profiling stories, streamsort package+Options docs, the DDF header, subtreeCtx lifecycle, the WriteCommitment contract, and the recsplit spill-key rationale (the seq LOAD-BEARING note stays). branchstore drops the chunk-era 1 GiB block cache + STATE_ACTOR_BRANCH_CACHE_GB env (the re-read path died with chunking) — 8 MiB cache + 64 MiB memtable like the other write-once stores. Comments only otherwise.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Cuts erigon 100 GB state generation (613 M accounts) from 6-7 h to 1 h 07 m — with zero configuration (no env vars) — and the state root byte-identical (
0x31e0bfb1…, cross-client MATCH vs reth, stock daemon boots the output).Peak RAM at 100 GB: RSS 50.6 GB, of which the committed footprint is small — 3.5 GiB Go heap (under an in-binary 8 GiB ceiling) + ~2-3 GiB off-heap pebble; the rest is kernel-reclaimable mmap page cache from the
.kvpasses that shrinks automatically on constrained hosts. NoGOMEMLIMITor tuning env needed: RAM is bounded by code defaults.Why the old fold was slow
The writer drove erigon's incremental commitment engine for a from-empty bulk build. The engine's never-spilled
Updatesdedup map (~50-67 GB at 613 M keys) forced chunking; chunking re-descends the trie spine every chunk, injecting ~1.5 B random branch prev-reads over a 44 GB store — that was essentially the whole 5 h 35 m fold, against a pure-compute cost of minutes.The fix — Direct-Drive Fold
internal/erigon/hph/(same model as the existingseg/btindex/recsplitports; the bench erigon daemon stays stock upstream). The vendored copy is trimmed to exactly the live surface (~3.7 k lines): Golden A pins it against ground truth — vendoredDirectFold== the upstream erigon-module engine, root + every branch row + HPHState byte-identical, side-by-side in one binary.hph.DirectFold: replicates the upstream concurrent engine's choreography, but each worker streams its nibble's already-sorted, already-unique sub-store cursor and callsfollowAndUpdate(hashedKey, plainKey, &update)with the Update inline — noUpdatesobject, no Touch dedup map, no etl re-sort, no chunking, no prev-reads, no per-keyctx.Account/StorageGets.STATE_ACTOR_COMMITMENT_DIRECT, default on). A single-shot fallback on the upstream-module engine remains (DIRECT=0); the old chunked path is removed — DDF made it obsolete as the low-RAM option.Golden B pins the byte contract: engine/plain == engine/hashed == DIRECT on root + branch count + every row byte + HPHState across adversarial fixtures (all-nibble span with storage, code-bearing accounts, multi-level storage, one-nibble cluster, empty trie).
The tail — snapshot writing
.idtintermediate is gone:seg.CompressFromSourcedrives the (repeatable) entry source twice — count pass, then encode straight into the.kv— byte-identical by construction and proven by the seg golden running BOTH writer paths against the same upstream bytes. Saves ~90 GB of writes and a 44 GB transient disk peak per 100 GB run..kviBuild (min(NumCPU,8)workers): producer → N workers → in-dispatch-order writer, byte-identical at any worker count (spike golden runsWorkers ∈ {0,4}). The 100 GB MPHF build: ~1 min.RAM by default (no env vars required)
Pebble memtable arenas and block caches are C-malloc'd under cgo — off the Go heap, invisible to GOMEMLIMIT — so the bound is shipped as code defaults: small write-once memtables (64-128 MiB), an 8 MiB/store commit-input cache under the (sequential-read) default path, and
debug.SetMemoryLimit(8 GiB)whenGOMEMLIMITis unset. Env knobs remain as opt-in overrides only.Supporting commits
WriteCommitmentstreams the fold's branch source directly (deletes a full 44 GB pebble re-sort), splices thestateanchor at its sort position, asserts ascending order, and fails loudly on producer errors/collisions instead of writing silently-truncated files.Validation
0x7acbe35d…unchanged, 4:08 wall.0x7fa5f442…), stock daemon boots.Notes
https://claude.ai/code/session_01RmVw3kWVHyhTKbVfwWUMs5