Skip to content

Registry-first sync: fetch the nodes table before the bulk snapshot so the mesh forms before catch-up #144

Description

@t-bhendarkar

Problem

Snapshot state-sync ships materialized state via dag.js:iterateCanonicalState(). The nodes (registry) table is yielded 9th, and the install is atomic, so a joining node cannot authorize any later-registered peer until the entire (~1500-row / ~30 MB) install plus root verification finishes.

During that whole window the joiner's replayed registry view lags, so its TIP handshake with a later-registered committee member is rejected (not in registry). That long stale window is the root cause behind the handshake-during-sync failures.

PR #143 added a re-handshake heal (retry the handshake the moment sync completes), which fixes correctness. This issue is the cleaner structural fix that prevents the window in the first place.

Proposal

Add a small dedicated registry-fetch step before the bulk snapshot:

authorize bootstrap
  -> fetch + verify the `nodes` (registry) table  (small, fast)
  -> dial + handshake the full committee mesh       (registry is now current)
  -> multi-source bulk state-sync from the mesh
  -> verify full state root -> ready -> participate

The mesh forms before catch-up, so handshakes succeed first try, multi-source sync starts immediately, and there is no rejection churn and no committee-pulls-from-a-catching-up-joiner thrash.

Trust trade-off (acceptable)

The registry is used before the full state root is verified. That is safe and contained:

  • It is fetched from the genesis-authenticated bootstrap peer.
  • A wrong registry only causes failed handshakes (honest peers verify you against their registry, so a poisoned local registry cannot forge consensus).
  • It is corrected when the full state root verifies later.

So: bootstrap-trust the registry early, re-verify with the full root afterward.

Relationship to existing work

Notes

TIP uses state-sync (transfer materialized state + verify root vs 2f+1), not replay-from-genesis, which is the correct model for a long chain. This change only reorders what is fetched first, not the sync model.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions