Skip to content

Feature: indexer backfill command to index from genesis or an arbitrary ledger #611

Description

@Psalmuel01

The ingester advances from a stored cursor and supports a configurable start ledger on a cold start, plus reconcile for reorgs. There is no way to backfill deliberately: an operator standing up a new indexer against a contract that has been live for months, or repairing a gap after an outage, has no supported path other than deleting the database and hoping the start-ledger config does the right thing.

Backfill is also different from normal ingestion in shape: it wants to process a bounded historical range as fast as the API allows, not poll at an interval with a finality lag.

Scope

  • Add a backfill mode or CLI command that indexes a specified ledger range (or from contract deployment to head) with batched requests and progress reporting.
  • Make it resumable so a long backfill can be interrupted without starting over.
  • Keep it distinct from the live poll loop so backfill throughput does not fight the finality-lag logic.
  • Document when to backfill and how long it takes for a given range.

Acceptance

  • An operator can backfill a bounded ledger range, resume an interrupted backfill, and see progress; live ingestion is unaffected.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Stellar WaveIssues in the Stellar wave programbackendAPI routes, indexer, key management, server-side servicesmediumModerate complexity

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions