Eth Block Extractor

Description

A VulcanizeDB transformer for creating IPLDs for Ethereum block data.

Dependencies

• Go 1.12
• Postgres 10
• Ethereum Node
  • Go Ethereum (1.8.23+)
• IPFS
• go-ipld-eth (Plugin enabling conversion of block headers to IPLDs in IPFS)

Installation

1. Setup Postgres and an Ethereum node - see VulcanizeDB README.
2. Sync VulcanizeDB to populate core block data (commands will read block data from VulcanizeDB to fetch and persist block RLP data).
3. go get github.com/vulcanize/eth-block-extractor
4. cd $GOPATH/src/github.com/vulcanize/eth-block-extractor
5. dep ensure
6. make build

Configuration

• To use a local Ethereum node, copy environments/public.toml.example to environments/public.toml and update the levelDbPath to the local node's levelDB filepath:
  • when using geth:
    • The LevelDB file is called chaindata.
    • The LevelDB file path is printed to the console when you start geth.
    • The default location is:
      • Mac: $HOME/Library/Ethereum
      • Linux: $HOME/.ethereum

Running the createIpldForBlockHeader command

• This command creates an IPLD for the header of a single Ethereum block.
• ./eth-block-extractor createIpldForBlockHeader --config <config.toml> --block-number <block-number>

Running the createIpldForBlockHeaders command

• This command creates IPLDs for headers in a range of Ethereum blocks.
• ./eth-block-extractor createIpldForBlockHeaders --config <config.toml> --starting-block-number <block-number> --ending-block-number <block-number>
• Note: ending block number must be greater than starting block number.

Running the createIpldsForBlockTransactions command

• This command creates IPLDs for transactions on an Ethereum block.
• ./eth-block-extractor createIpldsForBlockTransactions --config <config.toml> --block-number <block-number>

Running the createIpldsForBlocksTransactions command

• This command creates IPLDs for transactions on a range of Ethereum blocks.
• ./eth-block-extractor createIpldsForBlocksTransactions --config <config.toml> --starting-block-number <block-number> --ending-block-number <block-number>
• Note: ending block number must be greater than starting block number.

Running the createIpldsForBlockReceipts command

• This command creates IPLDs for transaction receipts on an Ethereum block.
• ./eth-block-extractor createIpldsForBlockReceipts --config <config.toml> --block-number <block-number>

Running the createIpldsForBlocksReceipts command

• This command creates IPLDs for transaction receipts in a range of Ethereum blocks.
• ./eth-block-extractor createIpldsForBlocksReceipts --config <config.toml> --starting-block-number <block-number> --ending-block-number <block-number>
• Note: ending block number must be greater than starting block number.

Running the createIpldsForStateTrie command

• Note: this command is very expensive in terms of time and memory. Probably only feasible to execute on an archive node for a narrow range of blocks.
• This command creates IPLDs for state and storage trie nodes in a range of Ethereum blocks.
• ./eth-block-extractor createIpldsForStateTrie --config <config.toml> --starting-block-number <block-number> --ending-block-number <block-number>
• Note:
  • Optionally pass the --compute-state flag if not running an archive node (in which case state is pruned) - this will dynamically generate the state for each block by processing transactions.
  • Computing state requires beginning at the genesis block, so starting block number flag is ignored if not 0.
  • Ending block number must be greater than starting block number.

Running the tests

make test

or

ginkgo -r