Hey Austin

Found a bug while running C-Chain re-execution benchmarks to measure the performance impact of your proposal-sharing changes.

The PR CI passes because it runs with config: "default" (hashdb) using a hashdb snapshot. It never exercises the Firewood code path.

When running with config: "firewood" starting from a restored Firewood snapshot at block 10M, it fails immediately on the first block https://github.com/ava-labs/avalanchego/actions/runs/20661797719/job/59326295398#step:4:831:

failed to verify block at height 10000001: no proposal found for block 10000001,
root 0x5f7138331cc9dd9d47cd78fb5bf632b0961a28d29d394f19e6d32f2a9e1f4266,
hash 0x90649defd375c9784a95bb01e752c2a06341213c59e25dedfe8aee7a7e8b8d3d

The proposal-sharing logic expects Hash() to populate t.possible before Update() is called. When loading from a Firewood snapshot and executing the first block, something in the initialization path skips Hash() so t.possible is empty when Update() runs.

To reproduce:

gh workflow run "C-Chain Re-Execution Benchmark w/ Container" \
    --repo ava-labs/avalanchego \
    --ref es/candidate-firewood \
    -f runner="avalanche-avalanchego-runner-2ti" \
    -f block-dir-src="cchain-mainnet-blocks-10m-20m-ldb" \
    -f current-state-dir-src="cchain-current-state-firewood-10m" \
    -f config="firewood" \
    -f start-block="10000001" \
    -f end-block="10050000"

To save you time digging through git history: es/candidate-firewood is your PR merged on top of es/baseline-firewood, which contains master plus benchmarking infrastructure (pprof, FFI metrics, cross-stack profiling). The baseline branch with the same Firewood config and snapshot works fine. https://github.com/ava-labs/avalanchego/actions/runs/20661268516/job/59324164005

Let me know if you need help reproducing or testing a fix.

After some investigation, I've discovered a couple ideas

I made a draft showing how number 3 could be implemented: #4835. It's definitely messier, but wouldn't break state. However, I do think that we will be making a breaking change to firewood state before the v0.1.0 release

I'm not sure about test coverage for this -- I would assume this refactor would introduce new things to test -- some ideas include -- like if Hash() gets called concurrently, or if any of the functions (e.g. Update()) are called in correctly.

These tests are not complete, but I do have an outstanding issue for guaranteeing minimum behavior. This just ensure that the basics are laced together correctly, and the specific multiple hashing thing you suggested

These tests are not complete, but I do have an outstanding issue for guaranteeing minimum behavior. This just ensure that the basics are laced together correctly, and the specific multiple hashing thing you suggested

fire - they can be built upon!!