Haven't read into this deeply just yet, but awesome to see someone doing this on a lower-level than communicating with ETH ELs over the engine API :)

Would be really neat to add a revm::Database implementation that's backed by commonware_storage::qmdb.

Would be really neat to add a revm::Database implementation that's backed by commonware_storage::qmdb.

This would be :100000000000:

Haven't read into this deeply just yet, but awesome to see someone doing this on a lower-level than communicating with ETH ELs over the engine API :)

Would be really neat to add a revm::Database implementation that's backed by commonware_storage::qmdb.

Thanks for the quick feedback. I really appreciate the suggestion. Agree that a QMDB-backed backend would be a great “Commonware-native” next step.

One caveat I’ve been thinking about: revm::Database is a synchronous, hot-path interface, while commonware_storage::qmdb is async (and REVM’s built-in async→sync adapter is Tokio-based). I don’t want to accidentally introduce blocking in the EVM execution loop or pull Tokio in outside runtime.

A direction that seems workable is a layered backend: keep a CacheDB/InMemoryDB overlay for sync reads during execution, and add explicit async load_from_qmdb / flush_to_qmdb at finalized block boundaries (so QMDB provides persistence, but the EVM still reads from an in-memory cache).

Does that match what you had in mind, or is there a better pattern you’d prefer for bridging the async/sync boundary here?

I'm sorry for the big PR. I know it's not ideal and hard to review. I split it into many small commits to make it more manageable. Please let me know if I should break it into smaller PRs.

Given the goal of external state sync/proofs, I propose switching the example to an authenticated QMDB using the ordered current variant (so we can provide inclusion + exclusion proofs and complete range proofs).

I think that would be the most powerful example, although I've always believed you could get away with any: https://commonware.xyz/blogs/adb-any
In any case, I think it'll be easy to swap out the QMDB as needed (for different variants).

The benefit to using a simpler one at first is that we already have state sync done. current is more complex (and will come later).

Please let me know what you think about the current solution when you have time

Haven't read into this deeply just yet, but awesome to see someone doing this on a lower-level than communicating with ETH ELs over the engine API :)

Would be really neat to add a revm::Database implementation that's backed by commonware_storage::qmdb.

Thanks a ton for continuing on this work. I'll make sure to give this a review by the end of the week!

Haven't read into this deeply just yet, but awesome to see someone doing this on a lower-level than communicating with ETH ELs over the engine API :)
Would be really neat to add a revm::Database implementation that's backed by commonware_storage::qmdb.

Thanks a ton for continuing on this work. I'll make sure to give this a review by the end of the week!

Thanks! I tried to make this easy to review with a clean history, docs/diagrams, and clear run/test instructions. If you’d prefer this split into smaller PRs or reviewed in a specific order, happy to adjust.

Haven't read into this deeply just yet, but awesome to see someone doing this on a lower-level than communicating with ETH ELs over the engine API :)
Would be really neat to add a revm::Database implementation that's backed by commonware_storage::qmdb.

Thanks a ton for continuing on this work. I'll make sure to give this a review by the end of the week!

Thanks! I tried to make this easy to review with a clean history, docs/diagrams, and clear run/test instructions. If you’d prefer this split into smaller PRs or reviewed in a specific order, happy to adjust.

There are a number of external teams now following this PR lol. Many are very keen to see how this goes 👀

Haven't read into this deeply just yet, but awesome to see someone doing this on a lower-level than communicating with ETH ELs over the engine API :)
Would be really neat to add a revm::Database implementation that's backed by commonware_storage::qmdb.

Thanks a ton for continuing on this work. I'll make sure to give this a review by the end of the week!

Thanks! I tried to make this easy to review with a clean history, docs/diagrams, and clear run/test instructions. If you’d prefer this split into smaller PRs or reviewed in a specific order, happy to adjust.

There are a number of external teams now following this PR lol. Many are very keen to see how this goes 👀

Noted, I appreciate the heads‑up. I'll try to keep things moving cleanly so everyone watching can track progress.

On a separate note, I’m seeing a reproducible stack overflow in the revm example after merging main. It hits on a tokio worker thread during persist_snapshot -> QmdbState::commit_changes -> open_stores, and the backtrace shows the crash inside commonware_runtime::storage::validate_partition_name while initializing QMDB stores. I captured an lldb trace confirming the call chain (marshal -> finalized reporter -> persist_snapshot -> QMDB init).

The workaround/fix for now is running the persistence path on the blocking pool (context.shared(true)), which eliminates the overflow and the test passes. I’m still investigating a minimal repro on main (so far I can’t trigger it with QMDB init alone). Will follow up if I isolate it to WrapDatabaseAsync or another specific path.

I dug into the stack overflow issue, and I believe the root cause is a nested block_in_place + block_on call stack on a Tokio worker thread.

Context:

• REVM uses WrapDatabaseAsync to bridge async QMDB reads into a sync DatabaseRef.
• WrapDatabaseAsync does block_in_place(|| handle.block_on(async_db_call)) on the current Tokio worker thread.
• Our finalized path was running persist_snapshot on that same worker thread.

Why this crashes:

• While block_on is active, Tokio keeps the runtime loop alive on that thread.
• That means other ready tasks can be polled inside that block_on call.
• If the finalize task runs there, persist_snapshot -> QMDB open_stores -> storage open ends up nested under the WrapDatabaseAsync stack.
• The call stack gets very deep and can overflow (the crash shows up at validate_partition_name, but it’s just where the stack blows).

Fix:

• Running persistence on the blocking pool (context.shared(true)) moves it to a dedicated blocking thread.
• This isolates the heavy QMDB commit path from worker threads that might already be inside WrapDatabaseAsync.
• With that change, the stack overflow disappears.

So I believe this is a wiring issue in the example, not a bug in WrapDatabaseAsync. The workaround is the correct fix given the sync REVM + async QMDB bridge.

Not sure if you've played around with QMDB's batch interface (haven't looked at your changes lately) but I suspect it'll be a huge performance boost.