[NIT-4075] Extract keccak for ZKVM precompile substitution #4001

pmikolajczyk41 · 2025-11-13T11:45:12Z

Native nitro

This PR doesn't change anything for natively compiled nitro.

Wasm nitro

For wasm target compilation any Keccak hash computation that was done with crypto.Keccak256() or crypto.Keccak256Hash() will be now realized by an external call - the resulting wasm binary/library expects an external arbkeccak module exposing keccak256 function (see OffchainLabs/go-ethereum#578).

Almost all keccaking is done with these two function. However, there are still some places where we unnecessarily use stateful keccak. This might be problematic, since as a result, our hash delegation will not be applied there. Specifically:

in nitro excluding go-ethereum we have 1 such usage; [NIT-4123] Use stateless keccak where possible #4025 has been filed to fix it
in go-ethereum there are ~20 such usages; I opened an upstream PR for this: all: use stateless keccak ethereum/go-ethereum#33222; if it doesn't get merged, we have a backup PR for our fork: arbitrum: Unify keccaking go-ethereum#580

Running `replay.wasm`

with JIT

we inject a Rust binding to the tiny_keccak library, which is no-std friendly and we already did use it in arbutil:

"arbkeccak" => {
    "keccak256" => func!(arbkeccak::keccak256),
}

with arbitrator

we build a new wasm library arbkeccak providing expected functionality:

pub unsafe extern "C" fn arbkeccak__keccak256(in_buf_ptr: GuestPtr, in_buf_len: u32, out_buf_ptr: GuestPtr)

companion PR: OffchainLabs/go-ethereum#578
part of NIT-4075

This reverts commit d3b6472.

…-keccak

github-actions · 2025-11-13T14:30:14Z

❌ 3 Tests Failed:

Tests completed	Failed	Passed	Skipped
2216	3	2213	0

View the top 3 failed tests by shortest run time

TestVersion40

Stack Traces | 6.250s run time

... [CONTENT TRUNCATED: Keeping last 20 lines]
INFO [11-18|15:40:03.028] HTTP server stopped                      endpoint=127.0.0.1:44239
TRACE[11-18|15:40:03.028] P2P networking is spinning down
    precompile_inclusion_test.go:90: goroutine 455450 [running]:
        runtime/debug.Stack()
        	/opt/hostedtoolcache/go/1.25.4/x64/src/runtime/debug/stack.go:26 +0x5e
        github.com/offchainlabs/nitro/util/testhelpers.RequireImpl({0x40e0e30, 0xc095383dc0}, {0x409e680, 0xc2a0fcb380}, {0x0, 0x0, 0x0})
        	/home/runner/work/nitro/nitro/util/testhelpers/testhelpers.go:29 +0x55
        github.com/offchainlabs/nitro/system_tests.Require(0xc095383dc0, {0x409e680, 0xc2a0fcb380}, {0x0, 0x0, 0x0})
        	/home/runner/work/nitro/nitro/system_tests/common_test.go:1759 +0x5d
        github.com/offchainlabs/nitro/system_tests.testPrecompiles(0xc095383dc0, 0x28, {0xc11b077df8, 0x5, 0x39?})
        	/home/runner/work/nitro/nitro/system_tests/precompile_inclusion_test.go:90 +0x371
        github.com/offchainlabs/nitro/system_tests.TestVersion40(0xc095383dc0?)
        	/home/runner/work/nitro/nitro/system_tests/precompile_inclusion_test.go:71 +0x64b
        testing.tRunner(0xc095383dc0, 0x3d251c8)
        	/opt/hostedtoolcache/go/1.25.4/x64/src/testing/testing.go:1934 +0xea
        created by testing.(*T).Run in goroutine 1
        	/opt/hostedtoolcache/go/1.25.4/x64/src/testing/testing.go:1997 +0x465
        
    precompile_inclusion_test.go:90: �[31;1m [] execution aborted (timeout = 5s) �[0;0m
--- FAIL: TestVersion40 (6.25s)

TestVersion30

Stack Traces | 7.050s run time

... [CONTENT TRUNCATED: Keeping last 20 lines]
=== PAUSE TestVersion30
=== CONT  TestVersion30
    precompile_inclusion_test.go:90: goroutine 455449 [running]:
        runtime/debug.Stack()
        	/opt/hostedtoolcache/go/1.25.4/x64/src/runtime/debug/stack.go:26 +0x5e
        github.com/offchainlabs/nitro/util/testhelpers.RequireImpl({0x40e0e30, 0xc095383c00}, {0x409e680, 0xc2aeadfa40}, {0x0, 0x0, 0x0})
        	/home/runner/work/nitro/nitro/util/testhelpers/testhelpers.go:29 +0x55
        github.com/offchainlabs/nitro/system_tests.Require(0xc095383c00, {0x409e680, 0xc2aeadfa40}, {0x0, 0x0, 0x0})
        	/home/runner/work/nitro/nitro/system_tests/common_test.go:1759 +0x5d
        github.com/offchainlabs/nitro/system_tests.testPrecompiles(0xc095383c00, 0x1e, {0xc0c885fdb0, 0x6, 0x4f5de9?})
        	/home/runner/work/nitro/nitro/system_tests/precompile_inclusion_test.go:90 +0x371
        github.com/offchainlabs/nitro/system_tests.TestVersion30(0xc095383c00?)
        	/home/runner/work/nitro/nitro/system_tests/precompile_inclusion_test.go:67 +0x798
        testing.tRunner(0xc095383c00, 0x3d251c0)
        	/opt/hostedtoolcache/go/1.25.4/x64/src/testing/testing.go:1934 +0xea
        created by testing.(*T).Run in goroutine 1
        	/opt/hostedtoolcache/go/1.25.4/x64/src/testing/testing.go:1997 +0x465
        
    precompile_inclusion_test.go:90: �[31;1m [] execution aborted (timeout = 5s) �[0;0m
--- FAIL: TestVersion30 (7.05s)

TestOutOfGasInStorageCacheFlush

Stack Traces | 7.140s run time

... [CONTENT TRUNCATED: Keeping last 20 lines]
�[34;1mChecking success (Rust =&gt; Solidity =&gt; Rust)�[0;0m
�[34;1mChecking call with value (Rust =&gt; EOA)�[0;0m
    program_test.go:2813: goroutine 685232 [running]:
        runtime/debug.Stack()
        	/opt/hostedtoolcache/go/1.25.4/x64/src/runtime/debug/stack.go:26 +0x5e
        github.com/offchainlabs/nitro/util/testhelpers.RequireImpl({0x40e0e30, 0xc0d6384fc0}, {0x409e200, 0xc15ac69560}, {0x0, 0x0, 0x0})
        	/home/runner/work/nitro/nitro/util/testhelpers/testhelpers.go:29 +0x55
        github.com/offchainlabs/nitro/system_tests.Require(0xc0d6384fc0, {0x409e200, 0xc15ac69560}, {0x0, 0x0, 0x0})
        	/home/runner/work/nitro/nitro/system_tests/common_test.go:1759 +0x5d
        github.com/offchainlabs/nitro/system_tests.TestOutOfGasInStorageCacheFlush(0xc0d6384fc0)
        	/home/runner/work/nitro/nitro/system_tests/program_test.go:2813 +0xfe6
        testing.tRunner(0xc0d6384fc0, 0x3d24ce8)
        	/opt/hostedtoolcache/go/1.25.4/x64/src/testing/testing.go:1934 +0xea
        created by testing.(*T).Run in goroutine 1
        	/opt/hostedtoolcache/go/1.25.4/x64/src/testing/testing.go:1997 +0x465
        
    program_test.go:2813: �[31;1m [] failed calculating position for validation: batch not found on L1 yet �[0;0m
INFO [11-18|15:39:41.418] HTTP server stopped                      endpoint=127.0.0.1:43583
TRACE[11-18|15:39:41.418] P2P networking is spinning down
--- FAIL: TestOutOfGasInStorageCacheFlush (7.14s)

📣 Thoughts on this report? Let Codecov know! | Powered by Codecov

bragaigor

In these set of PRs we introduce keccak for ZKVM precompile but only start using in some another PR, the ones mentioned in the description? Or am I missing something? And for my own sanity, the introduced keccak for replay.wasm uses the same implementation since wasm counterpart calls into arbkeccak::keccak256 as well?

arbitrator/wasm-libraries/arbkeccak/Cargo.toml

pmikolajczyk41 · 2025-11-19T11:14:50Z

In these set of PRs we introduce keccak for ZKVM precompile but only start using in some another PR, the ones mentioned in the description? Or am I missing something?

This set of PRs introduce just one thing: whenever we compile nitro into wasm (e.g. cmd/replay/main.go), instead of using keccak implementation from a Golang lib sha3 (it was wrapped in the go-ethereum/crypto package), we will use expect that there will be also a wasm module called arbkeccak provided, with a method keccak256. For 2 specific cases, JIT replay and Arbitrator replay we provide this module either as a hacky injection (JIT case) or a standard wasm module (arbitrator case).

The other PRs mentioned in the description are just about ensuring that there are no other keccak hashing across the codebase that would still use the old way.

And for my own sanity, the introduced keccak for replay.wasm uses the same implementation since wasm counterpart calls into arbkeccak::keccak256 as well?

after this PR, the replay.wasm will start using the new implementation that comes from Rust

pmikolajczyk41 added 17 commits November 13, 2025 12:10

Create arbkeccak dummy package

03c2484

Remove Keccak512 (not used, avoid future collision)

879e77e

Move crypto.Keccak* to arbkeccak

45a8707

Copy the implementation for the wasm world (temporarily)

b432bba

Do refactor

0e3a918

JIT working!

24ff162

Keep keccak implementation in jit

d3b6472

Add new wasm library (wip)

f4871f4

Revert "Keep keccak implementation in jit"

b2faa32

This reverts commit d3b6472.

Wasm library refer to caller-env

39786cc

Makefile

200d423

Merge

0eb3642

Merge remote-tracking branch 'origin/master' into pmikolajczyk/NIT-4075…

597f04e

…-keccak

fix

0022541

Add keccak_wasm.go

4c04594

Revert import

636279c

Lint

5dcb2ff

pmikolajczyk41 mentioned this pull request Nov 13, 2025

arbitrum: Outsource keccak implementation for wasm OffchainLabs/go-ethereum#578

Open

pmikolajczyk41 added 5 commits November 13, 2025 15:36

remove print

fea45e5

minor code adjustment

460c6b0

merge

c587535

minor refactor

395a162

linter

19c7403

pmikolajczyk41 requested review from bragaigor and tsahee November 18, 2025 13:25

pmikolajczyk41 assigned tsahee Nov 18, 2025

pmikolajczyk41 marked this pull request as ready for review November 18, 2025 13:25

linter

26bc970

bragaigor reviewed Nov 19, 2025

View reviewed changes

arbitrator/wasm-libraries/arbkeccak/Cargo.toml Show resolved Hide resolved

pmikolajczyk41 requested a review from bragaigor November 19, 2025 11:14

bragaigor approved these changes Nov 19, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[NIT-4075] Extract keccak for ZKVM precompile substitution #4001

[NIT-4075] Extract keccak for ZKVM precompile substitution #4001

pmikolajczyk41 commented Nov 13, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Nov 13, 2025 •

edited

Loading

Uh oh!

bragaigor left a comment •

edited

Loading

Uh oh!

Uh oh!

pmikolajczyk41 commented Nov 19, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

[NIT-4075] Extract keccak for ZKVM precompile substitution #4001

Are you sure you want to change the base?

[NIT-4075] Extract keccak for ZKVM precompile substitution #4001

Conversation

pmikolajczyk41 commented Nov 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Native nitro

Wasm nitro

Running replay.wasm

with JIT

with arbitrator

Uh oh!

github-actions bot commented Nov 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

❌ 3 Tests Failed:

Uh oh!

bragaigor left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

pmikolajczyk41 commented Nov 19, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

pmikolajczyk41 commented Nov 13, 2025 •

edited

Loading

Running `replay.wasm`

github-actions bot commented Nov 13, 2025 •

edited

Loading

bragaigor left a comment •

edited

Loading