Discrepancies in tracer outputs between Geth and Nethermind

[wurdum]

Description
After the unification of debug_trace* endpoints output, we've started spotting minor discrepancies between Geth and Nethermind 4byteTracer, callTracer, prestateTracer tracer outputs. Those diffs can be found in almost every block.

Steps to Reproduce
In order to replicate the behavior, please provide a detailed list of steps:

1. Download trace_blocks.sh script attached to the issue
2. Create .env file with NETH_URL and GETH_URL variables that point to the Nethermind and Geth nodes respectively
3. Make the script executable and run it
4. Check .diff files

Desktop (please complete the following information):
Please provide the following information regarding your setup:

• Geth version: Geth/v1.14.13-stable-eb00f169/linux-amd64/go1.23.5
• Nethermind version: Align debug_traceBlock/Transaction with geth #8186
• Chain: Sepolia

Additional context
All traces files and diffs contain block number in their name.

Logs
Example traces and diffs are attached to the issue and listed below.

# 4byteTracer
1013c1013,1015
<       "result": {},
---
>       "result": {
>         "0x60806040-1564": 1
>       },


# callTracer
4028a4029
>             "error": "execution reverted",
4031c4032
<             "gasUsed": "0x4f3f",
---
>             "gasUsed": "0x628c",
4033d4033
<             "output": "0x0000000000000000000000000000000000000000000000000000000000000000",


# prestateTracer
1943c1943,1946
<           "nonce": 2
---
>           "nonce": 2,
>           "storage": {
>             "0x360894a13ba1a3210667c828492db98dca3e2076cc3735a920a3ca505d382bbc": "0x00000000000000000000000054ef1ce07790c714920473d15178ccdaf1bde8ab"
>           }
2184c2187,2190
<           "nonce": 2
---
>           "nonce": 2,
>           "storage": {
>             "0x360894a13ba1a3210667c828492db98dca3e2076cc3735a920a3ca505d382bbc": "0x00000000000000000000000054ef1ce07790c714920473d15178ccdaf1bde8ab"
>           }
2239c2245,2248
<           "nonce": 2
---
>           "nonce": 2,
>           "storage": {
>             "0x360894a13ba1a3210667c828492db98dca3e2076cc3735a920a3ca505d382bbc": "0x00000000000000000000000054ef1ce07790c714920473d15178ccdaf1bde8ab"
>           }
2337c2346,2349
<           "nonce": 2
---
>           "nonce": 2,
>           "storage": {
>             "0x360894a13ba1a3210667c828492db98dca3e2076cc3735a920a3ca505d382bbc": "0x00000000000000000000000054ef1ce07790c714920473d15178ccdaf1bde8ab"
>           }
5592a5605,5607
>         },
>         "0x46899cfe8f727cf8b892a4712a038c2a5c2bde1d": {
>           "balance": "0x0"

traceBlock_0x757c82_geth_4byteTracer.json
traceBlock_0x757c82_geth_callTracer.json
traceBlock_0x757c82_geth_prestateTracer.json
traceBlock_0x757c82_neth_4byteTracer.json
traceBlock_0x757c82_neth_callTracer.json
traceBlock_0x757c82_neth_prestateTracer.json

traceBlock_0x757c82_geth-vs-neth_4byteTracer.diff.txt
traceBlock_0x757c82_geth-vs-neth_callTracer.diff.txt
traceBlock_0x757c82_geth-vs-neth_prestateTracer.diff.txt

trace_blocks.sh.txt

[wurdum]

Analysis results:

Discrepancies in 4byteTracer output are happening for transactions with the type=CREATE when the call is not nested in a CALL transaction (Geth doesn't have 4byte record).

Diff in the output:

{
    "result": {
++      "0x60806040-408": 1
    },
    "txHash": "0xb24844b0e2b53be7d68fa106f46f59b7bdaf6856ee4b9fb8f9faabfabd00f2ad"
}

An example of callTracer output for the same transaction:

{
    "result": {
        "from": "0x887dd7018e981083cc67e4109e409349598bf395",
        "gas": "0x31e54",
        "gasUsed": "0x20fd9",
        "input": "",
        "output": "0x608060405234801561001057600080f...",
        "to": "0x8958dfa32229e9edd0f1bcc12a89f1c17865e8ac",
        "type": "CREATE",
        "value": "0x0"
    },
    "txHash": "0xb24844b0e2b53be7d68fa106f46f59b7bdaf6856ee4b9fb8f9faabfabd00f2ad"
}

callTracer output discrepancies are happening in cases when a transaction has failed. It can be encountered as:

• Difference in gasUsed value of the nested call
• Presence of output field in the callTracer output (in Geth)
• Presence of nested CALL object in the callTracer output (in Geth)

In all mentioned cases the root transaction object of the tracer output is the same for both clients. See the files attached for the examples.

4byteTracer-1-create-geth.json
4byteTracer-1-create-neth.json

callTracer-1-out-of-gas-gasUsed-geth.json
callTracer-1-out-of-gas-gasUsed-neth.json
callTracer-2-revert-ins-balance-geth.json
callTracer-2-revert-ins-balance-neth.json
callTracer-3-revert-gasUsed-output-geth.json
callTracer-3-revert-gasUsed-output-neth.json
callTracer-4-revert-exceed-allowance-geth.json
callTracer-4-revert-exceed-allowance-neth.json

[wurdum]

Following the ticket in Geth repo ethereum/go-ethereum#31204 (comment), it looks like it's rather a bug in Nethermind and we shouldn't return 4byte IDs of contract creation calldata.