Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge upstream 20231016 #53

Merged
merged 50 commits into from
Oct 16, 2023
Merged

Conversation

jyellick
Copy link

No description provided.

Giulio2002 and others added 30 commits October 9, 2023 17:24
- Move borfinality out of eth package
- Adds nil pointer check in bor_verifier
This is an additional fix for BorRo to add bor config in the constructor
- otherwise code which accesses chain config will panic.
TruncateCanonicalHash should only have all pruned in the case of a
badBlock or a fork unwind
Fixes and issue with Polygon validators where locally mined blocks are
broadcast with invalid header hashes because the NewBlock message
constructor was removing the ReceiptHash which contributed to the header
hash.

The results in the bor header validation code not being able to
correctly identify the signer of the header - so header validation
fails.

This also likely fixes part of the bogon-block issue which was
identified by the polygon team.
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.15.0 to
0.17.0.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/golang/net/commit/b225e7ca6dde1ef5a5ae5ce922861bda011cfabd"><code>b225e7c</code></a>
http2: limit maximum handler goroutines to MaxConcurrentStreams</li>
<li><a
href="https://github.com/golang/net/commit/88194ad8ab44a02ea952c169883c3f57db6cf9f4"><code>88194ad</code></a>
go.mod: update golang.org/x dependencies</li>
<li><a
href="https://github.com/golang/net/commit/2b60a61f1e4cf3a5ecded0bd7e77ea168289e6de"><code>2b60a61</code></a>
quic: fix several bugs in flow control accounting</li>
<li><a
href="https://github.com/golang/net/commit/73d82efb96cacc0c378bc150b56675fc191894b9"><code>73d82ef</code></a>
quic: handle DATA_BLOCKED frames</li>
<li><a
href="https://github.com/golang/net/commit/5d5a036a503f8accd748f7453c0162115187be13"><code>5d5a036</code></a>
quic: handle streams moving from the data queue to the meta queue</li>
<li><a
href="https://github.com/golang/net/commit/350aad2603e57013fafb1a9e2089a382fe67dc80"><code>350aad2</code></a>
quic: correctly extend peer's flow control window after MAX_DATA</li>
<li><a
href="https://github.com/golang/net/commit/21814e71db756f39b69fb1a3e06350fa555a79b1"><code>21814e7</code></a>
quic: validate connection id transport parameters</li>
<li><a
href="https://github.com/golang/net/commit/a600b3518eed7a9a4e24380b4b249cb986d9b64d"><code>a600b35</code></a>
quic: avoid redundant MAX_DATA updates</li>
<li><a
href="https://github.com/golang/net/commit/ea633599b58dc6a50d33c7f5438edfaa8bc313df"><code>ea63359</code></a>
http2: check stream body is present on read timeout</li>
<li><a
href="https://github.com/golang/net/commit/ddd8598e5694aa5e966e44573a53e895f6fa5eb2"><code>ddd8598</code></a>
quic: version negotiation</li>
<li>Additional commits viewable in <a
href="https://github.com/golang/net/compare/v0.15.0...v0.17.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=golang.org/x/net&package-manager=go_modules&previous-version=0.15.0&new-version=0.17.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/ledgerwatch/erigon/network/alerts).

</details>

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: yperbasis <[email protected]>
This PR removes the empty headers check when responding to
`GetBlockHeaders` p2p request. Currently, as we return from the
function, the requests on the caller's end gets timed out causing peer
drop. This will prevent that from happening by sending empty header list
which will be of no use but still helps in maintaining the peer
connection.

With this change, those running erigon 2.48 with
`--sentry.drop-useless-peers` will drop peers as they consider receiving
empty headers a harsh move which is fine.

We have also discovered that if we don't remove this check maintaining
connectivity to Bor is not maintained. This is significant as Erigon
becomes a minority validator, as bor chain stability (reducing forks) is
dependent upon good validator connectivity.
…rigontech#8449)

## Description

Add the possibility to change the RPC address used in `devnet` without
requiring a rebuild.

## Test

- [x] `make devnet`
- [x] `./build/bin/devnet --datadir tmp`
- [x] `./build/bin/devnet --datadir tmp --rpc.host localhost --rpc.port
8546`
Mitigate [no space left on
device](https://github.com/ledgerwatch/erigon/actions/runs/6493703302/job/17639876934)
in GoReleaser. See
https://github.com/AdityaGarg8/remove-unwanted-software.

This is a simplified version of PR erigontech#8451 (there I had to add
`root-reserve-mb: 16384` to make it work).
Add a check against the inbound headers before publishing a newly mined
block after the wait delay.

If the node received a block while it was processing transactions, or
waiting for its publish slot, do a final check that another node hasn't
already published a block.
Giulio2002 and others added 19 commits October 13, 2023 15:22
… with empty receipt hash (erigontech#8467)

## Description

This PR is a follow-up of erigontech#8442.

1. The previous PR resolved the issue where newly mined blocks had their
receipt hash set to an empty root hash. But I think it introduced an
issue where the new blocks include no transactions. So this PR addresses
that.

2. On top of that, this new PR introduces a validation step for the
receipt hash when peers receive new blocks at the p2p layer. If the
receipt hash is set to the empty root hash, but the block contains
transactions, it will now raise an error.

## Test

- When running a local erigon devnet with the new receipt hash check and
before erigontech#8442 fix (with a few more logs).

**=> error message: `newBlock66: block has empty receipt hash:
56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421 but it
includes 2 transactions`**

```sh
Received new block request: &{Block:0xc000bd0230 TD:+7891}
Block header: &{ParentHash:0x998f5be682d09b9d57dbf9c42032c23754bc9569d39670a1d7479f74946df0a9 UncleHash:0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347 Coinbase:0x0000000000000000000000000000000000000000 Root:0x422bd23663d6d52b94cd537491eaa2b250b03e5e252b9c1dfabee5dc12673628 TxHash:0xe4ec65ce3aba6347c9e7139571cd23ec5e42b3c1952706a3d965a322699b9b5e ReceiptHash:0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421 Bloom:[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] Difficulty:+2 Number:+3945 GasLimit:30000000 GasUsed:70178 Time:1697192495 Extra:[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 197 254 60 127 243 72 143 204 98 97 12 136 59 50 115 115 204 239 143 157 234 71 55 113 77 66 2 227 74 65 226 190 44 226 215 32 177 90 161 172 41 69 31 229 166 221 185 14 128 47 169 132 130 249 179 129 153 159 134 124 162 234 81 187 1] MixDigest:0x0000000000000000000000000000000000000000000000000000000000000000 Nonce:[0 0 0 0 0 0 0 0] AuRaStep:0 AuRaSeal:[] BaseFee:+8 WithdrawalsHash:<nil> BlobGasUsed:<nil> ExcessBlobGas:<nil> ParentBeaconBlockRoot:<nil> Verkle:false VerkleProof:[] VerkleKeyVals:[]}
Block body: &{Transactions:[0xc000cde100 0xc000cde200] Uncles:[] Withdrawals:[]}
DBUG[10-13|12:21:35.789] Handling incoming message                stream=RecvMessage err="newBlock66: block has empty receipt hash: 56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421 but it includes 2 transactions"
...
```

- When running a local erigon devnet with the new receipt hash check
after erigontech#8442 fix (with a few more logs).

**=> error message: `newBlock66: block has invalid transaction hash
...`**

```sh
Received new block request: &{Block:0xc002c38930 TD:+11963}
Block header: &{ParentHash:0x89a5134df82c85b38f2e02524143b575c44c5c6df444df9fc4d86c5972601ce0 UncleHash:0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347 Coinbase:0x0000000000000000000000000000000000000000 Root:0x1ab62696e1b585c8e084e12781e2f36f2591f91fe45e5b125df2468abd1f54e5 TxHash:0x6e7642d32215ab23f9096ed36463730ffa6c17c26dbba50c6100d64b5dee6349 ReceiptHash:0x9c8f5e4d71a8bb3ace2c0ce45e00357278fcac6da41b9e353872f106298cf6ec Bloom:[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 8 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 8 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 16 0 0 0 0 0 32 0 0 0 0 0 0 0 16 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 128 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 8 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 16 0 16 0 0 0] Difficulty:+2 Number:+5981 GasLimit:30000000 GasUsed:70178 Time:1697193936 Extra:[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 187 75 33 182 137 174 13 214 132 232 242 141 222 107 53 186 250 113 185 149 118 134 112 84 167 16 126 38 158 252 205 66 117 96 205 198 79 158 82 42 199 29 37 92 23 3 140 113 216 66 105 174 144 209 110 121 119 43 165 204 81 54 113 1] MixDigest:0x0000000000000000000000000000000000000000000000000000000000000000 Nonce:[0 0 0 0 0 0 0 0] AuRaStep:0 AuRaSeal:[] BaseFee:+8 WithdrawalsHash:<nil> BlobGasUsed:<nil> ExcessBlobGas:<nil> ParentBeaconBlockRoot:<nil> Verkle:false VerkleProof:[] VerkleKeyVals:[]}
Block body: &{Transactions:[] Uncles:[] Withdrawals:[]}
DBUG[10-13|12:45:36.908] Handling incoming message                stream=RecvMessage err="newBlock66: block has invalid transaction hash: have 56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421, exp: 6e7642d32215ab23f9096ed36463730ffa6c17c26dbba50c6100d64b5dee6349"
...
```

- When running a local erigon devnet with the new receipt hash check and
after this PR's fix (include transactions in block)

**=> no error message 🎉**

```sh
Received new block request: &{Block:0xc000d5a000 TD:+1227}
Block header: &{ParentHash:0xf5d6a19c8d56f230eeb8d2654e5562b934236136d5fcc50cbcde5c9ccc659810 UncleHash:0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347 Coinbase:0x0000000000000000000000000000000000000000 Root:0x9c17098ebfda6e59ceef147778367c628a44a6f1421ef21e844a9d8450f64a74 TxHash:0x507daf8bc5b329226614a0cf535273242cbf15d5ad582a4d5db0d5881732f791 ReceiptHash:0xaea5f86a33cba20385d4ce630b5f7ff613139b374718d89eafe224bd4b57e903 Bloom:[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 4 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] Difficulty:+2 Number:+613 GasLimit:20918370 GasUsed:54544 Time:1697195980 Extra:[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 118 30 19 173 212 211 186 172 0 200 117 103 101 19 76 147 18 161 141 70 70 237 243 162 239 236 162 35 154 248 10 28 90 20 214 32 140 72 23 151 36 212 210 99 122 154 244 229 171 249 29 105 109 23 80 112 142 65 235 189 59 238 8 27 1] MixDigest:0x0000000000000000000000000000000000000000000000000000000000000000 Nonce:[0 0 0 0 0 0 0 0] AuRaStep:0 AuRaSeal:[] BaseFee:+8 WithdrawalsHash:<nil> BlobGasUsed:<nil> ExcessBlobGas:<nil> ParentBeaconBlockRoot:<nil> Verkle:false VerkleProof:[] VerkleKeyVals:[]}
Block body: &{Transactions:[0xc0000dc400 0xc0000dc600] Uncles:[] Withdrawals:[]}
...
```
If HeaderDownload.VerifyHeader always returns false, the memory usage
grows at a fast pace
due to Link objects (containing headers) not deallocated even after the
link queue pruning.
Consensus Specification Tests takes less than 8 minutes so I think they
can be in a PR's own CI for whenever it is ready. for reference it is
less than make test
db must not use "Durable" and "SyncPeriod" in same time

txpool and downloader must be: Durable
nodedb:  SafeNoSync+SyncPeriod

because txpool and downloader already can call sync when they need
(txpool after regular flush, downloader after piece download+validation
completion).
on 16-core mumbai's initFrozenSnapshot took 10min (to 38M block)
@jyellick jyellick changed the base branch from devel to boba-develop October 16, 2023 14:24
…eam-20231016

Dependent on proto change merging into v3-erigon-interfaces
@jyellick jyellick merged commit 22c2796 into boba-develop Oct 16, 2023
9 checks passed
jyellick pushed a commit that referenced this pull request Feb 26, 2024
Fix docker image tag in hive workflow
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.