-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
perf(tree): chunk multiproofs #14800
Merged
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
5cbcb6f
to
8cd112a
Compare
8cd112a
to
8f2ff1d
Compare
f688e1e
to
40eaed8
Compare
94bfa41
to
3a09209
Compare
3a09209
to
e70357a
Compare
CodSpeed Performance ReportMerging #14800 will not alter performanceComparing Summary
|
807dcd2
to
a45b67a
Compare
93b989d
to
1690584
Compare
c8ca55b
to
ed0829a
Compare
mattsse
reviewed
Mar 6, 2025
56970f8
to
688c4b5
Compare
688c4b5
to
37e6919
Compare
mattsse
approved these changes
Mar 6, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I can follow the logic here, defer to @rkrasiuk re chunking
looks sound to me
rkrasiuk
approved these changes
Mar 6, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Towards #14417
Problem
Given proof targets with 100 accounts, it's preferred to spawn 10 multiproofs with 10 accounts each, instead of spawning one multiproof for all 100 accounts at once. The reason for that is that 10 multiproofs can be executed in parallel (even considering that they may do redundant work because of the intersecting trie nodes), while one multiproof will be executed serially for all accounts.
Same idea can be applied to storage slots: proof targets with 1 account and 100 storage slots is better spawned as 10 multiproofs for 10 storage slots each, rather than one multiproof for all 100 storage slots at once.
Solution
This PR adds chunking on the
MultiProofTask
side.on_prefetch_proof
, proof targets are chunked and spawned as separate multiproofs.on_state_update
, state update is chunked, proof targets are created for each chunk, and spawned as separate multiproofs.Benchmark