Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Account Storage Maps] Reduce storage register reads when using StorageFormatV2Enabled #3683

Conversation

fxamacker
Copy link
Member

@fxamacker fxamacker commented Nov 19, 2024

Updates #3584

Currently in PR #3678, all domain registers are being read for a scenario that does not require it. This PR reduces register reads as discussed.

Problem

In some cases, we can reduce 10 register reads to only 2 register reads. 🚀

When StorageFormatV2 feature flag is enabled, Storage.GetDomainStorageMap() reads all domain registers to determine if account is new account or in v1 format.

These register reads are unnecessary when:

  • the given domain register exists, or
  • the given domain register doesn't exist and createIfNotExists param is false.

Solution

This PR modifies Storage.GetDomainStorageMap() to read the given domain register first (before iterating all domain registers) to reduce register reads.

  • If the given domain register exists, then account format is v1 and no more register reading is needed.

  • If the given domain register doesn't exist and createIfNotExists is false, then nil domain storage map is returned and no more register reading is needed.

  • Otherwise, account format (v1 account or new account) is determined by iterating all domain registers.

Reduced Reads

When using StorageFormatV2Enabled:

For new accounts (neither v1 or v2 yet):

For v1 accounts:

Caveat

This change increases code complexity in order to avoid reading all domain registers for this scenario. I added more comments to offset some of this tradeoff. For more context, see comment thread at #3678.

UPDATE:

I can add more tests for register reading after sync with Cadence team about this approach.
I added more tests in case we don't get chance to sync with Cadence team this week.


  • Targeted PR against master branch
  • Linked to Github issue with discussion and accepted design OR link to spec that describes this work
  • Code follows the standards mentioned here
  • Updated relevant documentation
  • Re-reviewed Files changed in the Github PR explorer
  • Added appropriate labels

Currently, all domain registers are being read for a scenario
that does not require it.

When StorageFormatV2 feature flag is enabled,
Storage.GetDomainStorageMap() reads all domain registers to
determine if account is new account or in V1 format.

These register reads are unnecessary for V1 accounts when:
- the given domain register exists, or
- the given domain register doesn't exist and
  createIfNotExists param is false.

This commit modifies Storage.GetDomainStorageMap() to read the
given domain register first (before iterating all domain
registers) to reduce register reads.

- If the given domain register exists, then account format is V1
and no more register reading is needed.

- If the given domain register doesn't exist and
createIfNotExists is false, then nil domain storage map is
returned and no more register reading is needed.

- Otherwise, account format (V1 account or new account) is
determined by iterating all domain registers.
@fxamacker fxamacker self-assigned this Nov 19, 2024
Base automatically changed from bastian/refactor-account-storage to feature/combine-domain-payloads-and-domain-storage-maps November 20, 2024 03:11
@turbolent turbolent requested a review from jsproz November 20, 2024 03:11
This commit adds tests for register reads from GetStorageMap()
for account storage format v1 and v2.
Copy link

github-actions bot commented Nov 20, 2024

Cadence Benchstat comparison

This branch with compared with the base branch onflow:feature/combine-domain-payloads-and-domain-storage-maps commit 157d322
The command for i in {1..N}; do go test ./... -run=XXX -bench=. -benchmem -shuffle=on; done was used.
Bench tests were run a total of 7 times on each branch.

Collapsed results for better readability

Copy link
Member

@turbolent turbolent left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work improving this!

The changes look good, I left some very minor suggestions.

I'm having a bit of a harder time understanding when/where we have unnecessary register reads currently, and how the changes of the PR remove the unnecessary register reads, especially because it seems to add an additional read in one of the existing tests.

Could we maybe add a test case to feature/combine-domain-payloads-and-domain-storage-maps before this PR that demonstrates the inefficiency (unnecessary reads), and then this PR could would show more clearly how the unnecessary read(s) are removed?

runtime/sharedstate_test.go Show resolved Hide resolved
runtime/storage.go Outdated Show resolved Hide resolved
runtime/storage.go Outdated Show resolved Hide resolved
runtime/storage.go Outdated Show resolved Hide resolved
runtime/storage.go Outdated Show resolved Hide resolved
runtime/storage.go Outdated Show resolved Hide resolved
runtime/storage.go Show resolved Hide resolved
runtime/storage.go Outdated Show resolved Hide resolved
Currently, tests for register reads only check sequence of
register reading.  However, same register can be read multiple
times and all register reads are included in the sequence.

This commit updates tests to explicitly checks unique registers
being read in addition to sequence of registers being read.
@turbolent turbolent changed the title Reduce storage register reads when using StorageFormatV2Enabled [Account Storage Maps] Reduce storage register reads when using StorageFormatV2Enabled Nov 25, 2024
@fxamacker
Copy link
Member Author

@turbolent there were some questions related to GetDomainStorageMap() and getAccountStorageFormat(), so I refactored the code to improve readability.

Also, the updated storage_tests.go shows the reduced register reads (expectedReadsSet).

PTAL. 🙏

@fxamacker fxamacker requested a review from turbolent November 26, 2024 00:07
Copy link
Member

@turbolent turbolent left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work! The optimization of reducing the register reads is really clearly evident in the tests now, well done 👏

@fxamacker fxamacker merged commit 5646c24 into feature/combine-domain-payloads-and-domain-storage-maps Nov 26, 2024
7 of 9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants