Add feature to enable migrations to fix references to non-existent registers #387
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The intended use case is to enable migration programs in onflow/flow-go to fix broken references. As of April 2024, only 10 registers in testnet (not mainnet) were found to have broken references and they seem to have resulted from a bug that was fixed 2 years ago by onflow/cadence#1565. A broken reference is a StorageID referencing a non-existent slab. This commit adds FixLoadedBrokenReferences(), which traverses loaded slabs and fixes broken references in maps. To fix a map containing broken references, new function replaces broken map with empty map having the same StorageID and also removes all slabs in the old map. Limitations: - only fix broken references in map (this is intentional) - only traverse loaded slabs in deltas and cache IMPORTANT: This should not be used to silently fix unknown problems. It should only be used by migration programs to fix known problems which were determined to be appropriate to fix.
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## feature/stable-cadence #387 +/- ##
==========================================================
+ Coverage 66.09% 66.48% +0.38%
==========================================================
Files 14 14
Lines 8233 8390 +157
==========================================================
+ Hits 5442 5578 +136
- Misses 2113 2124 +11
- Partials 678 688 +10 ☔ View full report in Codecov by Sentry. |
SupunS
reviewed
Apr 12, 2024
turbolent
reviewed
Apr 12, 2024
Co-authored-by: Bastian Müller <[email protected]>
Merged
7 tasks
fxamacker
force-pushed
the
fxamacker/add-fix-broken-reference-function
branch
from
1a7d3c8 to
46df965
Compare
Currently, calls to this function can be limited by migration programs by specifying the 9 testnet accounts affected by 10 registers with broken references on testnet. This commit allows callers to implement additional restrictions, so calling FixLoadedBrokenReferences for the affected 9 testnet accounts can be even more limited at the callers discretion. In practice, this change is not expected to produce different migration results because full migration tests before this change correctly fixed the 10 known registers in the 9 testnet accounts. This commit added predicate func(old Value) bool to PersistentSlabStorage.FixLoadedBrokenReferences() to control whether to fix a atree.Value containing broken references. Also modified PersistentSlabStorage.FixLoadedBrokenReferences() to return fixed storage IDs and skipped storage IDs. Both returned values are of type map[StorageID][]StorageID, with key as root slab ID and value as all slab IDs containing broken references connected to the root. Also added more tests and improved existing tests.
fxamacker
force-pushed
the
fxamacker/add-fix-broken-reference-function
branch
from
46df965 to
40aea0d
Compare
turbolent
approved these changes
Apr 16, 2024
This commit adds GetAllChildReferences() which essentially wraps an existing internal function.
6 tasks
…ences Add PersistentSlabStorage.GetAllChildReferences()
This was referenced Apr 17, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Updates #386
Problem
In testnet, broken references seem to have resulted from a bug that was fixed 2 years ago by onflow/cadence#1565.
A broken reference is a
StorageIDreferencing a non-existent register. So far, only 10 registers in testnet (none on mainnet) were found to contain broken references.Solution
This PR adds a feature to enable migration programs in onflow/flow-go to fix broken references in maps.
FixLoadedBrokenReferences()traverses loaded slabs and replaces broken map (if any) with empty map having the same StorageID and also removes all slabs in the old map.Limitations:
IMPORTANT: This should not be used to silently fix unknown problems. It should only be used by migration programs to fix known problems which were determined to be appropriate to fix in this manner.
TODO:
mainbranchFiles changedin the Github PR explorer