Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add feature to enable migrations to fix references to non-existent registers #387

Merged

Commits on Apr 11, 2024

  1. Add feature to fix references to non-existent register

    The intended use case is to enable migration programs in onflow/flow-go to fix
    broken references.  As of April 2024, only 10 registers in testnet (not mainnet)
    were found to have broken references and they seem to have resulted from a bug
    that was fixed 2 years ago by onflow/cadence#1565.
    
    A broken reference is a StorageID referencing a non-existent slab.
    
    This commit adds FixLoadedBrokenReferences(), which traverses loaded slabs and
    fixes broken references in maps.  To fix a map containing broken references,
    new function replaces broken map with empty map having the same StorageID and
    also removes all slabs in the old map.
    
    Limitations:
    - only fix broken references in map (this is intentional)
    - only traverse loaded slabs in deltas and cache
    
    IMPORTANT: This should not be used to silently fix unknown problems. It
    should only be used by migration programs to fix known problems which
    were determined to be appropriate to fix.
    fxamacker committed Apr 11, 2024
    Configuration menu
    Copy the full SHA
    479bc26 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    e9b3a1b View commit details
    Browse the repository at this point in the history
  3. Bump Go to 1.20 in ci.yml

    fxamacker committed Apr 11, 2024
    Configuration menu
    Copy the full SHA
    3a0c061 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    b7a81a3 View commit details
    Browse the repository at this point in the history
  5. Linted

    fxamacker committed Apr 11, 2024
    Configuration menu
    Copy the full SHA
    d5cb164 View commit details
    Browse the repository at this point in the history
  6. Improve test

    fxamacker committed Apr 11, 2024
    Configuration menu
    Copy the full SHA
    a887780 View commit details
    Browse the repository at this point in the history

Commits on Apr 15, 2024

  1. Update storage.go

    Co-authored-by: Bastian Müller <[email protected]>
    fxamacker and turbolent authored Apr 15, 2024
    Configuration menu
    Copy the full SHA
    3b05bb1 View commit details
    Browse the repository at this point in the history

Commits on Apr 16, 2024

  1. Allow callers to skip fixing a broken reference

    Currently, calls to this function can be limited by migration
    programs by specifying the 9 testnet accounts affected by 10
    registers with broken references on testnet.
    
    This commit allows callers to implement additional restrictions,
    so calling FixLoadedBrokenReferences for the affected 9 testnet
    accounts can be even more limited at the callers discretion.
    
    In practice, this change is not expected to produce different
    migration results because full migration tests before this change
    correctly fixed the 10 known registers in the 9 testnet accounts.
    
    This commit added predicate func(old Value) bool to
    PersistentSlabStorage.FixLoadedBrokenReferences() to control
    whether to fix a atree.Value containing broken references.
    
    Also modified PersistentSlabStorage.FixLoadedBrokenReferences()
    to return fixed storage IDs and skipped storage IDs.  Both returned
    values are of type map[StorageID][]StorageID, with key as
    root slab ID and value as all slab IDs containing broken references
    connected to the root.
    
    Also added more tests and improved existing tests.
    fxamacker committed Apr 16, 2024
    Configuration menu
    Copy the full SHA
    40aea0d View commit details
    Browse the repository at this point in the history
  2. Add PersistentSlabStorage.GetAllChildReferences()

    This commit adds GetAllChildReferences() which essentially wraps
    an existing internal function.
    fxamacker committed Apr 16, 2024
    Configuration menu
    Copy the full SHA
    3cead2a View commit details
    Browse the repository at this point in the history
  3. Merge pull request #391 from onflow/fxamacker/export-getAllChildRefer…

    …ences
    
    Add PersistentSlabStorage.GetAllChildReferences()
    fxamacker authored Apr 16, 2024
    Configuration menu
    Copy the full SHA
    4b0f707 View commit details
    Browse the repository at this point in the history