Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Region.mo library offering isolated regions of IC stable memory #580

Merged
merged 8 commits into from
Sep 5, 2023
+382 −1
Merged

feat: Region.mo library offering isolated regions of IC stable memory #580

Changes from 2 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
9aa7aa3
firt pass at Region.mo
crusso Jul 31, 2023
38c9b2d
fixes
crusso Aug 1, 2023
becf8d6
doc new and regionId
crusso Aug 1, 2023
1421587
examples for new and id
crusso Aug 2, 2023
85fd0da
reword
crusso Aug 2, 2023
23ae6a1
Apply suggestions from code review
crusso Aug 3, 2023
84397c1
Update Region.mo
crusso Aug 23, 2023
5691fd6
Merge remote-tracking branch 'origin/next-moc' into claudio/next-moc-…
crusso Sep 5, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
357 changes: 357 additions & 0 deletions src/Region.mo
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,357 @@
/// Isolated, byte-level access to (virtual) _stable memory_ regions.
///
/// This is a lightweight abstraction over IC _stable memory_ and supports persisting
/// raw binary data across Motoko upgrades.
/// Use of this module is fully compatible with Motoko's use of
/// _stable variables_, whose persistence mechanism also uses (real) IC stable memory internally, but does not interfere with this API.
/// It is also fully compatible with existing uses of the `ExperimentalStableMemory` library, which has a similar interface, but,
/// only supported a single memory region.
///
/// Memory is allocated, using `grow(pages)`, sequentially and on demand, in units of 64KiB logical pages, starting with 0 allocated pages.
/// New pages are zero initialized.
/// Growth is capped by a soft limit on physical page count controlled by compile-time flag
/// `--max-stable-pages <n>` (the default is 65536, or 4GiB).
///
/// Each `load` operation loads from byte address `offset` in little-endian
/// format using the natural bit-width of the type in question.
/// The operation traps if attempting to read beyond the current stable memory size.
///
/// Each `store` operation stores to byte address `offset` in little-endian format using the natural bit-width of the type in question.
/// The operation traps if attempting to write beyond the current stable memory size.
///
/// Text values can be handled by using `Text.decodeUtf8` and `Text.encodeUtf8`, in conjunction with `loadBlob` and `storeBlob`.
///
/// The current page allocation and page contents is preserved across upgrades.
///
/// NB: The IC's actual stable memory size (`ic0.stable_size`) may exceed the
/// total page size reported by summing all regions sizes.
/// This (and the cap on growth) are to accommodate Motoko's stable variables and bookkeeping for regions.
/// Applications that plan to use Motoko stable variables sparingly or not at all can
/// increase `--max-stable-pages` as desired, approaching the IC maximum (initially 8GiB, then 32Gib, currently 48Gib).
/// All applications should reserve at least one page for stable variable data, even when no stable variables are used.
///
/// Usage:
/// ```motoko no-repl
/// import Region "mo:base/Region";
/// ```

import Prim "mo:⛔";

module {

// A stateful handle to an isolated region of IC stable memory.
// Region is a stable type and regions can be stored in stable variables.
public type Region = Prim.Types.Region;

// Allocate a new, isolated Region of size 0.

This comment was marked as resolved.

Sign in to view

public let new = Prim.regionNew;

public let id = Prim.regionId;

This comment was marked as resolved.

Sign in to view


/// Current size of `region`, in pages.
/// Each page is 64KiB (65536 bytes).
/// Initially `0`.
/// Preserved across upgrades, together with contents of allocated
/// stable memory.
///
/// Example:
/// ```motoko no-repl
/// let region = Region.new();
/// let beforeSize = Region.size(region);
/// ignore Region.grow(region, 10);
/// let afterSize = Region.size(region);
/// afterSize - beforeSize // => 10
/// ```
public let size : (region : Region) -> (pages : Nat64) = Prim.regionSize;

/// Grow current `size` of `region` by the given number of pages.
/// Each page is 64KiB (65536 bytes).
/// Returns the previous `size` when able to grow.
/// Returns `0xFFFF_FFFF_FFFF_FFFF` if remaining pages insufficient.
/// Every new page is zero-initialized, containing byte 0x00 at every offset.
/// Function `grow` is capped by a soft limit on `size` controlled by compile-time flag
/// `--max-stable-pages <n>` (the default is 65536, or 4GiB).
///
/// Example:
/// ```motoko no-repl
/// import Error "mo:base/Error";
///
/// let region = Region.new();
/// let beforeSize = Region.grow(region, 10);
/// if (beforeSize == 0xFFFF_FFFF_FFFF_FFFF) {
/// throw Error.reject("Out of memory");
/// };
/// let afterSize = Region.size(region);
/// afterSize - beforeSize // => 10
/// ```
public let grow : (region : Region, newPages : Nat64) -> (pages : Nat64) = Prim.regionGrow;
crusso marked this conversation as resolved.
Show resolved Hide resolved


/// Within `region`, load a `Nat8` value from `offset`.
/// Traps on an out-of-bounds access.
///
/// Example:
/// ```motoko no-repl
/// let region = Region.new();
/// let offset = 0;
/// let value = 123;
/// Region.storeNat8(region, offset, value);
/// Region.loadNat8(region, offset) // => 123
/// ```
public let loadNat8 : (region : Region, offset : Nat64) -> Nat8 = Prim.regionLoadNat8;

/// Within `region`, store a `Nat8` value at `offset`.
/// Traps on an out-of-bounds access.
///
/// Example:
/// ```motoko no-repl
/// let region = Region.new();
/// let offset = 0;
/// let value = 123;
/// Region.storeNat8(region, offset, value);
/// Region.loadNat8(region, offset) // => 123
/// ```
public let storeNat8 : (region : Region, offset : Nat64, value : Nat8) -> () = Prim.regionStoreNat8;

/// Within `region`, load a `Nat16` value from `offset`.
/// Traps on an out-of-bounds access.
///
/// Example:
/// ```motoko no-repl
/// let region = Region.new();
/// let offset = 0;
/// let value = 123;
/// Region.storeNat16(region, offset, value);
/// Region.loadNat16(region, offset) // => 123
/// ```
public let loadNat16 : (region : Region, offset : Nat64) -> Nat16 = Prim.regionLoadNat16;

/// Within `region`, store a `Nat16` value at `offset`.
/// Traps on an out-of-bounds access.
///
/// Example:
/// ```motoko no-repl
/// let region = Region.new();
/// let offset = 0;
/// let value = 123;
/// Region.storeNat16(region, offset, value);
/// Region.loadNat16(region, offset) // => 123
/// ```
public let storeNat16 : (region : Region, offset : Nat64, value : Nat16) -> () = Prim.regionStoreNat16;

/// Within `region`, load a `Nat32` value from `offset`.
/// Traps on an out-of-bounds access.
///
/// Example:
/// ```motoko no-repl
/// let region = Region.new();
/// let offset = 0;
/// let value = 123;
/// Region.storeNat32(region, offset, value);
/// Region.loadNat32(region, offset) // => 123
/// ```
public let loadNat32 : (region : Region, offset : Nat64) -> Nat32 = Prim.regionLoadNat32;

/// Within `region`, store a `Nat32` value at `offset`.
/// Traps on an out-of-bounds access.
///
/// Example:
/// ```motoko no-repl
/// let region = Region.new();
/// let offset = 0;
/// let value = 123;
/// Region.storeNat32(region, offset, value);
/// Region.loadNat32(region, offset) // => 123
/// ```
public let storeNat32 : (region : Region, offset : Nat64, value : Nat32) -> () = Prim.regionStoreNat32;

/// Within `region`, load a `Nat64` value from `offset`.
/// Traps on an out-of-bounds access.
///
/// Example:
/// ```motoko no-repl
/// let region = Region.new();
/// let offset = 0;
/// let value = 123;
/// Region.storeNat64(region, offset, value);
/// Region.loadNat64(region, offset) // => 123
/// ```
public let loadNat64 : (region : Region, offset : Nat64) -> Nat64 = Prim.regionLoadNat64;

/// Within `region`, store a `Nat64` value at `offset`.
/// Traps on an out-of-bounds access.
///
/// Example:
/// ```motoko no-repl
/// let region = Region.new();
/// let offset = 0;
/// let value = 123;
/// Region.storeNat64(region, offset, value);
/// Region.loadNat64(region, offset) // => 123
/// ```
public let storeNat64 : (region : Region, offset : Nat64, value : Nat64) -> () = Prim.regionStoreNat64;

/// Within `region`, load a `Int8` value from `offset`.
/// Traps on an out-of-bounds access.
///
/// Example:
/// ```motoko no-repl
/// let region = Region.new();
/// let offset = 0;
/// let value = 123;
/// Region.storeInt8(region, offset, value);
/// Region.loadInt8(region, offset) // => 123
/// ```
public let loadInt8 : (region : Region, offset : Nat64) -> Int8 = Prim.regionLoadInt8;

/// Within `region`, store a `Int8` value at `offset`.
/// Traps on an out-of-bounds access.
///
/// Example:
/// ```motoko no-repl
/// let region = Region.new();
/// let offset = 0;
/// let value = 123;
/// Region.storeInt8(region, offset, value);
/// Region.loadInt8(region, offset) // => 123
/// ```
public let storeInt8 : (region : Region, offset : Nat64, value : Int8) -> () = Prim.regionStoreInt8;

/// Within `region`, load a `Int16` value from `offset`.
/// Traps on an out-of-bounds access.
///
/// Example:
/// ```motoko no-repl
/// let region = Region.new();
/// let offset = 0;
/// let value = 123;
/// Region.storeInt16(region, offset, value);
/// Region.loadInt16(region, offset) // => 123
/// ```
public let loadInt16 : (region : Region, offset : Nat64) -> Int16 = Prim.regionLoadInt16;

/// Within `region`, store a `Int16` value at `offset`.
/// Traps on an out-of-bounds access.
///
/// Example:
/// ```motoko no-repl
/// let region = Region.new();
/// let offset = 0;
/// let value = 123;
/// Region.storeInt16(region, offset, value);
/// Region.loadInt16(region, offset) // => 123
/// ```
public let storeInt16 : (region : Region, offset : Nat64, value : Int16) -> () = Prim.regionStoreInt16;

/// Within `region`, load a `Int32` value from `offset`.
/// Traps on an out-of-bounds access.
///
/// Example:
/// ```motoko no-repl
/// let region = Region.new();
/// let offset = 0;
/// let value = 123;
/// Region.storeInt32(region, offset, value);
/// Region.loadInt32(region, offset) // => 123
/// ```
public let loadInt32 : (region : Region, offset : Nat64) -> Int32 = Prim.regionLoadInt32;

/// Within `region`, store a `Int32` value at `offset`.
/// Traps on an out-of-bounds access.
///
/// Example:
/// ```motoko no-repl
/// let region = Region.new();
/// let offset = 0;
/// let value = 123;
/// Region.storeInt32(region, offset, value);
/// Region.loadInt32(region, offset) // => 123
/// ```
public let storeInt32 : (region : Region, offset : Nat64, value : Int32) -> () = Prim.regionStoreInt32;

/// Within `region`, load a `Int64` value from `offset`.
/// Traps on an out-of-bounds access.
///
/// Example:
/// ```motoko no-repl
/// let region = Region.new();
/// let offset = 0;
/// let value = 123;
/// Region.storeInt64(region, offset, value);
/// Region.loadInt64(region, offset) // => 123
/// ```
public let loadInt64 : (region : Region, offset : Nat64) -> Int64 = Prim.regionLoadInt64;

/// Within `region`, store a `Int64` value at `offset`.
/// Traps on an out-of-bounds access.
///
/// Example:
/// ```motoko no-repl
/// let region = Region.new();
/// let offset = 0;
/// let value = 123;
/// Region.storeInt64(region, offset, value);
/// Region.loadInt64(region, offset) // => 123
/// ```
public let storeInt64 : (region : Region, offset : Nat64, value : Int64) -> () = Prim.regionStoreInt64;


/// Within `region`, loads a `Float` value from the given `offset`.
/// Traps on an out-of-bounds access.
///
/// Example:
/// ```motoko no-repl
/// let region = Region.new();
/// let offset = 0;
/// let value = 1.25;
/// Region.storeFloat(region, offset, value);
/// Region.loadFloat(region, offset) // => 1.25
/// ```
public let loadFloat : (region : Region, offset : Nat64) -> Float = Prim.regionLoadFloat;

/// Within `region`, store float `value` at the given `offset`.
/// Traps on an out-of-bounds access.
///
/// Example:
/// ```motoko no-repl
/// let region = Region.new();
/// let offset = 0;
/// let value = 1.25;
/// Region.storeFloat(region, offset, value);
/// Region.loadFloat(region, offset) // => 1.25
/// ```
public let storeFloat : (region: Region, offset : Nat64, value : Float) -> () = Prim.regionStoreFloat;

/// Within `region,` load `size` bytes starting from `offset` as a `Blob`.
/// Traps on an out-of-bounds access.
///
/// Example:
/// ```motoko no-repl
/// import Blob "mo:base/Blob";
///
/// let region = Region.new();
/// let offset = 0;
/// let value = Blob.fromArray([1, 2, 3]);
/// let size = value.size();
/// Region.storeBlob(region, offset, value);
/// Blob.toArray(Region.loadBlob(region, offset, size)) // => [1, 2, 3]
/// ```
public let loadBlob : (region : Region, offset : Nat64, size : Nat) -> Blob = Prim.regionLoadBlob;

/// Within `region, write `blob.size()` bytes of `blob` beginning at `offset`.
/// Traps on an out-of-bounds access.
///
/// Example:
/// ```motoko no-repl
/// import Blob "mo:base/Blob";
///
/// let region = Region.new();
/// let offset = 0;
/// let value = Blob.fromArray([1, 2, 3]);
/// let size = value.size();
/// Region.storeBlob(region, offset, value);
/// Blob.toArray(Region.loadBlob(region, offset, size)) // => [1, 2, 3]
/// ```
public let storeBlob : (region : Region, offset : Nat64, value : Blob) -> () = Prim.regionStoreBlob;

}