GITBOOK-195: memiavl and local state sync guide

SUMMARY.md

@@ -50,6 +50,10 @@
   * [Caldera](for-dapp-developers/dev-tools-and-integrations/caldera.md)
   * [Flair](for-dapp-developers/dev-tools-and-integrations/flair.md)
 
+***
+
+* [Page](page.md)
+
 ## FOR NODE HOSTS
 
 * [Running nodes](for-node-hosts/running-nodes/README.md)
@@ -63,6 +67,8 @@
   * [Best Practices](for-node-hosts/running-nodes/cronos-node-best-practises.md)
   * [Cronosd build with Nix](for-node-hosts/running-nodes/cronosd-build-with-nix.md)
   * [VersionDB](for-node-hosts/running-nodes/versiondb.md)
+  * [MemIAVL](for-node-hosts/running-nodes/memiavl.md)
+  * [Local State Sync](for-node-hosts/running-nodes/local-state-sync.md)
 * [Cronosd](for-node-hosts/cli.md)
 
 ## CRONOS PLAY

for-node-hosts/running-nodes/local-state-sync.md

@@ -0,0 +1,89 @@
+# Local State Sync
+
+In [Cronos v1.0.12](https://github.com/crypto-org-chain/cronos/releases/tag/v1.0.12), we introduced a new set of commands to do local state sync, the full command help screen is:
+
+```bash
+$ cronosd snapshots --help
+Manage local snapshots
+
+Usage:
+  cronosd snapshots [command]
+
+Available Commands:
+  delete      Delete a local snapshot
+  dump        Dump the snapshot as portable archive format
+  export      Export app state to snapshot store
+  list        List local snapshots
+  load        Load a snapshot archive file (.tar.gz) into snapshot store
+  restore     Restore app state from local snapshot
+
+Flags:
+  -h, --help   help for snapshots
+
+Global Flags:
+      --home string         directory for config and data (default "/Users/yihuang/.cronos")
+      --log_format string   The logging format (json|plain) (default "plain")
+      --log_level string    The logging level (trace|debug|info|warn|error|fatal|panic) (default "info")
+      --trace               print out full stack trace on errors
+
+Use "cronosd snapshots [command] --help" for more information about a command.
+```
+
+#### Local State Sync
+
+Before we dive into local state sync, let's understand a "normal" state sync first. After you setup state-sync in config and start the node, the node will do the following procedure:
+
+1. Discover snapshots on the p2p network.
+2. Verify the snapshot metadata by verifying the block headers between the snapshot and the trusted block height.
+3. Download snapshot chunks from the p2p network.
+4. Restore app state from the downloaded snapshot chunks.
+5. Bootstrap cometbft state.
+6. Start normal sync.
+
+With a "normal" state sync, the procedure is done automatically on startup, it works great when the state is small, but as the chain grows, the procedure becomes slow and unstable.
+
+The new local state sync commands try to break down this procedure into smaller steps:
+
+1. Discover and download snapshots from out-of-band communications, for example community shared snapshots on a http server.
+2.  User downloads shared snapshots and imports into the local snapshot database.
+
+    ```bash
+    $ cronosd snapshots load /path/to/downloaded-snapshot.tar.gz
+    $ cronosd snapshots list
+    <height> <format>
+    ```
+3.  Restore app state from the snapshot (this step is much faster with [MemIAVL](https://github.com/crypto-org-chain/cronos/wiki/MemIAVL) than current iavl):
+
+    ```bash
+    $ cronosd snapshots restore <height> <format>
+    ```
+
+    3a. (Optional) If you want to enable versiondb together, you need to restore the versiondb from the snapshot as well:
+
+    ```bash
+    $ cronosd changeset restore-versiondb <height> <format>
+    ```
+4.  Verify and bootstrap cometbft state based on current app state and state sync configuration:
+
+    ```bash
+    $ cronosd tendermint bootstrap-state
+    ```
+
+    This step requires the `statesync.*` configurations in `config.toml`, to be the same as you would setup for a "normal" state sync.
+5. Start the node and sync normally.
+
+#### Snapshot Provider
+
+Alternatively, a snapshot provider can dump local snapshots into a portable tarball which can be shared across the network:
+
+```bash
+$ cronosd snapshots dump <height> <format> --output /path/to/snapshot.tar.gz
+```
+
+Normally the snapshots are generated automatically using config `state-sync.snapshot-interval`, with these new commands, one can also manually export arbitrary versions of the state.
+
+```bash
+$ cronosd snapshots export --height <height>
+$ cronosd snapshots list
+<height> <format>
+```

for-node-hosts/running-nodes/memiavl.md

@@ -0,0 +1,77 @@
+# MemIAVL
+
+{% hint style="danger" %}
+WARNING: EXPERIMENTAL
+{% endhint %}
+
+`Memiavl` is a drop-in replacement for the current iavl implementation, offering a huge performance boost ([benchmark](https://github.com/crypto-org-chain/cronos/wiki/MemIAVL-Benchmark)) for the node. MemIAVL can be enabled by turning on the `memiavl.enable` config item in the `app.toml.`It uses a standalone db directory `data/memiavl.db.` You can always disable it again, in that case the `data/application.db` will use the default iavl again, so it's ok to switch between them back and forth.
+
+`Memiavl` only supports **pruned nodes**, the default configuration(`memiavl.snapshot-keep-recent=0`) should be set equivalent to `pruning=everything.` In order to support historical grpc queries, you should enable versiondb together with memiavl, if you need to support archive merkle proof generations, don't use `memiavl!`
+
+
+
+The default `memiavl` section in `app.toml`:
+
+```bash
+[memiavl]
+
+# Enable defines if the memiavl should be enabled.
+enable = false
+
+# ZeroCopy defines if the memiavl should return slices pointing to mmap-ed buffers directly (zero-copy),
+# the zero-copied slices must not be retained beyond current block's execution.
+zero-copy = false
+
+# AsyncCommitBuffer defines the size of asynchronous commit queue, this greatly improve block catching-up
+# performance, -1 means synchronous commit.
+async-commit-buffer = 0
+
+# SnapshotKeepRecent defines what many old snapshots (excluding the latest one) to keep after new snapshots are taken.
+snapshot-keep-recent = 0
+
+# SnapshotInterval defines the block interval the memiavl snapshot is taken, default to 1000.
+snapshot-interval = 1000
+
+# CacheSize defines the size of the cache for each memiavl store, default to 1000.
+cache-size = 1000
+```
+
+### Use Cases
+
+#### Semi-Archived Node
+
+When versiondb was released, we recommend users to setup a pruned iavl tree together with versiondb, to setup a semi-archived node, you can replace the pruned iavl tree with `memiavl` now.
+
+#### State Sync Node
+
+`Memiavl` can do state-sync snapshot restoration much faster than the current iavl, it's actually much faster than the chunk downloading speed, with `memiavl`, allowing state-sync node to be bootstrapped in around 10 minutes depending on the internet speed. If you download a snapshot from CDN and do a local restoration, it can even be faster. Just enable memiavl in `app.toml` before starting a state-sync.
+
+#### Snapshot Providers
+
+`memiavl` can do state-sync snapshot exports much faster as well, on Cronos mainnet, snapshots can be exported in minutes instead of days, so it's recommended to run snapshot provider nodes with `memiavl`, so the snapshots will be much more up-to-date.
+
+### Migrate Semi-Archive Node
+
+To migrate a semi-archive versionDB node to memiavl, you just need to restore a memiavl db to the same current height as the versiondb, then continue normal syncing from there, we can use local state-sync commands to do that:
+
+```bash
+# download a snapshot whose version is smaller than versiondb's latest version
+$ cronosd snapshots load /path/to/downloaded-snapshot.tar.gz
+
+# check the snapshot height and format in list command
+$ cronosd snapshots list
+
+# edit app.toml to enable memiavl
+$ cronosd snapshots restore <snapshot height> 2
+
+# edit app.toml to disable versiondb, and set `memiavl.async-commit-buffer = -1`
+$ cronosd start --halt-height <versiondb height>
+
+# edit app.toml to enable versiondb, and set `memiavl.async-commit-buffer = 0 or small positive number` to re-enable async commit.
+$ cronosd start
+```
+
+### Compression
+
+Right now memiavl doesn't do any generic compressions to the data files, that would kill the simplicity of the current implementation, and probably hurt performance if not done right. Fortunately, on the linux filesystem level compressions work well with `mmap.` We can run memiavl on `btrfs` configured with `zstd` compression, and observe around \~60% compression rate on memiavl directory, with no visible performance regression. So this is the recommended way to run memiavl for efficient disk space usage.
+

page.md

@@ -0,0 +1,2 @@
+# Page
+