From fbe9cbf81550b9059b92c2d3748bdc461aad6977 Mon Sep 17 00:00:00 2001 From: walnut-the-cat <122475853+walnut-the-cat@users.noreply.github.com> Date: Mon, 22 Sep 2025 08:39:21 -0700 Subject: [PATCH 1/6] Delete docs/rpc/run-rpc-node-without-nearup.md --- docs/rpc/run-rpc-node-without-nearup.md | 228 ------------------------ 1 file changed, 228 deletions(-) delete mode 100644 docs/rpc/run-rpc-node-without-nearup.md diff --git a/docs/rpc/run-rpc-node-without-nearup.md b/docs/rpc/run-rpc-node-without-nearup.md deleted file mode 100644 index fd28b96..0000000 --- a/docs/rpc/run-rpc-node-without-nearup.md +++ /dev/null @@ -1,228 +0,0 @@ ---- -id: run-rpc-node-without-nearup -title: Run an RPC Node -sidebar_label: Run a Node 🚀 -sidebar_position: 2 -description: How to run an RPC Node without nearup ---- - -The following instructions are applicable across localnet, testnet, and mainnet. - -If you are looking to learn how to compile and run a NEAR RPC node natively for one of the following networks, this guide is for you. - -- [`testnet`](/rpc/run-rpc-node-without-nearup#testnet) -- [`mainnet`](/rpc/run-rpc-node-without-nearup#mainnet) - - -
-Heads up

- -Running a RPC node is very similar to running a [validator node](/validator/running-a-node) as both types of node use the same `nearcore` release. The main difference for running a validator node is requiring `validator_key.json` to be used by the validator node to support its work of validating blocks and chunks on the network. - -
- - -## Prerequisites - -- [Rust](https://www.rust-lang.org/). If not already installed, please [follow these instructions](https://doc.rust-lang.org/book/ch01-01-installation.html). -- [Git](https://git-scm.com/) -- Installed developer tools: - - MacOS - ```bash - $ brew install cmake protobuf clang llvm awscli - ``` - - Linux - ```bash - $ apt update - $ apt install -y git binutils-dev libcurl4-openssl-dev zlib1g-dev libdw-dev libiberty-dev cmake gcc g++ python docker.io protobuf-compiler libssl-dev pkg-config clang llvm cargo awscli - ``` ---- - -### Choosing your `nearcore` version - -When building your NEAR node you will have two branch options to choose from depending on your desired use: - -- `master` : _(**Experimental**)_ - - Use this if you want to play around with the latest code and experiment. This branch is not guaranteed to be in a fully working state and there is absolutely no guarantee it will be compatible with the current state of *mainnet* or *testnet*. -- [`Latest stable release`](https://github.com/near/nearcore/tags) : _(**Stable**)_ - - Use this if you want to run a NEAR node for *mainnet*. For *mainnet*, please use the latest stable release. This version is used by mainnet validators and other nodes and is fully compatible with the current state of *mainnet*. -- [`Latest release candidates`](https://github.com/near/nearcore/tags) : _(**Release Candidates**)_ - - Use this if you want to run a NEAR node for *tesnet*. For *testnet*, we first release a RC version and then later make that release stable. For testnet, please run the latest RC version. - - -## `testnet` - -### 1. Clone `nearcore` project from GitHub {#1-clone-nearcore-project-from-github} - -First, clone the [`nearcore` repository](https://github.com/near/nearcore). - -```bash -$ git clone https://github.com/near/nearcore -$ cd nearcore -$ git fetch origin --tags -``` - -Checkout to the branch you need if not `master` (default). Latest release is recommended. Please check the [releases page on GitHub](https://github.com/near/nearcore/releases). - -```bash -$ git checkout tags/1.25.0 -b mynode -``` - -### 2. Compile `nearcore` binary {#2-compile-nearcore-binary} - -In the `nearcore` folder run the following commands: - -```bash -$ make release -``` - -This will start the compilation process. It will take some time -depending on your machine power (e.g. i9 8-core CPU, 32 GB RAM, SSD -takes approximately 25 minutes). Note that compilation will need over -1 GB of memory per virtual core the machine has. If the build fails -with processes being killed, you might want to try reducing number of -parallel jobs, for example: `CARGO_BUILD_JOBS=8 make release`. - -If you’re familiar with Cargo, you could also run `cargo build -p neard ---release` instead, which might or might not be equivalent to `make release`. It -is equivalent at the time of writing, but we don't guarantee this. If in doubt, -consult the `Makefile`, or just stick with `make release`. - -The binary path is `target/release/neard` - -### 3. Initialize working directory {#3-initialize-working-directory} - -The NEAR node requires a working directory with a couple of configuration files. Generate the initial required working directory by running: - -```bash -$ ./target/release/neard --home ~/.near init --chain-id testnet --download-genesis --download-config rpc -``` - -> You can specify trusted boot nodes that you'd like to use by pass in a flag during init: `--boot-nodes ed25519:4k9csx6zMiXy4waUvRMPTkEtAS2RFKLVScocR5HwN53P@34.73.25.182:24567,ed25519:4keFArc3M4SE1debUQWi3F1jiuFZSWThgVuA2Ja2p3Jv@34.94.158.10:24567,ed25519:D2t1KTLJuwKDhbcD9tMXcXaydMNykA99Cedz7SkJkdj2@35.234.138.23:24567,ed25519:CAzhtaUPrxCuwJoFzceebiThD9wBofzqqEMCiupZ4M3E@34.94.177.51:24567` - -> You can skip the `--home` argument if you are fine with the default working directory in `~/.near`. If not, pass your preferred location. - -This command will create the required directory structure and will generate `config.json`, `node_key.json`, and `genesis.json` for `testnet` network. -- `config.json` - Configuration parameters which are responsive for how the node will work. This file should contain the following fields critical for RPC nodes: - - `"tracked_shards": [0]` - to track all shards. -- `genesis.json` - A file with all the data the network started with at genesis. This contains initial accounts, contracts, access keys, and other records which represents the initial state of the blockchain. -- `node_key.json` - A file which contains a public and private key for the node. Also includes an optional `account_id` parameter which is required to run a validator node (not covered in this doc). -- `data/` - A folder in which a NEAR node will write it's state. - -> **Heads up** -> The genesis file for `testnet` is big (6GB +) so this command will be running for a while and no progress will be shown. - - -### 4. Get data backup {#4-get-data-backup} - -⚠️ **FREE SNAPSHOT SERVICE BY FASTNEAR WILL BE DEPRECATGED STARTING JUNE 1ST, 2025. We strongly recommend all node operators to use [Epoch Sync](../intro/epoch_sync.md) when possible.** - -The node is ready to be started. However, you must first sync up with the network. This means your node needs to download all the headers and blocks that other nodes in the network already have. - -~~The latest daily snapshots are made available to the public by FastNear, and can be used to set up a validator node or RPC. -For detailed instructions, please refer to [HERE](https://docs.fastnear.com/docs/snapshots).~~ -While snapshot-based syncing was previously the recommended default, we now recommend Epoch Sync—a faster, more lightweight method that allows a node to catch up from genesis without downloading a large state snapshot. - -### 5. Run the node {#5-run-the-node} -To start your node simply run the following command: - -```bash -$ ./target/release/neard --home ~/.near run -``` - -That's all. The node is running you can see log outputs in your console. It will download a bit of missing data since the last backup was performed but it shouldn't take much time. - - -## `mainnet` - -### 1. Clone `nearcore` project from GitHub {#1-clone-nearcore-project-from-github-1} - -First, clone the [`nearcore` repository](https://github.com/near/nearcore). - -```bash -$ git clone https://github.com/near/nearcore -$ cd nearcore -$ git fetch origin --tags -``` - -Next, checkout the release branch you need (recommended) if you will not be using the default `master` branch. Please check the [releases page on GitHub](https://github.com/near/nearcore/releases) for the latest release. - -For more information on choosing between `master` and latest release branch [ [click here](/validator/compile-and-run-a-node#choosing-your-nearcore-version) ]. - -```bash -$ git checkout tags/1.25.0 -b mynode -``` - -### 2. Compile `nearcore` binary {#2-compile-nearcore-binary-1} - -In the `nearcore` folder run the following commands: - -```bash -$ make release -``` - -This will start the compilation process. It will take some time -depending on your machine power (e.g. i9 8-core CPU, 32 GB RAM, SSD -takes approximately 25 minutes). Note that compilation will need over -1 GB of memory per virtual core the machine has. If the build fails -with processes being killed, you might want to try reducing number of -parallel jobs, for example: `CARGO_BUILD_JOBS=8 make release`. - -If you’re familiar with Cargo, you could also run `cargo build -p neard ---release` instead, which might or might not be equivalent to `make release`. It -is equivalent at the time of writing, but we don't guarantee this. If in doubt, -consult the `Makefile`, or just stick with `make release`. - -The binary path is `target/release/neard` - -### 3. Initialize working directory {#3-initialize-working-directory-1} - -The NEAR node requires a working directory with a couple of configuration files. Generate the initial required working directory by running: - -```bash -$ ./target/release/neard --home ~/.near init --chain-id mainnet --download-genesis --download-config rpc -``` - -> You can specify trusted boot nodes that you'd like to use by pass in a flag during init: `--boot-nodes ed25519:86EtEy7epneKyrcJwSWP7zsisTkfDRH5CFVszt4qiQYw@35.195.32.249:24567,ed25519:BFB78VTDBBfCY4jCP99zWxhXUcFAZqR22oSx2KEr8UM1@35.229.222.235:24567,ed25519:Cw1YyiX9cybvz3yZcbYdG7oDV6D7Eihdfc8eM1e1KKoh@35.195.27.104:24567,ed25519:33g3PZRdDvzdRpRpFRZLyscJdbMxUA3j3Rf2ktSYwwF8@34.94.132.112:24567,ed25519:CDQFcD9bHUWdc31rDfRi4ZrJczxg8derCzybcac142tK@35.196.209.192:24567` - -> You can skip the `--home` argument if you are fine with the default working directory in `~/.near`. If not, pass your preferred location. - -This command will create the required directory structure by generating a `config.json`, `node_key.json`, and downloads a `genesis.json` for `mainnet`. -- `config.json` - Configuration parameters which are responsive for how the node will work. This file should contain the following fields critical for RPC nodes: - - `"tracked_shards": [0]` - to track all shards. -- `genesis.json` - A file with all the data the network started with at genesis. This contains initial accounts, contracts, access keys, and other records which represents the initial state of the blockchain. -- `node_key.json` - A file which contains a public and private key for the node. Also includes an optional `account_id` parameter which is required to run a validator node (not covered in this doc). -- `data/` - A folder in which a NEAR node will write it's state. - -### 4. Get data backup {#4-get-data-backup-1} - -⚠️ **FREE SNAPSHOT SERVICE BY FASTNEAR WILL BE DEPRECATGED STARTING JUNE 1ST, 2025. We strongly recommend all node operators to use [Epoch Sync](../intro/epoch_sync.md) when possible.** - -The node is ready to be started. However, you must first sync up with the network. This means your node needs to download all the headers and blocks that other nodes in the network already have. - -~~The latest daily snapshots are made available to the public by FastNear, and can be used to set up a validator node or RPC. -For detailed instructions, please refer to [HERE](https://docs.fastnear.com/docs/snapshots).~~ -While snapshot-based syncing was previously the recommended default, we now recommend Epoch Sync—a faster, more lightweight method that allows a node to catch up from genesis without downloading a large state snapshot. - -### 5. Run the node {#5-run-the-node} -To start your node simply run the following command: - -```bash -$ ./target/release/neard --home ~/.near run -``` - -That's all. The node is running and you can see log outputs in your console. It will download a bit of missing data since the last backup was performed but it shouldn't take much time. - - - -## Running a node in `light` mode -Running a node in `light` mode allows the operator to access chain level data, not state level data. You can also use the `light` node to submit transactions or verify certain proofs. To run a node in a `light` mode that doesn't track any shards, the only change required is to update the `config.json` whereby `tracked_shards` is set to an empty array. - -``` -"tracked_shards": [] -``` - - ->Got a question? - - Ask it on StackOverflow! From 70e79c1bb531c51112b6f1ed96a1de15d5a67d15 Mon Sep 17 00:00:00 2001 From: walnut-the-cat <122475853+walnut-the-cat@users.noreply.github.com> Date: Mon, 22 Sep 2025 08:40:08 -0700 Subject: [PATCH 2/6] Update state-sync.md --- docs/rpc/state-sync.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/rpc/state-sync.md b/docs/rpc/state-sync.md index ecbbb46..ac6916c 100644 --- a/docs/rpc/state-sync.md +++ b/docs/rpc/state-sync.md @@ -2,7 +2,7 @@ id: state-sync title: State Sync sidebar_label: State Sync Configuration -sidebar_position: 4 +sidebar_position: 3 description: State Sync Configuration --- From 84fa824919adb0fd6ed4c2db3d002085f5000d22 Mon Sep 17 00:00:00 2001 From: walnut-the-cat <122475853+walnut-the-cat@users.noreply.github.com> Date: Mon, 22 Sep 2025 08:40:46 -0700 Subject: [PATCH 3/6] Delete docs/archival/run-archival-node-with-nearup.md --- .../archival/run-archival-node-with-nearup.md | 78 ------------------- 1 file changed, 78 deletions(-) delete mode 100644 docs/archival/run-archival-node-with-nearup.md diff --git a/docs/archival/run-archival-node-with-nearup.md b/docs/archival/run-archival-node-with-nearup.md deleted file mode 100644 index f736430..0000000 --- a/docs/archival/run-archival-node-with-nearup.md +++ /dev/null @@ -1,78 +0,0 @@ ---- -id: run-archival-node-with-nearup -title: Run an Archival Node (with nearup) -sidebar_label: Run a Node (with nearup) -sidebar_position: 3 -description: How to run an Archival Node with nearup ---- - -## This page is DEPRECATED Please Use [Run a Split Storage Archival](/archival/split-storage-archival) - -*We encourage you to set up your node with Neard instead of Nearup as Nearup is not used on Mainnet. Please head to [Run a Node](/archival/run-archival-node-without-nearup) for instructions on how to setup an archival node with Neard.* - -
-Heads up

- -Running an archival node is very similar to running a [validator node](/validator/running-a-node) as both types of node use the same `nearcore` release. The main difference for running an archival node is a modification to the `config.json` by changing `archive` to `true`. See below for more details. - -
- - -## Prerequisites {#prerequisites} - -- [Git](https://git-scm.com/) -- [Nearup](https://github.com/near-guildnet/nearup): Make sure [`nearup`](https://github.com/near-guildnet/nearup) is installed. You can install `nearup` by following the instructions at https://github.com/near-guildnet/nearup. - ---- - -### Steps to Run an Archival Node using `nearup` {#steps-to-run-an-archival-node-using-nearup} - -First, retrieve a copy of the latest archival snapshot from S3: -```bash -$ aws s3 --no-sign-request cp s3://near-protocol-public/backups/testnet/archive/latest . -$ LATEST=$(cat latest) -$ aws s3 --no-sign-request cp --no-sign-request --recursive s3://near-protocol-public/backups/testnet/archive/$LATEST ~/.near/data -``` - - -### Configuration Update {#configuration-update} - -Running an archival node is the same as a [validator node](/validator/running-a-node) as both types of node use the same `nearcore` release. The main difference for running an archival node is a modification to the `config.json` by changing `archive` to `true`. - -The `config.json` should contain the following fields. Currently, NEAR testnet and mainnet have only 1 (indexed [0]) shard and that shard is tracked. In the future, there will be the possibility to track different or multiple shards. - -``` -{ - ... - "archive": true, - "tracked_shards": [0], - ... -} -``` - -Please make sure that the node is stopped while changing the `config.json`. - -Once the config has been changed, you can restart the node and the node will start syncing new archival data. In the case where you want the full archival history, you can delete the data dir and start the node from scratch syncing full history or use one of the latest backups containing the data directory snapshot which can be copied under the near home dir (default: ~/.near/data). - - - -Then run: -```bash - $ nearup run testnet -``` -Wait until initialization finishes, use the following command to follow logs: -```bash - $ nearup logs --follow -``` -Then run: -```bash - $ nearup stop -``` -Finally, run the following command and the node should start syncing headers at ~97%: -```bash - $ nearup run testnet -``` - ->Got a question? - - Ask it on StackOverflow! From d97c4b98910b0475d3d5ecb38b0e7364f67f6154 Mon Sep 17 00:00:00 2001 From: walnut-the-cat <122475853+walnut-the-cat@users.noreply.github.com> Date: Mon, 22 Sep 2025 08:49:00 -0700 Subject: [PATCH 4/6] Update split-storage-archival.md --- docs/archival/split-storage-archival.md | 58 +++---------------------- 1 file changed, 5 insertions(+), 53 deletions(-) diff --git a/docs/archival/split-storage-archival.md b/docs/archival/split-storage-archival.md index cfc3087..46ac3f3 100644 --- a/docs/archival/split-storage-archival.md +++ b/docs/archival/split-storage-archival.md @@ -19,14 +19,14 @@ With the 1.35.0 release we are rolling out split storage, a new option for stora In split storage mode, the Near Client only works with a smaller hot database to produce blocks, which results in increased performance. Cold database is not accessed for reads during normal block production and is only read when historical data is specifically requested. Thus, we recommend keeping the cold database on cheaper and slower drives such as an HDD and only optimize speed for the hot database, which is about 10 times smaller. Split storage is disabled by default, and can be enabled with a config change and going through a migration process that requires manual steps. We have several choices for the migration: -* Use Pagoda-provided S3 snapshots of nodes that have split-storge configured. +* Use snapshots (e.g. FASTNEAR snapshot) of nodes that have split-storge configured. * Do a manual migration using S3 snapshots of the existing RPC single database * Do a manual migration using your own buddy RPC node | Migration option | Pros | Cons | | --------------------------------------------------- | ------------------------------- | ------------------------------------------------------ | -| Pagoda-provided S3 snapshots of split-storage nodes | Fastest. Little to no downtime. | Requires trust in migration performed by Pagoda nodes | -| Manual migration + S3 RPC snapshots | No need for extra node. Cold storage is initialized in a trustless way. | Requires trust in Pagoda RPC snapshots. The node will be out of sync at the end of the migration and will need to block sync several epochs. Migration takes days and you cannot restart your node during that time. | +| Snapshots of split-storage nodes | Fastest. Little to no downtime. | Requires trust in Snapshot provider | +| Manual migration + S3 RPC snapshots | No need for extra node. Cold storage is initialized in a trustless way. | Requires trust in RPC snapshot provider. The node will be out of sync at the end of the migration and will need to block sync several epochs. Migration takes days and you cannot restart your node during that time. | | Manual migration + your own node | Trustless. Little to no downtime | Requires an extra RPC node with bigger storage. Migration takes days and you cannot restart your node during that time. | ## Important config fields {#config} @@ -66,57 +66,9 @@ Example: } ``` -## Using Pagoda-provided S3/CloudFrontsplit-storage snapshots {#S3 migration} +## Using Snaphots {#S3 migration} -Prerequisite: - -Recommended download client [`rclone`](https://rclone.org). -This tool is present in many Linux distributions. There is also a version for Windows. -And its main merit is multithread. -You can [read about it on](https://rclone.org) -** rclone version needs to be v1.66.0 or higher - -First, install rclone: -``` -$ sudo -v ; curl https://rclone.org/install.sh | sudo bash -``` -Next, prepare config, so you don't need to specify all the parameters interactively: -``` -mkdir -p ~/.config/rclone -touch ~/.config/rclone/rclone.conf -``` - -, and paste exactly the following config into `rclone.conf`: -``` -[near_cf] -type = s3 -provider = AWS -download_url = https://dcf58hz8pnro2.cloudfront.net/ -acl = public-read -server_side_encryption = AES256 -region = ca-central-1 - -``` - -1. Find latest snapshot -```bash -chain=testnet/mainnet -rclone copy --no-check-certificate near_cf://near-protocol-public/backups/${chain:?}/archive/latest_split_storage ./ -latest=$(cat latest_split_storage) -latest=$(cat latest_split_storage) -``` -2. Download cold and hot databases -```bash -NEAR_HOME=/home/ubuntu/.near -rclone copy --no-check-certificate --progress --transfers=6 --checkers=6 \ - near_cf://near-protocol-public/backups/${chain:?}/archive/${latest:?} $NEAR_HOME -``` -This will download cold database to `$NEAR_HOME/cold-data`, and hot database to `$NEAR_HOME/hot-data` -3. Make changes to your config.json -```bash -cat <<< $(jq '.save_trie_changes = true | .cold_store = .store | .cold_store.path = "cold-data" | .store.path = "hot-data" | .split_storage.enable_split_storage_view_client = true' $NEAR_HOME/config.json) > $NEAR_HOME/config.json -``` -4. Restart the node +FASTNEAR is now provider of Archival node snapshot. Please refer to their documentation [HERE](https://docs.fastnear.com/docs/snapshots/mainnet#archival-mainnet-snapshot) ### If your node is out of sync after restarting from the latest snapshot {#syncing node} Downloading the cold database can take a long time (days). In this time your database can become very far behind the chain. From 1bf38a479bfb8809d3c6888f0caeb10b0efe91a2 Mon Sep 17 00:00:00 2001 From: walnut-the-cat <122475853+walnut-the-cat@users.noreply.github.com> Date: Mon, 22 Sep 2025 08:49:18 -0700 Subject: [PATCH 5/6] Delete docs/archival/run-archival-node-without-nearup.md --- .../run-archival-node-without-nearup.md | 233 ------------------ 1 file changed, 233 deletions(-) delete mode 100644 docs/archival/run-archival-node-without-nearup.md diff --git a/docs/archival/run-archival-node-without-nearup.md b/docs/archival/run-archival-node-without-nearup.md deleted file mode 100644 index 9a09c72..0000000 --- a/docs/archival/run-archival-node-without-nearup.md +++ /dev/null @@ -1,233 +0,0 @@ ---- -id: run-archival-node-without-nearup -title: Run an Archival Node -sidebar_label: Run a Node 🚀 -sidebar_position: 2 -description: How to run an Archival Node without nearup ---- -## This page is DEPRECATED in favor of split storage archival. Please Use [Run a Split Storage Archival](/archival/split-storage-archival) - -The following instructions are applicable across localnet, testnet, and mainnet. - -If you are looking to learn how to compile and run a NEAR Archival node natively for one of the following networks, this guide is for you. - -- [`testnet`](/archival/run-archival-node-without-nearup#testnet) -- [`mainnet`](/archival/run-archival-node-without-nearup#mainnet) - - -
-Heads up

- -Running an archival node is very similar to running a [validator node](/validator/running-a-node) as both types of node use the same `nearcore` release. The main difference for running an archival node is a modification to the `config.json` by changing `archive` to `true`. See below for more details. - -
- - -## Prerequisites {#prerequisites} - -- [Rust](https://www.rust-lang.org/) -- [Git](https://git-scm.com/) -- Installed developer tools: - - MacOS - ```bash - $ brew install cmake protobuf clang llvm awscli - ``` - - Linux - ```bash - $ apt update - $ apt install -y git binutils-dev libcurl4-openssl-dev zlib1g-dev libdw-dev libiberty-dev cmake gcc g++ python docker.io protobuf-compiler libssl-dev pkg-config clang llvm cargo awscli - ``` ---- - -### Choosing your `nearcore` version {#choosing-your-nearcore-version} - -When building your NEAR node you will have two branch options to choose from depending on your desired use: - -- `master` : _(**Experimental**)_ - - Use this if you want to play around with the latest code and experiment. This branch is not guaranteed to be in a fully working state and there is absolutely no guarantee it will be compatible with the current state of *mainnet* or *testnet*. -- [`Latest stable release`](https://github.com/near/nearcore/tags) : _(**Stable**)_ - - Use this if you want to run a NEAR node for *mainnet*. For *mainnet*, please use the latest stable release. This version is used by mainnet validators and other nodes and is fully compatible with the current state of *mainnet*. -- [`Latest release candidates`](https://github.com/near/nearcore/tags) : _(**Release Candidates**)_ - - Use this if you want to run a NEAR node for *tesnet*. For *testnet*, we first release a RC version and then later make that release stable. For testnet, please run the latest RC version. - - -## `testnet` {#testnet} - -### 1. Clone `nearcore` project from GitHub {#1-clone-nearcore-project-from-github} - -First, clone the [`nearcore` repository](https://github.com/near/nearcore). - -```bash -$ git clone https://github.com/near/nearcore -$ cd nearcore -$ git fetch origin --tags -``` - -Checkout to the branch you need if not `master` (default). Latest release is recommended. Please check the [releases page on GitHub](https://github.com/near/nearcore/releases). - -```bash -$ git checkout tags/1.28.0 -b mynode -``` - -### 2. Compile `nearcore` binary {#2-compile-nearcore-binary} - -In the `nearcore` folder run the following commands: - -```bash -$ make release -``` - -This will start the compilation process. It will take some time -depending on your machine power (e.g. i9 8-core CPU, 32 GB RAM, SSD -takes approximately 25 minutes). Note that compilation will need over -1 GB of memory per virtual core the machine has. If the build fails -with processes being killed, you might want to try reducing number of -parallel jobs, for example: `CARGO_BUILD_JOBS=8 make release`. - -By the way, if you’re familiar with Cargo, you could wonder why not -run `cargo build -p neard --release` instead. While this will produce -a binary, the result will be a less optimized version. On technical -level, this is because building via `make neard` enables link-time -optimisation which is disabled by default. - -The binary path is `target/release/neard` - -### 3. Initialize working directory {#3-initialize-working-directory} - -The NEAR node requires a working directory with a couple of configuration files. Generate the initial required working directory by running: - -```bash -$ ./target/release/neard --home ~/.near init --chain-id testnet --download-genesis --download-config archival -``` - -> You can specify trusted boot nodes that you'd like to use by pass in a flag during init: `--boot-nodes ed25519:4k9csx6zMiXy4waUvRMPTkEtAS2RFKLVScocR5HwN53P@34.73.25.182:24567,ed25519:4keFArc3M4SE1debUQWi3F1jiuFZSWThgVuA2Ja2p3Jv@34.94.158.10:24567,ed25519:D2t1KTLJuwKDhbcD9tMXcXaydMNykA99Cedz7SkJkdj2@35.234.138.23:24567,ed25519:CAzhtaUPrxCuwJoFzceebiThD9wBofzqqEMCiupZ4M3E@34.94.177.51:24567` - -> You can skip the `--home` argument if you are fine with the default working directory in `~/.near`. If not, pass your preferred location. - -This command will create the required directory structure and will generate `config.json`, `node_key.json`, and `genesis.json` for `testnet` network. -- `config.json` - Configuration parameters which are responsive for how the node will work. This file should contain the following fields critical for archival nodes: - - `"archive": true` - to save the data for all blocks to storage. - - `"tracked_shards": [0]` - to track all shards. -- `genesis.json` - A file with all the data the network started with at genesis. This contains initial accounts, contracts, access keys, and other records which represents the initial state of the blockchain. -- `node_key.json` - A file which contains a public and private key for the node. Also includes an optional `account_id` parameter which is required to run a validator node (not covered in this doc). -- `data/` - A folder in which a NEAR node will write it's state. - -> **Heads up** -> The genesis file for `testnet` is big (8GB +) so this command will be running for a while and no progress will be shown. - -Please make sure that the node is not running while changing the `config.json`. - -Once the config has been changed, you can restart the node and the node will start syncing new archival data. In the case where you want the full archival history, you can delete the data dir and start the node from scratch syncing full history or use one of the latest backups containing the data directory snapshot which can be copied under the near home dir (default: ~/.near/data). - - -### 4. Get data backup {#4-get-data-backup} - -The node is ready to be started. However, you must first sync up with the network. This means your node needs to download all the headers and blocks that other nodes in the network already have. - -```bash -$ aws s3 --no-sign-request cp s3://near-protocol-public/backups/testnet/archive/latest . -$ LATEST=$(cat latest) -$ aws s3 --no-sign-request cp --no-sign-request --recursive s3://near-protocol-public/backups/testnet/archive/$LATEST ~/.near/data -``` - -In the case where you want the full archival history again (for example, if the database gets into an invalid state), you can delete the data dir and start the node from scratch syncing full history or use one of the latest backups containing the data directory snapshot which can be copied under the near home dir (default: ~/.near/data). - -### 5. Run the node {#5-run-the-node} -To start your node simply run the following command: - -```bash -$ ./target/release/neard --home ~/.near run -``` - -That's all. The node is running you can see log outputs in your console. It will download a bit of missing data since the last backup was performed but it shouldn't take much time. - - -## `mainnet` {#mainnet} - -### 1. Clone `nearcore` project from GitHub {#1-clone-nearcore-project-from-github-1} - -First, clone the [`nearcore` repository](https://github.com/near/nearcore). - -```bash -$ git clone https://github.com/near/nearcore -$ cd nearcore -$ git fetch origin --tags -``` - -Next, checkout the release branch you need (recommended) if you will not be using the default `master` branch. Please check the [releases page on GitHub](https://github.com/near/nearcore/releases) for the latest release. - -For more information on choosing between `master` and latest release branch [ [click here](/validator/compile-and-run-a-node#choosing-your-nearcore-version) ]. - -```bash -$ git checkout tags/1.26.1 -b mynode -``` - -### 2. Compile `nearcore` binary {#2-compile-nearcore-binary-1} - -In the `nearcore` folder run the following commands: - -```bash -$ make release -``` - -This will start the compilation process. It will take some time -depending on your machine power (e.g. i9 8-core CPU, 32 GB RAM, SSD -takes approximately 25 minutes). Note that compilation will need over -1 GB of memory per virtual core the machine has. If the build fails -with processes being killed, you might want to try reducing number of -parallel jobs, for example: `CARGO_BUILD_JOBS=8 make release`. - -By the way, if you’re familiar with Cargo, you could wonder why not -run `cargo build -p neard --release` instead. While this will produce -a binary, the result will be a less optimized version. On technical -level, this is because building via `make neard` enables link-time -optimisation which is disabled by default. - -The binary path is `target/release/neard` - -### 3. Initialize working directory {#3-initialize-working-directory-1} - -The NEAR node requires a working directory with a couple of configuration files. Generate the initial required working directory by running: - -```bash -$ ./target/release/neard --home ~/.near init --chain-id mainnet --download-genesis --download-config archival -``` - -> You can specify trusted boot nodes that you'd like to use by pass in a flag during init: `--boot-nodes ed25519:86EtEy7epneKyrcJwSWP7zsisTkfDRH5CFVszt4qiQYw@35.195.32.249:24567,ed25519:BFB78VTDBBfCY4jCP99zWxhXUcFAZqR22oSx2KEr8UM1@35.229.222.235:24567,ed25519:Cw1YyiX9cybvz3yZcbYdG7oDV6D7Eihdfc8eM1e1KKoh@35.195.27.104:24567,ed25519:33g3PZRdDvzdRpRpFRZLyscJdbMxUA3j3Rf2ktSYwwF8@34.94.132.112:24567,ed25519:CDQFcD9bHUWdc31rDfRi4ZrJczxg8derCzybcac142tK@35.196.209.192:24567` - -> You can skip the `--home` argument if you are fine with the default working directory in `~/.near`. If not, pass your preferred location. - -This command will create the required directory structure by generating a `config.json`, `node_key.json`, and downloads a `genesis.json` for `mainnet`. -- `config.json` - Configuration parameters which are responsive for how the node will work. This file should contain the following fields critical for archival nodes: - - `"archive": true` - to save the data for all blocks to storage. - - `"tracked_shards": [0]` - to track all shards. -- `genesis.json` - A file with all the data the network started with at genesis. This contains initial accounts, contracts, access keys, and other records which represents the initial state of the blockchain. -- `node_key.json` - A file which contains a public and private key for the node. Also includes an optional `account_id` parameter which is required to run a validator node (not covered in this doc). -- `data/` - A folder in which a NEAR node will write it's state. - - -### 4. Get data backup {#4-get-data-backup-1} - -The node is ready to be started. However, you must first sync up with the network. This means your node needs to download all the headers and blocks that other nodes in the network already have. - -```bash -$ aws s3 --no-sign-request cp s3://near-protocol-public/backups/mainnet/archive/latest . -$ LATEST=$(cat latest) -$ aws s3 --no-sign-request cp --no-sign-request --recursive s3://near-protocol-public/backups/mainnet/archive/$LATEST ~/.near/data -``` - -In the case where you want the full archival history again (for example, if the database gets into an invalid state), you can delete the data dir and start the node from scratch syncing full history or use one of the latest backups containing the data directory snapshot which can be copied under the near home dir (default: ~/.near/data). - -### 5. Run the node {#5-run-the-node-1} -To start your node simply run the following command: - -```bash -$ ./target/release/neard --home ~/.near run -``` - -That's all. The node is running and you can see log outputs in your console. It will download a bit of missing data since the last backup was performed but it shouldn't take much time. - - ->Got a question? - - Ask it on StackOverflow! From 91b552c128f6a40e3e64cd02aec009a4bb4f1181 Mon Sep 17 00:00:00 2001 From: walnut-the-cat <122475853+walnut-the-cat@users.noreply.github.com> Date: Mon, 22 Sep 2025 08:50:36 -0700 Subject: [PATCH 6/6] Update split-storage-archival.md --- docs/archival/split-storage-archival.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/archival/split-storage-archival.md b/docs/archival/split-storage-archival.md index 46ac3f3..a4184f2 100644 --- a/docs/archival/split-storage-archival.md +++ b/docs/archival/split-storage-archival.md @@ -2,7 +2,7 @@ id: split-storage-archival title: Split Storage for NEAR Archival Nodes sidebar_label: Split Storage -sidebar_position: 4 +sidebar_position: 2 description: Split Storage for NEAR Archival Nodes ---