IMPORTANT NOTE TO THE READER: This document aims to be the "get me going document", so it is unlikely here will still be a lot of things and tools you do not know after finishing it. However, it is intended for you to go through from start to finish in order to lay a foundation with regard to everything you need to know. PLEASE update it with additional resources you find helpful along the way.
TODO(@Olshansk, @bryanchriswhite): Make sure to update this document after #730 and #874 are merged in.
- LFG - Development
- Code Organization
- Maintaining Documentation
- Code Review Guidelines
- Documentation Resources and Implementation
- Install Docker
- Install Docker Compose
- Install Golang
protoc-gen-go,protoc-go-inject-tagandmockgenby runningmake install_cli_deps
Note to the reader: Please update this list if you found anything missing.
Last tested by with:
docker --version
# Docker version 20.10.14, build a224086
protoc --version
# libprotoc 3.19.4
which protoc-go-inject-tag && echo "protoc-go-inject-tag Installed"
# /your$HOME/go/bin/protoc-go-inject-tag
# protoc-go-inject-tag Installed
go version
# go version go1.20.5 darwin/arm64
mockgen --version
# v1.6.0
system_profiler SPSoftwareDataType
# Software:
#
# System Software Overview:
#
# System Version: macOS 12.3.1 (21E258)
# Kernel Version: Darwin 21.4.0
Generate local files
git clone [email protected]:pokt-network/pocket.git && cd pocket
make develop_startOptionally activate changelog pre-commit hook
cp .githooks/pre-commit .git/hooks/pre-commit
chmod +x .git/hooks/pre-commitNOTE: The pre-commit changelog verification has been disabled during the developement of V1 as of 2023-05-16 to unblock development velocity; see more details here. This check is no longer done in the CI and is not recommended for local development either currently.
The Pocket node provides a CLI for interacting with Pocket RPC Server. The CLI can be used for both read & write operations by users and to aid in automation.
In order to build the CLI:
- Generate local files
make develop_start- Build the CLI binary
make buildThe cli binary will be available at bin/p1 and can be used instead of go run app/client/*.go
The commands available are listed here or accessible via bin/p1 --help
2.1 [OPTIONAL] Add the binary to your .rc
IMPORTANT: Note that this helper ONLY works with the docker-compose LocalNet setup and DOES NOT work with the k8s setup
TODO: This section can be deleted once the CLI accepts a --remoteURL option.
You can add the following function so you can run the p1 from anywhere on your host.
function p1 {
export POCKET_WORKDIR="${HOME}/workspace/pocket/pocket/"
export CONFIG_PATH="${POCKET_WORKDIR}/build/config/config.validator1.json"
export GENESIS_PATH="${POCKET_WORKDIR}/build/config/genesis_localhost.json"
${POCKET_WORKDIR}/bin/p1 "$@"
}Execute the following command from the root of the project to add it to your ~/.bash_aliases or similar.
cat >> ~/.bash_aliases << FUNC
function p1 {
export POCKET_WORKDIR="$(pwd)"
export CONFIG_PATH="${POCKET_WORKDIR}/build/config/config.validator1.json"
export GENESIS_PATH="${POCKET_WORKDIR}/build/config/genesis_localhost.json"
${POCKET_WORKDIR}/bin/p1 "\$@"
}
FUNCYou can via a demo of it here.
Swagger UI is available to help during the development process.
In order to spin a local instance of it with the API definition for the Pocket Network Node RPC interface automatically pre-loaded you can run:
make swagger-uimakemake test_allNote that there are a few tests in the library that are prone to race conditions and we are working on improving them. This can be checked with make test_race.
At the time of writing, we have two basic approaches to running a LocalNet. We suggest getting started with the Docker Compose (aka lightweight LocalNet) approach outlined below before moving to the advanced Kubernetes (aka LocalNet) configuration.
The full documentation on running a LocalNet on kubernetes can be found here.
- Delete any previous docker state
make docker_wipe- In one shell, run the 4 nodes setup:
make lightweight_localnet- In another shell, run the development client:
make lightweight_localnet_client && make lightweight_localnet_client_debug- Check the state of each node:
✔ PrintNodeState- Trigger the next view to ensure everything is working:
✔ TriggerNextView- Reset the ResetToGenesis if you want to:
✔ ResetToGenesis- Set the client to automatic and watch it go:
✔ TogglePacemakerMode
✔ TriggerNextView- [Optional] Common manual set of verification steps
✔ ResetToGenesis
✔ PrintNodeState # Check committed height is 0
✔ TriggerNextView
✔ PrintNodeState # Check committed height is 1
✔ TriggerNextView
✔ PrintNodeState # Check committed height is 2
✔ TogglePacemakerMode # Check that it’s automatic now
✔ TriggerNextView # Let it rip!- Send a transaction
make send_local_tx- Trigger the next view
✔ TriggerNextView # Commit the transactiontl;dr Attach the push-image label to the PR to build and push an image from your branch
If you need to test or run non-merged pocket code outside LocalNet, for example on DevNet, TestNet or somewhere else, you can force the CI to build and push a container image. This can be done by attaching a push-image label on your Pull Request. Next CI build will push the container image to our container registry.
A lot of features have been added since this doc was first added. See docs/demo. We should update it to:
- Show k8s LocalNet
- Add more details related to transactions
- Add details related to the keybase
- Add state sync tooling
- Add P2P tooling
If you need to profile the node for CPU and/or memory usage, you can use the pprof tool.
A quick guide is available here.
Pocket
├── app # Entrypoint to running the Pocket node and clients
│ ├── client # Entrypoint to running a local Pocket debug client
│ └── pocket # Entrypoint to running a local Pocket node
├── bin # Destination for compiled pocket binaries
├── build # Build related source files including Docker, scripts, etc
│ ├── config # Configuration files for to run nodes in development
│ ├── deployments # Docker-compose to run different cluster of services for development
│ ├── docs # Links to V1 Protocol implementation documentation (excluding the protocol specification)
├── consensus # Implementation of the Consensus module
├── docs # Links to V1 Protocol implementation documentation (excluding the protocol specification)
├── internal # Internal package following the convention established in Go 1.4: https://go.dev/doc/go1.4#internalpackages
│ └── testutil # Internal package for reusable and/or common test code
├── logger # Implementation of the Logger module
├── p2p # Implementation of the P2P module
├── persistence # Implementation of the Persistence module
├── rpc # Implementation of the RPC module
├── runtime # Implementation of the Runtime module
│ ├── configs # Configuration struct definitions
│ │ └── proto # Protobuf representing the specific configuration of the various modules
│ ├── defaults # Default values for the configuration structs
│ ├── genesis
│ │ └── proto # Protobuf representing the genesis state of the Pocket blockchain
│ └── test_artifacts # Componentry used for generating test artifacts such as particular genesis states used in testing
├── shared # Shared types, modules and utils
│ ├── codec
│ │ └── proto
│ ├── converters
│ ├── core # Core types (Actor, Pools, etc.) used throughout the codebase
│ │ └── types
│ │ └── proto # Protobuf representing the core types used throughout the codebase
│ ├── crypto
│ ├── docs
│ │ └── flows
│ ├── messaging # Messaging structs and functions
│ │ └── proto
│ └── modules # Shared modules definitions (interfaces)
├── telemetry # Implementation of the Telemetry module
├── utility # Implementation of the Utility module
└── Makefile # The source of targets used to develop, build and testDocumentation files currently found by the following command find . -name "*.md" | grep -v -e "vendor" -e "app" are added to the Github Wiki Repository. The Wiki will be improved overtime but in its current form, provides an organized overview of the Repository.
To keep the Wiki organized, a comment is added at the end of each .md file. For example, you can find the following one at the end of this file <!-- GITHUB_WIKI: guides/development/readme -->. The structure of the comment indicates the category (guides), subcategory(ies) (development) and filename (readme): <!-- GITHUB_WIKI: <category>/<subcategory 1>/.../<filename>. You can see the example output in the Wiki Page.
If you are adding a new .md file for documentation please included a similar comment. Use your best judgment for the category and subcategory if its a new directory. Otherwise, copy the comment from a similar file in the directory and choose a relevant filename.
If you plan to be a regular contributor, you might benefit from having your own dashboard similar to this to help track your work. Reach out to one of the maintainers to help you set it up.
We have various labels on GitHub for filtering and tagging different tickets, but some of them trigger special logic.
cl validate- shares the comments related to changelog validation as a PR comment; merging is still blocked by defaultgpt review- adds a code review by GPT to your PRdo not merge- blocks merging the PR even if it is approved and CI is greencommunity- is automatically added to Pocket's dashboard on app.dework.xyz so it becomes a bounty for a community contributorpush-image- builds and pushes an image at the HEAD of the PRe2e-devnet-test- runs E2E tests on your PR- Not a label but close: Add the
/reviewpad summarizecomment to automatically get an LLM to summarize your PR
The Action is triggered when there is a change to any Markdown file on the main branch of the Repository. When triggered, environment variables are set for a Python script that updates the Github Wiki Repository based on Pocket Repository files.
The script finds the relevant Markdown files in the repository and organizes them for the Wiki Repository. Currently, the find command is filtered to exclude the ./app and ./vendor directories. Based on the list of .md file paths, it maps the formatting spec from above to some information about the file. Using the map, it creates a Sidebar file format which Github uses as a Table of Contents for the wiki. Also, from the Pocket repo we copy over the files with titles linking to the Sidebar format.
Below, you can see some of the patterns between the Sidebar format, folder of markdowns in the Wiki Repository, and final sidebar/table of contents display.
| Format | Folder | Wiki |
|---|---|---|
 |
 |
 |
We utilize golangci-lint to run the linters. It is a wrapper around a number of linters and is configured to run many at once. The linters are configured to run on every commit and pull request via CI, and all code issues are populated as GitHub annotations to let developers and reviewers easily locate an issue.
Please follow the instructions on the golangci-lint website.
You can run golangci-lint locally against all packages with:
make go_lintIf you need to specify any additional flags, you can run golangci-lint directly as it picks up the configuration from the .golangci.yml file.
golangci-lint has an integration with VSCode. Per documentation, recommended settings are:
"go.lintTool":"golangci-lint",
"go.lintFlags": [
"--fast"
]golangci-lint is a sophisticated tool including both default and custom linters. The configuration file, which can grow quite large, is located at .golangci.yml.
The official documentation includes a list of different linters and their configuration options. Please refer to this page for more information.
We can write custom linters using go-ruleguard. The rules are located in the build/linters directory. The rules are written in the Ruleguard DSL, if you've never worked with ruleguard in the past, it makes sense to go through introduction article and Ruleguard by example tour.
Ruleguard is run via gocritic linter which is a part of golangci-lint, so if you wish to change configuration or debug a particular rule, you can modify the .golangci.yml file.