This is the accompanying code to the paper "Ladon: High-Performance Multi-BFT Consensus via Dynamic Global Ordering" which was accepted to EuroSys 2025. A technical report is available here.
Multi-BFT consensus runs multiple leader-based consensus instances in parallel, circumventing the leader bottleneck of a single instance. However, it contains an Achilles’ heel: the need to globally order output blocks across instances. Deriving this global ordering is challenging because it must cope with different rates at which blocks are produced by instances. Prior Multi-BFT designs assign each block a global index before creation, leading to poor performance. We propose Ladon, a high-performance Multi-BFT protocol that allows varying instance block rates. Our key idea is to order blocks across instances dynamically, which eliminates blocking on slow instances. We achieve dynamic global ordering by assigning monotonic ranks to blocks. We pipeline rank coordination with the consensus process to reduce protocol overhead and combine aggregate signatures with rank information to reduce message complexity. Besides, the dynamic ordering enables blocks to be globally ordered according to their generation, which respects inter-block causality.
Ladon is a distributed consensus framework built on top of the ISS framework from Hyperledger-labs. For a detailed modular explanation and a glossary of terms, please refer to here. The main task of the service is to maintain a totally ordered Log of client Requests, using multiple consensus protocol instances whose outputs are multiplexed into the final Log. The Manager module orchestrates these instances, determining parameters and creating Segments—logical parts of the Log assigned to specific protocol instances. Each Log entry has a sequence number (SN) and contains a Batch of Requests. Client Requests are partitioned into Buckets based on their hashes, and each Segment is assigned a unique Bucket. The Manager monitors the Log, creating new Segments as needed and triggering the Orderer to commit Entries with the corresponding SNs. Periodically, the Manager initiates Checkpointer to create checkpoints and advances the Segments as the watermark window shifts.
Ladon extends the capabilities of ISS by incorporating the following modifications:
Pipeline the Rank Collection in the Consensus Protocol: Ladon introduces a pipelined step in the consensus protocol for collecting the ranks of participating nodes, which is then utilized for the global ordering of blocks.
Dynamic Global Ordering Algorithm: Ladon replaces the existing global ordering algorithm in ISS with a dynamic ordering algorithm. This new algorithm enhances system performance and efficiency, particularly in the presence of stragglers.
- PBFT (Practical Byzantine Fault Tolerance)
- HotStuff
-
Module responsible for creating checkpoints of the Log. The Checkpointer listens to the Manager, which notifies the Checkpointer about each SN at which a checkpoint should occur. The Checkpointer triggers a separate instance of the checkpointing protocol for each such SN. When a checkpoint is stable, the Checkpointer submits it to the Log.
-
Implements cryptographic primitives, such as encryption and digital signatures.
-
Deployment scripts and configurations for setting up and running Ladon in different environments. For more detailed descriptions, please refer to the deployment folder.
-
log:
The Log is a sequence of Entries. Each Entry has a 32-bit integer sequence number (SN) defining its position in the Log, and contains a Batch of Requests.
-
The Manager observes the Log and creates new Segments as the Log fills up. When the Manager creates a new Segment, it triggers the Orderer that orders the Segment. Ordering a Segment means committing new Entries with the SNs of that Segment. Periodically, the Manager triggers the Checkpointer to create checkpoints of the Log. The Manager observes the created checkpoints and issues new Segments as the checkpoints advance, respecting the watermark window.
-
The membership module is designed to manage the identities and membership information of nodes within a distributed system. It plays a crucial role in maintaining a consistent view of the network's composition.
-
Module implementing the consensus protocols which actual order Batches, i.e., committing new Entries to the Log. The Orderer listens to the Manager for new Segments. Whenever the Manager issues a new Segment, the Orderer creates a new instance of the ordering protocol that proposes and agrees on Request Batches, one for each SN that is part of the Segment. When a Batch has been agreed upon for a particular SN, the Orderer commits the (SN, Batch) pair as an Entry to the Log.
-
Opaque client data. Each Request deterministically maps to a Bucket, which is a subset of all possible client Requests. Each Request maps to exactly one Bucket (mapping is based on the Request's hash). The Manager assigns one Bucket to each Segment and the Orderer of the Segment only uses Requests from the assigned Bucket to propose new Batches. Batch is an ordered sequence of client Requests. All Requests in a Batch must belong to the same Bucket.
-
The tls-data module is designed for generating and managing TLS certificates and keys. It stores these certificates and keys and provides scripts to automate their creation and management, ensuring secure communication by providing all necessary cryptographic materials and tools for their efficient generation and management.
Create a GOPATH directory and make sure you are the owner of it:
sudo mkdir -p /opt/gopath/
sudo chown -R $user:$group /opt/gopath/
where $user and $group your user and group respectively.
Create a directory to clone the repository into:
mkdir -p /opt/gopath/src/github.com/hyperledger-labs/
Clone this repository unter the directory you created:
cd /opt/gopath/src/github.com/hyperledger-labs/
git clone https://github.com/hyperledger-labs/ladon.git
Checkout theresearch-ladon branch.
With /opt/gopath/src/github.com/hyperledger-labs/ladon as working directory, go to the deployment directory:
cd deployment
Configure the user and group in vars.sh
We use Ubuntu 22.04, Go 1.21.2 and Python 3.10. You will need to install the required dependencies.
-
Ubuntu: (Included in
source scripts/install-local.sh)- protobuf-compiler
- protobuf-compiler-grpc
- git
- openssl
- jq
- graphviz
- make
- gcc
- libc6
-
Go: (Included in
source scripts/install-local.sh)- google.golang.org/grpc
- github.com/golang/protobuf/protoc-gen-go
- github.com/rs/zerolog/log
- github.com/c9s/goprocinfo/linux
- go.dedis.ch/kyber
- go.dedis.ch/fixbuf
- golang.org/x/crypto/blake2b
- gopkg.in/yaml.v2
- github.com/op/go-logging
-
Python: (Need to manually install)
- sqlite3
- numpy
- matplotlib
To install Golang and other related requirements:
source scripts/install-local.sh
NOTE: The install-local.sh script, among other dependencies, installs Go in the home directory, sets GOPATH to /opt/gopath/bin/ and edits ~/.bashrc.
The default path to the repository is set to: /opt/gopath/src/github.com/hyperledger-labs/ladon/.
The run-protoc.sh script needs to be run from the project root directory (i.e. ladon) before compiling the Go files.
IMPORTANT: go modules are not supported. Disable with the command: export GO111MODULE=off before installation.
Compile and install the go code by running go install ./... from the project root directory.
Detailed instructions can be found deployment folder.