Swappable Consensus

[rphmeier]

cc @andresilva @svyatonik @gnunicorn

One of the features we want for Substrate is the ability to switch between consensus algorithms. This is not trivial to do, because different consensus algorithms have different notions of finality and it is not always clear when to switch, especially when something like a hybrid (block authorship + finality) system is used.

Consensus in Substrate is done via long-running Future instances that perform authorship or finality tasks. The way we will do pluggable consensus is just by writing a consensus manager future which knows how to multiplex and start other futures for registered consensus algorithms.

"Restarting" the consensus algorithm like this is very similar to the way we've implemented GRANDPA set changes, and indeed you can think of GRANDPA set changes as a switch between two instances of the consensus algorithm.

Design Constraints

1. We want to support hybrid consensus. In particular, when switching from a hybrid consensus algorithm to another hybrid consensus algorithm, the block authorship and finality handoffs should be done differently.
2. Fork-choice rules need to somehow span across consensus implementations
3. Stretch: switching to/from probabilistic finality consensus

Runtime Workings

There is a special :consensus key in storage. This can generally be empty, but when changing consensus, runtimes must do two things:

• Set :consensus to a (u32, Vec<([u8; 4], Vec<u8>)>), which is the "consensus index" (incremented each time) and the configuration of the consensus processes that should be run from this point onwards
• Issue ConsensusChange digest.

Runtimes can switch out consensus algorithm by making a change to the :consensus key in storage.
:consensus stores a (u32, Vec<([u8; 4], Vec<u8>)>) which indicates all the consensus processes that are currently being run and the "index" of this consensus configuration.

If there are duplicate [u8; 4] magic numbers in the vector, only earlier instances will be mentioned.

Changes to the :consensus key must correspond to issuance of the ConsensusChange digest item. This signals to clients that there is a scheduled change in consensus algorithms.

The storage in :consensus may be cleared after the signal is set, as long as the first 4 bytes still represent the consensus index.

Node-side

Each consensus algorithm has a "magic number": this is the [u8; 4] which the :consensus key will contain (potentially many) of. Each Vec<u8> is a corresponding configuration for the engine.

The node manages a ConsensusOverseer which is essentially a

// how to launch consensus instances
Map<[u8; 4], Fn(Vec<u8>) -> Result<ConsensusInstance>>,
// live consensus instances:
Map<u32, Vec<ConsensusInstance>>,

There will be a standard configuration and mapping of [u8; 4] to consensus engines but more can be added by custom substrate nodes.

The question is mostly about when we stop old consensus futures. For instant-finality or finality providers, it's quite clear that the changes will take effect after a block containing the signal is finalized.

New block authorship workers will have to be started instantly, but will have to be limited to chains that contain the correct consensus-index. Likewise, the old authorship workers would have to be limited to author on chains that contain the correct old consensus-index, but they would also shut down as soon as a block with the new consensus-index was finalized.

e.g. Aura process X is allowed to build on blocks with consensus-index N, and Ouroboros process Y is allowed to build on blocks with consensus-index N+1. They both run in parallel, but Aura shuts down as soon as a block with index N+1 is finalized.

Changing From probabilistic finality.

Let's consider a situation where we fork from PoW to Aura/GRANDPA. The new GRANDPA instance won't want to start until a block is finalized, but Aura will start immediately. Neither PoW or Aura will finalize anything, so where do we tell the new GRANDPA process to start?

I'll leave this as an open question for now, but the solution probably has to do with forcing all consensus configurations to "finalize" in some sense. I.e. the ProgPoW configuration Vec<u8> might be the encoded representation of

version: 1
start_difficulty: X,
finalization_depth: Option<BlockNumber>, // can be `None`, but only when coupled with a finality gadget. This is hardcoding a reversion limit at the consensus level.

Aura would be similar.