Networking: DoS protection

[adizere]

Context that triggered this discussion

1. Once a messages goes through PubSub it is gone. We would need to add some form of storage/replay at the application layer if we want to ignore messages and receive them later.
2. We are trying to prevent malicious nodes DoSing other nodes. Specifically we are considering an OOM attack (what a node stores), not a network card attack (spam saturating our bandwidth).
3. We assume that there is a sync mechanism, so nodes which lag in height can receive old decisions. Our question then is making sure that nodes, once they reach the active height, can join the active round.
4. The options are:
   a. A node is responsible for its own messages and needs to remember (some) future messages.
   b. Nodes share messages with peers to help them catch up. (cometBFT)

Notes

Date: June 5, 2024
People: Sergio, Hernan, Romain, Greg, Adi

Some takeaways:

• Look at replay support and protection in GossipSub
• We might investigate adding a new protocol (separate from GossipSub), using libp2p, to handle pairwise communication between nodes s.t. pairs of nodes assist each other in catching up
• We will likely add a block sync mode that allows nodes to quickly catch up with a network using a different communication model than the random all-to-all model that GossipSub provides
• We should tackle DoS protection in a layered approach, ideally in the p2p layer, without involving application layer
• Lesson to learn from CometBFT: We should probably support seamless and often and automatic switch between consensus mode and block sync mode; presumably consensus mode leverages the randomized gossip communication model, whereas block sync would leverage a smarter and pairwise model of communication where peers could "stick" to other peers that are more getting data from

[ancazamfir]

I think the main questions are around point 4. We assume there is blocksync like mechanism.
(if we are to implement a blocksync it will have to be interoperable with starknet implementation so it needs to be jointly specified)

Regarding 4, i believe it’s around the two approaches specified in the comet gossip spec

One part that is not clear is how to also support validator restart with a) (also Approach One in the spec above) and have the ability to join the “active” round. I think this is something we should look into.

And to clarify b) refers to Comet's CRDT approach (implemented via HasVoteMessage, VoteSetMaj23Message, VoteSetBitsMessage)

Another note for validator start or longer restart, the blocksync does not necessarily brings the validator up to the current consensus "active height" (it could be one or two heights behind) so it still needs a consensus sync (via a or b approach) over multiple heights.

[cason]

The simplest scenario of this discussion is defined in #235.

This is the case where there is no desynchronization in terms of the height each validator is at, but when we have some missed vote messages, potentially blocking success.

The topic 4. above is addressed with the two proposed solutions:

• 4a. a node that is stuck in a voting step, re-broadcast the votes it has produced in the current round
• 4b. we need a protocol for requesting missing votes to peers

My reasoning there tends to prefer the first option, because it is simpler and does not require a new protocol. But this is only one of the scenarios we have to consider.

[cason]

I've also created a tracking issue to cover all the components discussed here: #260