[Meta] Collator Selection Improvement

[Dinonard]

Description

This meta issue describes a few improvement ideas for the collator selection pallet.
Main goal is to reduce number of low performant collators, and prevent them from freely rejoining the collator candidate list.

The task list isn't definite and can be expanded and/or modified according to the implementor's design.

1) More advanced collator performance tracking

• histogram of collator performance per session
• track peak and/or median value
• immediately detect if some collator isn't up to par (e.g. falls too much below the median/peak or some other metric)
• slash, kick & maybe blacklist collators if this happens (automatic)
• collators shouldn't be able to manipulate this metric (e.g. produce blocks one session, then another session do nothing)

2) Leave period

• introduce "leave period" for collators during funds remain reserved, instead of allowing instant leave

3) Removal of permissionless collators

• collators can no longer become candidates just by bonding 3.2M ASTR
• they can only apply for this position by bonding the aforementioned amount
• it's up to the council (e.g. can be Community council and/or Main council) to decide if this is someone we want to have or not
• council approves the application, or rejects it
• council should also be able to kick out some collator
• this eliminates completely anonymous collators
• if improved performance tracking kicks them out, they cannot just rejoin again via a new anonymous account

4) RPC call

• ensure collator performance can be tracked via some RPC call
• can be just reading some pallet storage item, but it should be documented (e.g. for the DevOps team to use)

[Luca-Poggi]

Hello, I'm Luca Poggi from 🧊 Iceberg Nodes 🧊 and we are active collators on Astar, Shiden and Shibuya since almost the genesis block of each network, so we followed all the improvements to the validators in Plasm (Astar was called Plasm at the beginning, and it was a standalone network so with a validators set instead of collators set) and collators in Astar.

I'm here in order to ask some questions and maybe provide my feedback and improvements to the pallet. I was about to open a new pull request in order to request the possibility for permissionless collator to use the locked funds (3.2M ASTR) in the governance, but since I found this already open, I will ask and comment directly here. Hope this will be helpful to Astar and to your work.

• introduce "leave period" for collators during funds remain reserved, instead of allowing instant leave

Do you have any idea how long this might take? Because as you know, the challenges for a collator today are:

• low rewards, or at least comparable to staking in dApp Staking (which is risk-free, almost completely passive, etc.)
• the presence of slashing, which makes it more dangerous
• active staking due to monitoring and the associated infrastructure and management costs
  With the only advantage: rapid or almost immediate unstake

• collators can no longer become candidates just by bonding 3.2M ASTR
• they can only apply for this position by bonding the aforementioned amount

I think this is the right time for my question: is it possible to design the new pallet so that the amount required for bonding (3.2M ASTR) can be used for governance purposes (voting in referendums, etc.) just like capital locked in stake in the DApp Staking?

With the launch and subsequent increased adoption of governance in Astar, I believe it's very important to allow all parties (collators included) the ability to express their will with their ASTR tokens. Ultimately, a collator has ASTR bonded in order to provide their services to the network, also it is an ASTR holder and should have the right to express its choices in the various referenda on-chain.
In the past, I recall that the amount required to become a collator was bonded in the collator, but could even be used for staking in the DApp Staking. This option was later removed, likely in conjunction with the introduction of slashing (which perhaps prevents the capital from being used for other purposes, including governance?!). If this were the case, I believe the capital locked up for collators should be allowed to be used in governance (as a minimum requirement). I've received several requests to do so and believe it's very important to ensure that participation in various referendums increases over time (it's a fact that a large amount of ASTR is locked up in collators). It would be even better if it could also be used for staking in the DApp Staking, allowing collators to have a double income, rewarding their work and repaying infrastructure costs. To give more context to this request: unfortunately, to date, the collator category has seen a 50% reduction in rewards, followed by a further 50% reduction a few months later due to tokenomics changes.

• it's up to the council (e.g. can be Community council and/or Main council) to decide if this is someone we want to have or not
• council approves the application, or rejects it

So we can say that we are going to migrate from a permissionless collator system to a permissioned one (PoA)? Or Council will evaluate only based in performance or also about decentralization (for example avoiding to some entities to run more that one permissionless collator?)

And finally I have one last question:
How will the migration to the new collator pallet be managed? Will current permissionless collators have to take any action (such as re-registering), or will it only be necessary after being removed the first time due to poor performance?

Furthermore, will the role of whitelisted collators still exist with this new system, since as if I understood correctly the collators should be approved by council? If the whitelisted collators still remain, don't you think it's appropriate to draw up some sort of regulation defining both the rules for joining and the rules for remaining a member, as well as the obligations to the community required for that role (e.g., supporting Shibuya and Shiden, providing a public RPC endpoint, snapshots, managing the local community, etc.), which is currently completely undefined. The evaluation role could be carried out by the community council or council.

I hope this post has helped clarify some of the concerns that may arise during the design and implementation of the new pallet. I'm also available here, and/or in the Astar forum, and/or privately on Telegram (t.me/LucaPoggi) to answer any questions and provide further feedback. I'm delighted that there's finally been a focus on bringing a breath of fresh air to the technical aspects of Astar's collator system!

Best Regards
Luca Poggi
Iceberg Nodes

[Dinonard]

Hi @Luca-Poggi,
thanks a lot for your comments. It's great to see someone from the community comment on something like this.

For the leave period, we'll see - based on your comment, my inclination right now is more to not do it.
We can come up with a solution later which would allow collator bonded funds to be used for governance.

The thing is, if some funds are slashable (as in collator bonding), they are not allowed for use in e.g. dApp staking or governance since it creates problems in case that amount is slashed. Maybe we can find some workaround for this in the future.

---

For the council approval, we see it more as something temporary to get rid of low performant collators.
Implementing on-chain metric can be tricky, has to be audited, and can bring all sort of problems.
Current one is very basic, and bad.

Adding additional gating before collator just jumps in is more of a "sanity" check.
But more importantly, if collator is constantly missing blocks or messing with the network, the council can just remove them.

It won't be PoA because even though the council can approve application, the same can be done via a referendum.
So it's still PoS, but with some PoA-like element, like a KYC (I hate that word but it is what it is).