[GOT-38] Scope a proposal for versioning the execution pipeline (code, spec, communication to validators)

[jsign]

We need to scope how backward-compatible protocol upgrades would work:

• In the validator.
• The spec should now start explaining which feature lives in which network version.
• Include network versioning in our docs.
• Have playbooks to know how to deploy a new version at a future point in time, and communicate to validators to update their node with enough time, etc.
• See if we want to include contract upgrades as part of network versioning and how this fits into the validator, SDKs, etc.

This will take a reasonable amount of work, and there're more things to figure out. The above points are a brain dump of obvious things. This will probably need multiple people participating in this design.

cc @brunocalza @sanderpick @carsonfarmer @joewagner

_GOT-38

[brunocalza]

Sounds like a good start.

One thing that's still a bit blurry in my head is the reasoning behind what constitutes a new version of the execution pipeline or what is a protocol upgrade. In other words, what kind of features should follow this process and which not? Sounds like something we need to figure out too.

[jsign]

Yes, at a minimum any protocol change that would cause a divergence in the state of non updated validators.

[sanderpick]

👍

Some other small thing to mention:

• Figure out how validator advertises its protocol version (header field in API responses and/or dedicated API endpoint). This way we can visualize the rollout of a new version w/ some global dash

[joewagner]

Lots of moving parts here. I think the crux of versioning will be found in the Validator and the Registry Contract. I'll try to add to the brain dumps, sorry ahead of time if this turns into a bit of a ramble...

When I think about versioning something, I always try to divide it into 2 parts:

• producer: an entity that makes (produces) something, e.g. a source of truth.
• consumer: an entity that uses the output of a consumer. The consumer might ask for information, or broadcast their existence and wait to receive information.
  When a version change takes place the consumers need to be able to keep using the their old producers while they transition to using the new producers in an asynchronous way. The producers need to keep producing their old product while also producing the new product for a long enough amount of time to allow the consumers to upgrade.

In regard to the Validator, the consumer that comes to mind is the SC event execution and the producer is the http rest and rpc endpoints.
In regard to the Registry, the consumer is the exposed public functions and the producer is the events emitted.

Thinking about the Registry Contract in this way, we need to do two things to handle new versions:

• Producer: We need to emit version 1 compatible events while simultaneously emitting version 2 compatible events.
• Consumer: We need to expose version 1 compatible public functions while simultaneously exposing version 2 compatible functions.

Thinking about the Validator in this way, we need to do two things to handle new versions:

• Producer: We need to serve version 1 compatible http endpoints while simultaneously serving version 2 compatible http endpoints.
• Consumer: We need to listen for version 1 compatible Registry events while simultaneously listening for version 2 compatible Registry events.

For the Validator some of the questions and comments I have are:

• for versioned http endpoints do we want to put the version in the URI path, querystring, custom header, content negotiation? e.g. http://localhost:8080/api/{api_version}/chain/{chain_id}/tables/controller/{owner_address}.
  My inclination would be to use uri or querystring so market places don't have to worry about headers or content negotiation.
• versioning the event processor and http endpoints adds a large overhead to the codebase. Not much to say about this just mentioning.

For the Registry Contract some of the questions and comments I have are:

• We have an upgradeable contract, which I think we should keep, but perhaps when we upgrade a protocol version we could deploy an entirely new (upgradable) contract. This would let the version 1 compatible contract continue to play its role in version 1 while the version 2 compatible contract plays its role in version 2. In this way we have as much overlap in supported versions as we want.
• If we have a multiple contracts deployed on the same chain emitting events for the same table in the same block, does that cause a non-determinism issue? or some other issue?

There's probably a lot more to discuss here, but I'll stop there.