diff --git a/proposals/arbitrary-broker-resources.md b/proposals/arbitrary-broker-resources.md new file mode 100644 index 0000000..9ba40e4 --- /dev/null +++ b/proposals/arbitrary-broker-resources.md @@ -0,0 +1,131 @@ +# Deploy arbitrary Kubernetes resources as broker + +## Background + +Currently, a broker is defined as either a Job or a Pod, and can be accompanied by a Configuration level Service and an Instance level service, as shown in following YAML example. + +```yaml +apiVersion: akri.sh/v0 +kind: Configuration +metadata: + name: akri-debug-echo-foo +spec: + brokerSpec: + brokerPodSpec: + containers: + - name: akri-debug-echo-foo-broker + image: "nginx:stable-alpine" + resources: + requests: + "{{PLACEHOLDER}}" : "1" + memory: 10Mi + cpu: 10m + limits: + "{{PLACEHOLDER}}" : "1" + memory: 30Mi + cpu: 29m + instanceServiceSpec: + type: ClusterIP + ports: + - name: akri-debug-echo-foo-instance-service + port: 6052 + protocol: TCP + targetPort: 6052 + configurationServiceSpec: + type: ClusterIP + ports: + - name: akri-debug-echo-foo-configuration-service + port: 6052 + protocol: TCP + targetPort: 6052 +``` + +This limits the interoperability of Akri with other Kubernetes based components that would benefit from Akri creating CRs or other Kubernetes native resources. +For example Akri could act as the discovery component of a larger system that leverage Digital Twins with [Shifu](https://shifu.dev/). + +## New broker definition + +The idea behind the new broker definition is to provide two fields `instanceBrokerResources` and `configurationBrokerResources` that will contain a list of Kubernetes resources to deploy. + +Instance level resources will get deployed for every instance, and Configuration level ones will get deployed once for a Configuration with at least a discovered Instance. + +Akri will still add Configuration and Instance labels to resources it creates. + +## New template variables + +All those possibilities requires something a bit more advanced than the current `{{PLACEHOLDER}}` replacement, as the resources might need additional fields like the Instance name, or some of its properties. +To cover this let's have a couple of other "variables" that would replace the `PLACEHOLDER` one: + +| Variable | Description | +| -------- | ----------- | +| INSTANCE_NAME | The name of the instance | +| INSTANCE_PROPERTIES | Dictionary with the `brokerProperties` of the Instance | +| LABEL_SELECTOR | Label Selector that matches instance level resources (only this instance when used in an instance level resource, all instances when used in configuration level resource)| +| NODES | List of node the instance is available on | + +## Example + +The previous example would the looks like this: + +```yaml +apiVersion: akri.sh/v0 +kind: Configuration +metadata: + name: akri-debug-echo-foo +spec: + instanceBrokerResources: + - apiVersion: v1 + kind: Pod + metadata: + name: "{{INSTANCE_NAME}}-pod" + spec: + containers: + - name: akri-debug-echo-foo-broker + image: "nginx:stable-alpine" + resources: + requests: + "akri.sh/{{INSTANCE_NAME}}" : "1" + memory: 10Mi + cpu: 10m + limits: + "akri.sh/{{INSTANCE_NAME}}" : "1" + memory: 30Mi + cpu: 29m + - apiVersion: v1 + kind: Service + metadata: + name: "{{INSTANCE_NAME}}-svc" + spec: + type: ClusterIP + ports: + - name: akri-debug-echo-foo-instance-service + port: 6052 + protocol: TCP + targetPort: 6052 + selector: {{LABEL_SELECTOR}} + configurationBrokerResources: + - apiVersion: v1 + kind: Service + metadata: + name: akri-debug-echo-foo-svc + spec: + type: ClusterIP + ports: + - name: akri-debug-echo-foo-configuration-service + port: 6052 + protocol: TCP + targetPort: 6052 + selector: {{LABEL_SELECTOR}} +``` + +## Technical implementation ideas + +This proposal implies many changes, and most importantly makes a breaking change to the CRD. It also implies a lot of changes to the controller and the admission webhook. + +For the CRD itself, we can make use of `x-kubernetes-embedded-resource` that should help with validating the asked object is valid. + +For the deployment, the RBAC would need to be much more permissive for the controller, probably allowing pretty much everything. + +The admission webhook will also have to deal with mostly unspecified objects, one way to effectively solve this would be to try dry-run submit the resources to be created with a dummy instance. + +The controller will have to watch over every object it owns without prior knowledge of its type.