Allow for more generic CRDs to be deployed when device is discovered

[agracey]

Is your feature request related to a problem? Please describe.
Some of my workloads are better served by VMs or WebAssembly than pods but I would still like to control their lifecycle and placement with Akri as the discovery mechanism.

Is your feature request related to a way you would like Akri extended? Please describe.
I would like to be able to control additional CRDs in addition to a broker pod

Describe the solution you'd like
An additional field in the Configuration alongside spec.brokerSpec.brokerPodSpec and spec.brokerSpec.brokerJobSpec called something like spec.brokerSpec.additionalYAML that would be freeform.

Describe alternatives you've considered
I've thought about using a job or pod to trigger the creation of the yaml (with a finalizer to remove). This has some obvious drawbacks and is likely very brittle...

Additional context
Another use-case that I ran into that would be served by the same change is an app where I need to use ConfigMaps to control what devices the app expects to reach out to.

[kate-goldenring]

From the start of Akri, we envisioned creation of other CRDs (beyond Akri Instances) upon discovery to suit more use cases. Sounds like the moment is here. The main goal of the Akri Controller is to watch for Akri Instance creation/deletion and create resources (Pods or Jobs). CRDs could be added to the list of resources the controller could create.

I think this could introduce an evolution of terminology. A broker Pod/Job could be a subcategory of a larger category of resourceCreatedUponDiscovery (hopefully better naming). Do you envision there being a scenario where you'd want both brokers and CRDs deployed upon instance creation?

Would be great to discuss this at the community meeting next week.

To echo the alternative you mentioned. I tried to kickstart this outside of Akri using KEDA. I wanted to see if we could use KEDA to create other resources (such as new CRDs) upon Instance CR creation but KEDA cannot be driven by CRD creation events. Even if this had worked, its easier using the Akri Controller since we can add clear Ownership to the created resources so they are owned by the Configuration.

[agracey]

Do you envision there being a scenario where you'd want both brokers and CRDs deployed upon instance creation?

I don't know. But I would argue that at the end of the day it doesn't really matter since a Pod is really just a CR with a built-in controller and the Pod/Job specific fields would be shorthand with better validation. I don't know enough about the internals of Akri to know if there is a different treatment.

Would be great to discuss this at the community meeting next week.

I'm traveling next week but can add January's call to my schedule if I can't join from the venue.