v0.7.0

Provider CRD & support for multiple compiler frameworks

Summary

This release introduces the Provider Custom Resource, allowing providers to be individually configured and their use restricted. Provider resources own and manage a Provider Service, which makes calls to individual providers and receives run completion events, removing a large portion of the argo workflow templates in the process.
Another key feature of this release is the introduction of support for multiple pipeline frameworks, notably KFP SDK, so users are no longer restricted to only using TFX.

---

What's Changed

Notable updates:

• Define new Provider CRD and use within operator by @Ankitha-Krisshnan, @grahamia, @alexgeorgousis, @TobyPinfold in #353
• KFP Operator run completion feed service by @grahamia, @aidandunlop in #368
• Implement a gRPC event trigger to publish events to NATS by @TobyPinfold, @Ankitha-Krisshnan, @chr12c in #377
• Replace Argo Events stack with Provider Service component by @alexgeorgousis, @TobyPinfold, @peter-hazell, @grahamia, @chr12c, @aidandunlop in #391
• Implement controller to manage Provider service Deployment by @alexgeorgousis, @TobyPinfold, @aidandunlop, @peter-hazell, @grahamia in #408
• Add multi-compiler functionality by @grahamia, @chr12c, @aidandunlop in #483
• Implement KFP SDK compiler by @PeterFras, @paoloambrosio-skyuk, @jmendesky, @edinekj, @Ankitha-Krisshnan, @aidandunlop in #521
• Remove expired start time from VAI schedules by @TobyPinfold, @aidandunlop in #583
• Support cross-namespace Pipeline Dependencies by @aidandunlop, @TobyPinfold in #676
• Run timestamps in run completion events by @TobyPinfold, @grahamia in #689
• Several @dependabot upgrades

Full Changelog: v0.6.0...v0.7.0

---

Migration

1. This release will require the deployment of a Provider resource configured for the provider of your choosing.
2. If you want to deploy the provider to a separate namespace for security purposes, ensure all RBAC permissions are setup accordingly, following the steps under Role-based access control (RBAC) for providers in https://sky-uk.github.io/kfp-operator/docs/getting-started/installation
3. Be sure to update the operator's configuration (either via Helm or Kustomize) to include the manager.runcompletionWebhook,runcompletionEventTrigger and provider configuration blocks.
4. Each user resource (Pipelines, RunConfigurations, Experiment) will then need to be updated to point to the Provider resource. See the user migration guides for more detail on how to do this.

Caution

When upgrading, all resources will reconcile causing any RunConfigurations using the onChange trigger to start new runs in the given provider