[feature] Allow recurring runs to default to the latest pipeline version

[mprahl]

Feature Area

/area frontend
/area backend
/area sdk

What feature would you like to see?

Allow recurring runs to always use the latest pipeline version of a pipeline.

What is the use case or pain point?

A user may want to have a recurring run/scheduled workflow on the latest pipeline version so they don't need to modify their recurring run/scheduled workflow whenever there is a new pipeline version uploaded. This is particularly useful if the maintainer of the pipeline version (i.e. included in a distribution of Kubeflow Pipelines) is different than the person maintaining the recurring run/scheduled workflow.

Desired outcome:

• Allow for creating a recurring run/scheduled workflow with just specifying the pipeline ID. Thus, the pipeline version ID would be optional. The ScheduledWorkflow controller will find the latest pipeline version in the pipeline and compile to Argo Workflow YAML. The ScheduledWorkflow controller should directly compile it without having to connect to the API server. The Argo Workflow YAML should be cached per pipeline version to avoid repeating this.
• The ScheduledWorkflow controller can do the compilation for all new ScheduledWorkflows to avoid the compiled Argo Workflow YAML (implementation details) from going stale in the ScheduledWorkflow object.
• The SDK/clients allow creating such recurring runs.

Is there a workaround currently?

The user must manually update/recreate the recurring run every time there is a new pipeline version.

---

Love this idea? Give it a 👍.

[vishalmishra369]

hey if this problem has not been taken by anyone can i work on it ?
If yes then please do assign it to me !!.

regards,
vishal

[mprahl]

@vishalmishra369 sorry, I already started working on it and I forgot to assign myself. Thanks so much for volunteering though!

Please feel free to look at other open issues. 😄

[mprahl]

/assign @mprahl