Support sequences in the reation of composite resources

[twobiers]

What problem are you facing?

We currently maintain a couple of medium-sized compositions using function-go-templating. We observe, that dependencies between the composite resources are unavoidable and therefore we are seeing ourselves a lot inside conditional blocks, which get fairly complex to oversee on a larger scale.
Also, we can't express dependencies on resource-healthiness properly out of the same reasons expressed in #85.

How could this Function help solve your problem?

One possible solution would be to extend function-go-templating with a feature similar to Argos Sync waves. The function will incrementally extend the desired state with composite resources from the corresponding sync-wave.
Also take a look at the outlined alternatives down below.

This could look like the following:

apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
  name: example-function-get-composed-resource
spec:
  compositeTypeRef:
    apiVersion: example.crossplane.io/v1beta1
    kind: XR
  mode: Pipeline
  pipeline:
    - step: render-templates
      functionRef:
        name: function-go-templating
      input:
        apiVersion: gotemplating.fn.crossplane.io/v1beta1
        kind: GoTemplate
        source: Inline
        inline:
          template: |
            ---
            {{ $flexServerResourceName := "flexServer" }}
            apiVersion: dbforpostgresql.azure.upbound.io/v1beta1
            kind: FlexibleServer
            metadata:
              annotations:
                {{ setResourceNameAnnotation $flexServerResourceName }}
                gotemplating.fn.crossplane.io/sync-wave: "1" # Server is in sync wave 1
            spec:
              forProvider:
                storageMb: 32768
              providerConfigRef:
                name: my-provider-cfg
            ---
            {{ $flexServer := getComposedResource . $flexServerResourceName }}
            
            apiVersion: dbforpostgresql.azure.upbound.io/v1beta1
            kind: FlexibleServerConfiguration
            metadata:
              annotations:
                {{ setResourceNameAnnotation "flexServerConfig" }}
                gotemplating.fn.crossplane.io/sync-wave: "2" # Config is in sync wave 2
            spec:
              forProvider:
                # Populate the field using the observed status of the retrieved resource
                serverId: {{ get $flexServer.status "id" }}

              providerConfigRef:
                name: my-provider-cfg

Within the sync-wave 1 the desired state would consist of only [flexServer]. Once all resources become ready, the desired state moves on the next wave 2 and we will add flexServerConfig to it.

Alternatives

• One alternative would be to use a dedicated function for that like function-sequencer. However, it would probably not be a fully equivalent solution to the problem as we would lose template capabilities
• We could also enhance the function-sequencer to add support for reading the sequence from a customizable annotation.

[bobh66]

This is the purpose of function-sequencer, to work along with other pipeline functions to sequence the resources that are generated.

There have been ongoing discussions regarding the ability to pass input programmatically from one function to another, so that for instance function-sequencer could have it's inputs defined dynamically by function-go-templating by passing data through the context.

I think this is a more desirable solution than extending function-go-templating to include functionality that is beyond the scope of it's main purpose.