Specify sequence using context from the previous function (e.g Annotations)

[twobiers]

Related issue: crossplane-contrib/function-go-templating#154

What problem are you facing?

We currently maintain a couple of medium-sized compositions using function-go-templating. We observe, that dependencies between the composite resources are unavoidable and therefore we are seeing ourselves a lot inside conditional blocks, which get fairly complex to oversee on a larger scale.
While this function addresses the problem, we lack a proper way of defining the sequences in a declarative way. Since we use the go-templating, resources names can be templated using user input and we cannot determine it well enough using regular expression matching.

How could this Function help solve your problem?

We would somehow need a way of passing context from the previous function down to function-sequencer which bridges the gap between free templating and configuration.

One possible solution would be to extend this function to determine the sequence based on a special annotation similar to Argos Sync waves. The function will read the corresponding sequencing from the annotation.

It could look somehow like this:

apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
  name: example-function-get-composed-resource
spec:
  compositeTypeRef:
    apiVersion: example.crossplane.io/v1beta1
    kind: XR
  mode: Pipeline
  pipeline:
    - step: render-templates
      functionRef:
        name: function-go-templating
      input:
        apiVersion: gotemplating.fn.crossplane.io/v1beta1
        kind: GoTemplate
        source: Inline
        inline:
          template: |
            ---
            {{ $flexServerResourceName := "flexServer" }}
            apiVersion: dbforpostgresql.azure.upbound.io/v1beta1
            kind: FlexibleServer
            metadata:
              annotations:
                {{ setResourceNameAnnotation $flexServerResourceName }}
                gotemplating.fn.crossplane.io/sync-wave: "1" # Server is in sync wave 1
            spec:
              forProvider:
                storageMb: 32768
              providerConfigRef:
                name: my-provider-cfg
            ---
            {{ $flexServer := getComposedResource . $flexServerResourceName }}
            
            apiVersion: dbforpostgresql.azure.upbound.io/v1beta1
            kind: FlexibleServerConfiguration
            metadata:
              annotations:
                {{ setResourceNameAnnotation "flexServerConfig" }}
                gotemplating.fn.crossplane.io/sync-wave: "2" # Config is in sync wave 2
            spec:
              forProvider:
                # Populate the field using the observed status of the retrieved resource
                serverId: {{ get $flexServer.status "id" }}

              providerConfigRef:
                name: my-provider-cfg

    - step: sequence-creation
      functionRef:
        name: function-sequencer
      input:
        apiVersion: sequencer.fn.crossplane.io/v1beta1
        kind: Input
        sequenceAnnotation: "gotemplating.fn.crossplane.io/sync-wave"

[briantopping]

This is interesting, I wonder if the targeting could be made a little more generic. In my current configuration, a namespace holds many claims for an XR. The name of the XR is the base name of the composite resources. Many of the composite resources are templated by function-go-templating, like yours.

In my situation, the use of a single static annotation wouldn't work because there could be dozens of parallel sequences, one for each claim.

It might work if the annotation name could be generated from some field of the claim. The annotation name would basically need to be a templated property. Such a name would be easy to use as a unique key for the sync-wave pattern and isolate it from parallel sequences in the namespace.