postBuild.substituteFrom ignores strategic-merge `patches:` on the source ConfigMap/Secret

[banschikovde]

postBuild.substituteFrom ignores strategic-merge patches: on the source ConfigMap/Secret

flate version: v0.4.10 (latest at time of writing)

Summary

When a ConfigMap/Secret used as a postBuild.substituteFrom source is built
by a kustomization that adds keys via a strategic-merge patches: entry, flate
resolves only the keys present in the resources: version of the object. Keys
introduced exclusively by patches: are missing from the substitution variable
map (resolve to empty). kustomize build of the same kustomization merges them
correctly, and Flux applies the merged object in-cluster — so this is an
offline-model gap, not a malformed manifest.

Minimal reproducer

Directory tree:

cluster/
  ks.yaml                   # Flux Kustomization (substituteFrom: ConfigMap/cluster-settings)
  vars/
    kustomization.yaml      # resources:[base.yaml] + patches:[patch.yaml]
    base.yaml               # ConfigMap cluster-settings { VAR_A: base-value }
    patch.yaml              # ConfigMap cluster-settings { VAR_B: patch-value }  (strategic-merge patch)
  app/
    kustomization.yaml
    cm.yaml                 # ConfigMap using ${VAR_A} and ${VAR_B}

cluster/vars/base.yaml:

apiVersion: v1
kind: ConfigMap
metadata:
  name: cluster-settings
  namespace: flux-system
data:
  VAR_A: base-value

cluster/vars/patch.yaml (the strategic-merge patch — adds a key):

apiVersion: v1
kind: ConfigMap
metadata:
  name: cluster-settings
  namespace: flux-system
data:
  VAR_B: patch-value

cluster/vars/kustomization.yaml:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - base.yaml
patches:
  - path: patch.yaml

cluster/ks.yaml:

apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: app
  namespace: flux-system
spec:
  interval: 5m
  path: ./cluster/app
  prune: true
  sourceRef:
    kind: GitRepository
    name: flux-system
  postBuild:
    substituteFrom:
      - kind: ConfigMap
        name: cluster-settings

cluster/app/cm.yaml:

apiVersion: v1
kind: ConfigMap
metadata:
  name: app-cm
data:
  a: ${VAR_A}
  b: ${VAR_B}

Expected vs actual

kustomize build merges both keys (this is what Flux applies in-cluster):

$ kustomize build cluster/vars
apiVersion: v1
data:
  VAR_A: base-value
  VAR_B: patch-value
kind: ConfigMap
metadata:
  name: cluster-settings
  namespace: flux-system

flate drops VAR_B (the patch-only key) → resolves to null:

$ flate build ks --path cluster app
---
apiVersion: v1
data:
  a: base-value
  b: null              # <-- expected: patch-value
kind: ConfigMap
metadata:
  name: app-cm
  namespace: flux-system

Analysis

flate loads the substituteFrom source object from its resources: form but
does not apply the enclosing kustomization's patches: (strategic-merge) step,
so the merged object that kustomize/Flux produce is not the one flate feeds into
the substitution var map. When two objects collide on metadata.name (base +
patch, both named cluster-settings), the resources: version wins and the
patch's keys are lost.

Real-world impact

Breaks reconciliation of any KS whose rendered resources reference a variable
that is defined only via a strategic-merge patch on the substituteFrom
ConfigMap/Secret. A common shape is per-cluster overrides: a shared base
cluster-settings (cluster-domain, etc.) extended per-cluster via a patch that
adds CLUSTER_NAME / CLUSTER_PROJECT. With flate those resolve empty; an
expression like ${CLUSTER_PROJECT}-${CLUSTER_NAME} then collapses to a lone
-, which go-yaml reads as a block-sequence indicator and the whole KS fails:

substitute: unmarshal doc: go-yaml load error in scanner at L18.C18:
block sequence entries are not allowed in this context

…and every dependsOn consumer cascades to FAILED.

Related observation (possibly separate)

HelmReleases/HelmRepositories that receive their namespace from a kustomization
namespace: field (rather than an explicit metadata.namespace) appear twice
in flate test output: once correctly namespaced (e.g. HelmRelease kafka/akhq,
✓) and once as a phantom namespace-less copy (HelmRelease akhq, ✗ — "chart
source ... not ready: dependency not found"). The phantom seems to come from a
direct file walk that bypasses the kustomize namespace transform, duplicating
resources that are also loaded via the kustomize-built KS path. Happy to file a
separate minimal repro for that if useful.

[buroa]

Confirmed, thanks for the precise reproducer.

Root cause: the substituteFrom source object is materialized into the store via a raw YAML parse (the Existence/promote path), which follows only resources: and never applies the enclosing kustomization's patches: step. So patch-only keys are absent from the ConfigMap's data by the time substitution reads it, and they expand to empty. The producing kustomization does emit the patched object later, but the consumer's substitution has already run during its prepare phase.

Plan: resolve substituteFrom ConfigMap/Secret sources through the kustomize build of their enclosing package (applying patches:) rather than the raw-file parse, with a regression test covering a patches:-added key consumed downstream.