A swiss army knife for Kubernetes manifests — multi-stage pipelines for fetching, processing, and rendering.
many is a GitOps tool to process Kubernetes manifests from many sources in many formats.
Sources can live locally, in an OCI registry, behind an HTTPS URL, or in an
OCM component --- many fetches and processes them in a pipeline.
many discovers .many.yaml pipeline definitions in a directory tree, executes them and writes the results to a output
directory.
Each pipeline composes steps --- Go templating (with Sprig), Kustomize builds, Helm renders, file generation, and YAML splitting --- to turn a template repository into ready-to-apply Kubernetes manifests.
many can fan out outputs to many instances, check the examples.
- Installation
- Examples
.many.yamlReference- CLI Reference
- Pipeline Steps
- Sources
- Context
- Execution Model
- Environment Variables
go install github.com/systemstart/many-templates/cmd/many@latestOr build from source:
make build # produces ./manyexamples/source-https/ fetches a YAML file from a
GitHub release and renders it as a Go template.
# examples/source-https/.many.yaml
pipeline:
- name: fetch-and-render
type: template
source:
# renovate: datasource=github-releases depName=kubernetes-sigs/metrics-server
https: "https://github.com/kubernetes-sigs/metrics-server/releases/download/v0.7.2/components.yaml"
sha256: "f103539a54ed72efe66616afc74a8bfaed651703cb3918797599046af5617441"
template:
files:
include: ["**/*.yaml"]The source on the step fetches files into the working directory before the step
executes. Here an HTTPS source is downloaded (tarballs are auto-extracted), then
all YAML files are rendered as Go templates with
Sprig functions.
SHA-256 verification --- the sha256 field pins a source to a known checksum.
When set, many verifies the download matches before proceeding. An empty string
disables verification, useful during development. The checksum will be set if
empty, unless -no-sha256-update is provided. Note: that doesn't update an
existing checksum; to update, empty it manually first.
Renovate integration --- the # renovate: comment is a
Renovate annotation. Renovate can be configured
to update both the URL and the sha256 checksum.
many -input examples/source-https -output-directory ./outputexamples/kubevirt/ downloads upstream release YAML files,
splits each into individual resources, and generates a kustomization.
# examples/kubevirt/source/.many.yaml (abbreviated --- 4 similar split steps)
pipeline:
- name: split-kubevirt-operator
type: split
source:
# renovate: datasource=github-releases depName=kubevirt/kubevirt
- https: "https://github.com/kubevirt/kubevirt/releases/download/v1.7.2/kubevirt-operator.yaml"
sha256: "b74106509bbce107a01b228083b4030890b2278cf918fe5c2cf380fdc2a27aef"
path: build/
split:
input: build/kubevirt-operator.yaml
by: resource
outputDir: manifests/operator/
exclude:
- "build/**"
# ... split-kubevirt-cr, split-cdi-operator, split-cdi-cr ...
- name: create-kustomization
type: kustomize-create
kustomize-create:
autodetect: true
recursive: trueEach source fetches a release YAML into build/. The split step breaks it
into one file per resource under manifests/, and exclude cleans up the
intermediate build/ directory. The final kustomize-create
step generates a kustomization.yaml referencing all discovered manifests:
output/
├── kustomization.yaml
└── manifests/
├── cdi/
│ ├── cdi-cdi.yaml
│ ├── deployment-cdi-operator.yaml
│ └── ...
└── operator/
├── kubevirt-kubevirt.yaml
├── deployment-virt-operator.yaml
└── ...
many -input examples/kubevirt/source -output-directory ./outputexamples/many-sites/ renders the same services for
multiple environments. Each instance selects a subset of services and provides
per-instance context values.
instances.yaml defines two instances with different service selections:
instances:
- name: fediverse
input: "services"
output: fediverse.example/
include: [dex, eso, lldap, mastodon, matrix, mobilizon, pixelfed]
context:
domain: "fediverse.example"
siteName: "fediverse"
- name: development
input: "services"
output: development.example/
include: [dex, eso, lldap, forgejo, harbor, woodpecker]
context:
domain: "development.example"
siteName: "development"context.yaml provides shared configuration. String values can reference other context keys via Go template interpolation (single pass after merge):
subdomains:
mastodon: "social"
forgejo: "git"
# ...
smtp:
from: "noreply@{{ .domain }}"
eso:
remotePathPrefix: "site/{{ .siteName }}"Services follow two patterns. Helm-based services (dex, mastodon, ...) use
kustomize-build with Helm and split the output:
# services/dex/.many.yaml
context:
namespace: dex
pipeline:
- name: render-templates
type: template
source:
- file: kustomization.yaml
- file: values.yaml
- file: externalsecret.yaml
template:
files:
include: ["**/*.yaml"]
- name: build
type: kustomize-build
kustomize-build:
enableHelm: true
outputFile: kustomize-output.yaml
- name: split-output
type: split
exclude: ["kustomize-output.yaml"]
split:
input: kustomize-output.yaml
by: resource
outputDir: manifests/
- name: create-kustomization
type: kustomize-create
kustomize-create:
autodetect: true
recursive: true
dir: manifests/Raw-manifest services (lldap, woodpecker, ...) render templates and generate a kustomization:
# services/lldap/.many.yaml
pipeline:
- name: render-templates
type: template
source:
file: manifests/
path: manifests/
template:
files:
include: ["**/*.yaml"]
- name: create-kustomization
type: kustomize-create
kustomize-create:
autodetect: true
recursive: true
namespace: lldapContext is merged in layers --- global (-context-file) → instance
(context in instances.yaml) → pipeline-local (context in .many.yaml) ---
with later layers overriding earlier ones via deep merge.
many \
-input examples/many-sites \
-output-directory output \
-instances examples/many-sites/instances.yaml \
-context-file examples/many-sites/context.yamlOutput:
output/
├── fediverse.example/
│ ├── dex/
│ │ ├── kustomization.yaml
│ │ └── manifests/...
│ ├── mastodon/...
│ └── ...
└── development.example/
├── dex/...
├── forgejo/...
└── ...
A .many.yaml file placed anywhere in the input tree defines a pipeline.
Below is the complete structure with all available fields:
# Optional: pipeline-local context variables, available as {{ .key }} in templates.
# Deep-merged on top of global and instance context (see Context).
context:
key: "value"
nested:
key: "value"
# Required: at least one step.
pipeline:
- name: step-name # required, must be unique within pipeline
type: template # required: template | kustomize-build | kustomize-create
# helm | split | generate | copy
# --- Source (optional) ---------------------------------------------------
# Fetch files into the working directory before the step runs.
# Single entry:
source:
https: https://example.com/v1.tar.gz
sha256: "abc123..." # 64-char hex; empty string disables verification
path: subdir/ # target subdirectory
# Or a list of entries:
source:
- oci: ghcr.io/org/manifests:v1
- file: ./local-overrides
path: patches/
# --- Exclude (optional) --------------------------------------------------
# Glob patterns removed from the working directory after the step runs.
exclude: ["build/**", "*.tmp"]
# --- Step-type config (exactly one, matching `type`) ---------------------
template: # type: template
files:
include: ["**/*.yaml"] # default: ["**/*"]
exclude: ["kustomization.yaml"] # default: []
kustomize-build: # type: kustomize-build
dir: "." # default: "."
enableHelm: false # default: false
outputFile: out.yaml # required
kustomize-create: # type: kustomize-create
dir: "." # default: "."
autodetect: true # at least one of autodetect / resources required
recursive: true # requires autodetect
resources: [] # explicit resource list
namespace: "staging"
nameprefix: "acme-"
namesuffix: "-v2"
annotations: {}
labels: {}
helm: # type: helm
chart: ./charts/my-app # required
releaseName: my-app # required
namespace: default # default: "default"
valuesFiles: ["values.yaml"]
set: { image.tag: "v1.2.3" }
outputFile: helm-output.yaml
split: # type: split
input: multi-doc.yaml # required
by: resource # kind | resource | group | kind-dir | custom
outputDir: manifests/ # default: "."
fileNameTemplate: "" # required when by=custom
canonicalKeyOrder: true # default: true
generate: # type: generate
output: manifests/config.yaml # required
template: | # required, inline Go template
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ .app_name }}
copy: # type: copy
files:
include: ["manifests/**/*.yaml"] # default: ["**/*"]
exclude: []
dest: manifests/ # default: "."| Flag | Description | Default |
|---|---|---|
-input, -input-directory |
Source directory (or remote URI) to process | required |
-output-directory |
Destination for rendered output | required |
-overwrite-output-directory |
Delete and recreate output directory | false |
-context-file |
Global context YAML file (removed from output if inside input) | none |
-max-depth |
Max directory recursion depth (-1 = unlimited, 0 = root only) |
-1 |
-processing |
Single .many.yaml to run (skips directory discovery) |
none |
-instances |
Instances YAML file for matrix mode | none |
-env-file |
Load environment variables from the specified file | none |
-no-sha256-update |
Disable sha256 writeback to .many.yaml files |
false |
-log-level |
debug, info, warn, error |
info |
-logging-type |
json, text, tint |
tint |
-version |
Print version and exit |
When neither -processing nor -instances is given, many walks the input directory
collecting all .many.yaml files, sorts them by depth (parents before children), and
executes each pipeline independently.
many \
-input ./infrastructure \
-output-directory ./output \
-max-depth 2Run one specific .many.yaml (must be within the input directory):
many \
-processing ./infrastructure/cert-manager/.many.yaml \
-input ./infrastructure \
-output-directory ./outputRun the same input tree multiple times with different contexts, producing separate output
directories. Incompatible with -processing.
many \
-input ./services \
-output-directory ./output \
-instances instances.yaml \
-context-file global.yamlInstance file format:
instances:
- name: prod-east
output: prod-east/ # required --- subdirectory of -output-directory
input: "" # optional --- subdirectory of -input (or remote URI)
include: [ api, frontend ] # optional --- filter immediate subdirectories (empty = all)
context: # optional --- merged on top of global context
region: us-east-1
replicas: 3For each instance, many copies the input tree (filtered by include), merges
global + instance context, discovers and runs pipelines, then removes .many.yaml
files. If an instance fails, remaining instances still run. The exit code is non-zero
if any instance failed.
Fetch a remote source directly to a local directory, without running any pipeline:
many pull <ref> <dir>The <ref> supports the same schemes as -input: bare paths, file://, oci://,
https://, and ocm://.
many pull oci://ghcr.io/myorg/manifests:v1 ./local-copy
many pull https://example.com/archive.tar.gz ./extractedEach step has a name (unique within the pipeline) and a type. Steps execute
sequentially.
Every step supports these fields regardless of type:
| Field | Description | Default |
|---|---|---|
name |
Unique identifier within the pipeline | required |
type |
Step type: template, kustomize-build, kustomize-create, helm, split, generate, copy |
required |
source |
Fetch files before the step runs (single entry or list --- see Sources) | none |
exclude |
Glob patterns to remove from the working directory after the step completes | [] |
In addition, each step has a type-specific config block (e.g. template:, split:)
documented below.
Renders files in-place using Go's text/template
with Sprig functions.
- name: render
type: template
template:
files:
include: [ "**/*.yaml" ]
exclude: [ "kustomization.yaml" ]| Field | Description | Default |
|---|---|---|
files.include |
Glob patterns for files to template | ["**/*"] |
files.exclude |
Glob patterns for files to skip | [] |
Globs are relative to the pipeline directory and support ** for recursive matching
via doublestar.
Runs kustomize build and captures the multi-document YAML output. Requires
kustomize on PATH.
- name: build
type: kustomize-build
kustomize-build:
enableHelm: true
outputFile: kustomize-output.yaml| Field | Description | Default |
|---|---|---|
dir |
Directory containing kustomization.yaml |
"." |
enableHelm |
Pass --enable-helm to kustomize |
false |
outputFile |
File to write the build output to | required |
Runs kustomize create to auto-generate a kustomization.yaml file. Useful when
pulling sources from OCI/HTTPS and you want to avoid hand-writing the kustomization
file. Requires kustomize on PATH.
- name: create-kustomization
type: kustomize-create
kustomize-create:
dir: "."
autodetect: true
recursive: true
resources:
- deployment.yaml
- ../base
namespace: "staging"
nameprefix: "acme-"
namesuffix: "-v2"
annotations:
app.kubernetes.io/managed-by: many
labels:
env: production| Field | Description | Default |
|---|---|---|
dir |
Directory in which to create kustomization.yaml |
"." |
autodetect |
Pass --autodetect to discover resources |
false |
recursive |
Pass --recursive (requires autodetect) |
false |
resources |
Explicit list of resources to include | [] |
namespace |
Set namespace in the generated kustomization | none |
nameprefix |
Set name prefix | none |
namesuffix |
Set name suffix | none |
annotations |
Map of annotations to add | {} |
labels |
Map of labels to add | {} |
At least one of autodetect or resources must be set. recursive requires
autodetect to be enabled.
Runs helm template to render a chart. Requires helm on PATH.
- name: render-chart
type: helm
helm:
chart: ./charts/my-app
releaseName: my-app
namespace: default
valuesFiles: [ "values.yaml" ]
set:
image.tag: "v1.2.3"| Field | Description | Default |
|---|---|---|
chart |
Path to chart directory or chart reference | required |
releaseName |
Helm release name | required |
namespace |
Target namespace | "default" |
valuesFiles |
List of values files | [] |
set |
Map of --set overrides |
{} |
outputFile |
File to write the rendered output to | none |
Creates a file from an inline Go template rendered against the pipeline context.
Unlike template (which renders existing files in-place), generate synthesizes
new files purely from context data.
- name: gen-config
type: generate
generate:
output: manifests/config.yaml
template: |
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ .app_name }}
data:
domain: {{ .domain }}| Field | Description | Default |
|---|---|---|
output |
Output file path relative to the pipeline directory | required |
template |
Inline Go template string | required |
Parent directories are created automatically.
Copies files from the original source/input directory into the pipeline working directory. Useful for pulling in static manifests, CRs, or other files that don't need templating.
- name: copy-cr-files
type: copy
copy:
files:
include: ["manifests/**/*.yaml"]
exclude: ["manifests/tmp/**"]
dest: manifests/| Field | Description | Default |
|---|---|---|
files.include |
Glob patterns for files to copy from the source directory | ["**/*"] |
files.exclude |
Glob patterns for files to skip | [] |
dest |
Destination subdirectory within the working directory | "." |
Files are copied preserving their relative directory structure. Globs use
doublestar syntax (** for recursive
matching).
Takes a multi-document YAML file and splits it into individual files.
- name: split-manifests
type: split
split:
input: kustomize-output.yaml
by: resource
outputDir: manifests/| Field | Description | Default |
|---|---|---|
input |
File path to the multi-document YAML to split | required |
by |
Splitting strategy (see below) | "kind" |
outputDir |
Directory to write split files into | "." |
fileNameTemplate |
Go template for file paths (only with custom strategy) |
--- |
canonicalKeyOrder |
Reorder keys: apiVersion, kind, metadata first | true |
Splitting strategies:
| Strategy | Layout |
|---|---|
kind |
One file per Kind (deployment.yaml, service.yaml). Multiple resources of the same Kind share a file. |
resource |
One file per resource (deployment-api.yaml, service-api.yaml). |
group |
Directories per API group (apps/deployment-api.yaml, core/service-api.yaml). |
kind-dir |
Directories per Kind, pluralized (deployments/api.yaml, services/api.yaml). |
custom |
File paths from a Go template: `fileNameTemplate: "{{ .metadata.namespace }}/{{ .kind |
Each step can declare a source to fetch files into the working directory before
execution. A source is a single entry or a list of entries, applied in order:
source:
https: https://example.com/archive.tar.gz
sha256: "abc123..."
path: subdir/source:
- oci: ghcr.io/myorg/manifests:v1.0.0
- file: ./local-overrides
path: patches/| Scheme | Description | Example |
|---|---|---|
file |
Local file or directory (relative to .many.yaml) |
file: ../shared/postgres.yaml |
https |
URL to a single file or tarball (auto-extracted) | https: https://example.com/v1.tar.gz |
oci |
OCI image reference | oci: ghcr.io/myorg/config:v1 |
ocm |
OCM component version | ocm: github.com/myorg/component//res |
helm |
Helm chart (requires repo) |
helm: my-chart |
| Option | Description |
|---|---|
path |
Target subdirectory to place fetched files into |
sha256 |
SHA-256 checksum for https sources (see below) |
recursive |
Recursively resolve OCM references (OCM only) |
repo |
Helm chart repository URL (Helm only) |
version |
Helm chart version (Helm only) |
SHA-256 verification --- when sha256 is set to a hex digest, many verifies
the downloaded content matches before proceeding. On the first run you can leave
it empty (sha256: "") to disable verification; many computes the actual
checksum and writes it back to the .many.yaml file so you can pin it. Use
-no-sha256-update to disable this writeback. Once pinned, any mismatch
(e.g. an upstream release change) fails the pipeline. Combined with a
# renovate: comment, Renovate updates both the URL and the checksum
automatically.
Each .many.yaml can define a context block. These values are available as
template data in all template and generate steps:
context:
domain: "example.com"
replicas: 3
pipeline:
- name: render
type: templateTemplates reference values with {{ .domain }}, {{ .replicas }}, etc.
A global context file provided via -context-file applies to all pipelines:
many -input ./infra -output-directory ./output -context-file global.yamlContext is merged in layers (later layers override earlier ones, deep-merged for nested maps):
-context-file(global)- Instance
context(instances mode only) .many.yamlcontext(pipeline-local)
# global.yaml
domain: "default.example.com"
database:
host: "db.default"
# .many.yaml --- domain overridden, database.host inherited, database.port added
context:
domain: "app.example.com"
database:
port: 5432After merging, all string values are rendered as Go templates against the full context (single pass). This lets context values reference other context values:
domain: "example.com"
forgejo_url: "https://forgejo.{{ .domain }}"
smtp_from: "noreply@{{ .domain }}"After interpolation, forgejo_url becomes https://forgejo.example.com.
Sprig functions are available
(e.g. {{ .name | upper }}). Non-string values (ints, bools) are left unchanged.
- The source tree is copied to the output directory.
.many.yamlfiles are discovered, sorted by directory depth (parents first).- Each pipeline executes in-place within the output tree.
templatesteps modify files in-place.kustomize-build/helmsteps write their output to a file (outputFile).splitsteps read a multi-document YAML file and write individual files..many.yamlfiles and the context file are removed from the output.
A failing step aborts its pipeline. Other pipelines continue. The exit code is non-zero if any pipeline failed.
Use -env-file to load environment variables from a file
via godotenv:
many -env-file .env -input ./infra -output-directory ./outputLoaded variables are available to kustomize-build, kustomize-create, and
helm steps via environment inheritance.