Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 11 additions & 10 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,11 +28,12 @@ Read the [README](README.md) for an overview of the project and its features.

- `cmd/flux-mirror/` - the `main` package. One file per cobra command or command concern (`sync.go`, `login.go`, `create_secret.go`, `keygen.go`, `version.go`, `completion.go`, `progress.go`, `logging.go`). `main.VERSION` is overridden at build time by the Makefile.
- `cmd/flux-mirror/main_test.go` - hosts `TestMain`, shared `executeCommand(...)` helpers, and `resetCmdArgs()`, which restores global cobra flag state between tests. New commands or flags must update this reset path so tests do not leak state across subtests.
- `api/v1beta1/` - the `mirror.plugin.fluxcd.io/v1beta1` API types: the `Config` wire model (`hosts`, `artifacts`, `charts`, selector, verification) and the sync `Report` envelope, annotated with kubebuilder markers. `zz_generated.deepcopy.go` and the published JSON Schemas are generated from these types; the package stays dependency-light (apimachinery + stdlib) and carries only the wire structs, enums, consts, and `Effective*` helpers — no semver/OCI/SPIFFE deps.
- `internal/config/` - decoding (`Decode`), semantic validation (`Validate`, `ValidateNoEntriesOK`), and path resolution (`ResolvePaths`) for `api/v1beta1.Config` (`apiVersion: mirror.plugin.fluxcd.io/v1beta1`, `kind: Config`). Validation is free functions over the API types, not methods.
- `tools/schema-gen/` - generator that turns controller-gen CRD output into a standalone draft 2020-12 JSON Schema. Run via `make generate`; do not edit `docs/{config,report}/*-v1beta1.json` by hand.
- `api/v1beta2/` - the `mirror.plugin.fluxcd.io/v1beta2` API types: the `Config` wire model (`hosts`, `artifacts`, `charts`, selector, verification) and the sync `Report` envelope, annotated with kubebuilder markers. `zz_generated.deepcopy.go` and the published JSON Schemas are generated from these types; the package stays dependency-light (apimachinery + stdlib) and carries only the wire structs, enums, consts, and `Effective*` helpers — no semver/OCI/SPIFFE deps. `convert.go` maps a deprecated `api/v1beta1.Config` onto the current shape.
- `api/v1beta1/` - the deprecated `mirror.plugin.fluxcd.io/v1beta1` wire types, kept so old configs still decode. Frozen: no new fields or behavior, only the conversion in `api/v1beta2/convert.go` reads it.
- `internal/config/` - decoding (`Decode`), semantic validation (`Validate`, `ValidateNoEntriesOK`), and path resolution (`ResolvePaths`) for `api/v1beta2.Config` (`apiVersion: mirror.plugin.fluxcd.io/v1beta2`, `kind: Config`). A v1beta1 document is decoded with `api/v1beta1` and converted to v1beta2 in memory, emitting a migration warning (`DeprecationWarning`); validation requires v1beta2. Validation is free functions over the API types, not methods.
- `tools/schema-gen/` - generator that turns controller-gen CRD output into a standalone draft 2020-12 JSON Schema. Run via `make generate`; do not edit `docs/{config,report}/*-v1beta2.json` by hand.
- `internal/selector/` - tag selection pipeline for OCI artifacts: regex prefilter, semver filter, sort strategy (`semver`, `alphabetical`, `numerical`), then top-N limit.
- `internal/sync/` - sync runner, retry/timeout behavior, per-entry execution, summaries, outcomes, and exit-code aggregation. Report assembly (`NewReport`, `RenderReport`) builds the `api/v1beta1.Report` envelope; the wire report types live in `api/v1beta1`.
- `internal/sync/` - sync runner, retry/timeout behavior, per-entry execution, summaries, outcomes, and exit-code aggregation. Report assembly (`NewReport`, `RenderReport`) builds the `api/v1beta2.Report` envelope; the wire report types live in `api/v1beta2`.
- `internal/artifacts/` - OCI artifact mirroring from source repository tags to destination repository tags, including drift handling, dry-run outcomes, referrers, verification, and concurrency fan-out.
- `internal/charts/` - Helm chart mirroring to OCI destinations, including version selection, deterministic Helm-OCI publication, drift handling, and dry-run outcomes.
- `internal/oci/` - OCI client wrapper, auth/keychain setup, transport customization, digest checks, blob copy, Helm artifact helpers, referrers, and cosign verification.
Expand All @@ -49,7 +50,7 @@ All development goes through the Makefile - do not invoke `go build` directly, b
- `make build` - build `./bin/flux-mirror` with VERSION stamped from git
- `make test` - runs `tidy`, `fmt`, `generate`, `vet`, then `go test ./... -coverprofile cover.out`
- Single test pattern: `make test GO_TEST_ARGS="-run TestVersionCmd"`
- `make generate` - download `controller-gen` if needed, regenerate `api/v1beta1/zz_generated.deepcopy.go`, and regenerate the config/report JSON Schemas under `docs/`. Run after changing any `api/v1beta1` type or marker; CI fails if the tree is left dirty.
- `make generate` - download `controller-gen` if needed, regenerate `api/v1beta2/zz_generated.deepcopy.go`, and regenerate the config/report JSON Schemas under `docs/`. Run after changing any `api/v1beta2` type or marker; CI fails if the tree is left dirty.
- `make lint` - runs golangci-lint with revive, staticcheck, goimports, errcheck, misspell, and related checks
- `make run GO_RUN_ARGS="version -o json"` - build then run the CLI with args
- `make docker-build` - build the container image
Expand All @@ -67,16 +68,16 @@ CI (`.github/workflows/test.yaml`) runs `make test`, `make lint`, `make build`,
- Cobra commands keep flag state in package-level `*Args` variables. When adding a command or flag, update `resetCmdArgs()` in `cmd/flux-mirror/main_test.go`.
- Command output should go through cobra command streams (`cmd.Printf`, `cmd.PrintErrf`, `cmd.OutOrStdout()`, `cmd.ErrOrStderr()`) rather than writing to process-wide stdout/stderr directly. Tests rely on command-local streams.
- Error handling should wrap context with `%w` and preserve the distinct sync exit-code behavior (`0` clean, `1` failed jobs, `2` drift-only by default).
- Config and report wire types live in `api/v1beta1`; semantic config validation lives in `internal/config`. Do not defer schema or semantic validation to mirror execution if it can be rejected before network operations. After changing an `api/v1beta1` type or kubebuilder marker, run `make generate` and commit the regenerated deepcopy and JSON Schemas.
- Config and report wire types live in `api/v1beta2`; semantic config validation lives in `internal/config`. Do not defer schema or semantic validation to mirror execution if it can be rejected before network operations. After changing an `api/v1beta2` type or kubebuilder marker, run `make generate` and commit the regenerated deepcopy and JSON Schemas.
- Tests use Gomega (`. "github.com/onsi/gomega"` dot-import is accepted - staticcheck ST1001 is disabled project-wide). Table-driven tests are the norm.

## Writing Documentation

User-facing changes (flags, commands, config fields, report shape, GitHub Action inputs) must be reflected in the docs. The tree is:

- `README.md` - features list, install, quickstart, CI usage, GitHub Action example, and doc links.
- `docs/config.md` + `docs/config-v1beta1.json` - the single config doc: YAML specification (hosts, artifacts, charts, selectors, overwrite behavior, signature verification, defaults), envelope reference, and generated JSON Schema.
- `docs/report.md` + `docs/report-v1beta1.json` - sync report envelope reference and its generated JSON Schema.
- `docs/config.md` + `docs/config-v1beta2.json` - the single config doc: YAML specification (hosts, artifacts, charts, selectors, overwrite behavior, signature verification, defaults), envelope reference, and generated JSON Schema. The frozen `docs/config-v1beta1.json` remains published for existing configs.
- `docs/report.md` + `docs/report-v1beta2.json` - sync report envelope reference and its generated JSON Schema. The frozen `docs/report-v1beta1.json` remains published for existing reports.
- `docs/sync.md` - `sync` command reference, config source resolution, authentication, flags, output, outcomes, exit codes, and dry-run behavior.
- `docs/login.md`, `docs/secret.md`, `docs/keygen.md` - auth login, Kubernetes Secret generation, and JWK key generation command references.
- `actions/setup/README.md` + `actions/setup/action.yaml` - GitHub Action inputs and example workflows.
Expand All @@ -85,8 +86,8 @@ User-facing changes (flags, commands, config fields, report shape, GitHub Action
Apply these rules:

- New or changed CLI flag: update the flag table in `docs/sync.md` and any relevant examples in `README.md`.
- New or changed config field: add it to the `api/v1beta1.Config` types (with kubebuilder markers), run `make generate`, then update `internal/config` validation/tests, `docs/config.md`, and examples when appropriate.
- New or changed report field: add it to the `api/v1beta1` report types, run `make generate`, then update `internal/sync` assembly/tests and `docs/report.md`.
- New or changed config field: add it to the `api/v1beta2.Config` types (with kubebuilder markers), run `make generate`, then update `internal/config` validation/tests, `docs/config.md`, and examples when appropriate.
- New or changed report field: add it to the `api/v1beta2` report types, run `make generate`, then update `internal/sync` assembly/tests and `docs/report.md`.
- New command: add or update README command guidance and create a dedicated reference doc under `docs/` if the command has meaningful flags or output.
- Auth command changes: update `docs/login.md`, `docs/secret.md`, or `docs/keygen.md` as applicable.
- Output or outcome shape change: update `docs/sync.md`, README examples, and tests that assert text/YAML/JSON output.
Expand Down
21 changes: 21 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -80,13 +80,34 @@ generate-json-schemas: generate-api ## Generate the config and report JSON Schem
-schema-field \
-id "https://raw.githubusercontent.com/fluxcd/flux-mirror/main/docs/report-v1beta1.json" \
-out ./docs/report-v1beta1.json
go run ./tools/schema-gen \
-controller-gen "$(CONTROLLER_GEN)" \
-group "mirror.plugin.fluxcd.io" \
-version "v1beta2" \
-kind "Config" \
-type "github.com/fluxcd/flux-mirror/api/v1beta2.Config" \
-title "Config is the flux-mirror configuration file." \
-id "https://raw.githubusercontent.com/fluxcd/flux-mirror/main/docs/config-v1beta2.json" \
-out ./docs/config-v1beta2.json
go run ./tools/schema-gen \
-controller-gen "$(CONTROLLER_GEN)" \
-group "mirror.plugin.fluxcd.io" \
-version "v1beta2" \
-kind "Report" \
-type "github.com/fluxcd/flux-mirror/api/v1beta2.ReportSpec" \
-field "report" \
-schema-field \
-id "https://raw.githubusercontent.com/fluxcd/flux-mirror/main/docs/report-v1beta2.json" \
-out ./docs/report-v1beta2.json

.PHONY: package-schemas
package-schemas: ## Package JSON Schemas for release assets.
rm -rf $(RELEASE_DIR)/schemas
mkdir -p $(RELEASE_DIR)/schemas
cp ./docs/config-v1beta1.json $(RELEASE_DIR)/schemas/
cp ./docs/report-v1beta1.json $(RELEASE_DIR)/schemas/
cp ./docs/config-v1beta2.json $(RELEASE_DIR)/schemas/
cp ./docs/report-v1beta2.json $(RELEASE_DIR)/schemas/
tar -czf $(RELEASE_DIR)/schemas.tar.gz -C $(RELEASE_DIR) schemas

.PHONY: docker-build
Expand Down
8 changes: 4 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ on upstream chart repositories.
- **Registry auth** — OCI auth supports Docker config and credential helpers,
cloud workload identity for ECR/ACR/GAR, and per-host bearer credentials from
GitHub/Forgejo OIDC, GCP, Azure, AWS STS, SPIFFE JWT-SVID, env vars, files, or
JWK-signed JWTs. Per-host TLS/mTLS (custom CA, client cert, or SPIFFE X.509-SVID)
JWK-signed JWTs. Per-host TLS/mTLS (a certificate directory, inline PEMs, or SPIFFE X.509-SVID)
is also supported. Helm HTTP/S credentials come from Helm's repositories config.
- **Structured output** — `text` and `yaml`/`json` for downstream
tooling, plus a verbose mode that streams every blob and manifest digest
Expand Down Expand Up @@ -72,7 +72,7 @@ Write a config file describing what to mirror:

```yaml
# flux-mirror.yaml
apiVersion: mirror.plugin.fluxcd.io/v1beta1
apiVersion: mirror.plugin.fluxcd.io/v1beta2
kind: Config
charts:
- name: external-dns
Expand Down Expand Up @@ -217,9 +217,9 @@ Run `flux mirror <command> --help` for the full flag list.
exit codes, and example invocations.
- [Config specification](docs/config.md) — YAML schema for `hosts`,
`artifacts`, and `charts` entries, selector pipeline, overwrite semantics,
defaults, and the published [JSON Schema](docs/config-v1beta1.json).
defaults, and the published [JSON Schema](docs/config-v1beta2.json).
- [Sync report reference](docs/report.md) — report envelope and its
published [JSON Schema](docs/report-v1beta1.json) for `-o json` / `-o yaml`.
published [JSON Schema](docs/report-v1beta2.json) for `-o json` / `-o yaml`.
- [Keygen command reference](docs/keygen.md) — generate EdDSA JWK pairs for
JWK-based registry auth.
- [Login command reference](docs/login.md) — store configured credentials in
Expand Down
Loading
Loading