diff --git a/README.md b/README.md index 839f34292c..152ea0e5bd 100644 --- a/README.md +++ b/README.md @@ -1,11 +1,14 @@ -# Rancher Image Mirror +# Rancher Artifact Mirror -This repo is dedicated to mirroring images from other organizations to the +This repo is dedicated to mirroring artifacts from other organizations to the places they need to be for Rancher and its associated projects to use them. +> [!NOTE] +> This repository used to be called `image-mirror`; It was renamed to better reflect its usecases, see [this](https://github.com/rancher/rancher/issues/52259) issue for more information + ## Overview -`regsync` is used for mirroring images from one repository to another. `regsync` +`regsync` is used for mirroring OCI artifacts from one repository to another. `regsync` is configured via `regsync.yaml`. `regsync.yaml` is generated by running the `generate-regsync` subcommand of the Go code in `tools/`, using `config.yaml` as input. You can build the latest version of this code to `bin/artifact-mirror-tools` @@ -18,14 +21,14 @@ Once `regsync.yaml` has been updated, you may run `regsync` via the command ``` regsync once --verbosity error --config regsync.yaml --missing ``` -`autoupdate.yaml` is used to configure automatic updates for images. When +`autoupdate.yaml` is used to configure automatic updates for artifacts. When an update is found, the automation creates a pull request that a human user can then review and merge. See [`autoupdate.yaml`](#autoupdateyaml) for more information. ### Adding New Artifacts -When adding new artifacts to the repo, please indicate so in the pull request. +When adding new OCI artifacts to the repo, please indicate so in the pull request. You will need to submit a request to EIO, who will create the repo in DockerHub. If this is not done, mirroring the artifact will fail. Nothing special needs to be done for mirroring a new artifact to the Rancher Prime @@ -78,7 +81,7 @@ This section roughly correlates to the `creds` section of `regsync.yaml`. | `Password` | yes | The password to use when authenticating against the registry. See [the regsync documentation](https://regclient.org/usage/regsync/) for more details. | `Registry` | yes | The registry URL. See [the regsync documentation](https://regclient.org/usage/regsync/) for more details. | `ReqConcurrent` | no | The number of concurrent requests that are made to this registry. See [the regsync documentation](https://regclient.org/usage/regsync/) for more details. -| `DefaultTarget` | no | Whether the Repository is used as a target repository for a given artifact when the `TargetRepositories` field of the artifact is not set. +| `DefaultTarget` | no | Whether the Repository is used as a target repository for a given artifact when the `TargetRepositories` field of the `Artifact` is not set. | `Username` | yes | The username to use when authenticating against the registry. See [the regsync documentation](https://regclient.org/usage/regsync/) for more details. #### `Artifacts` @@ -88,17 +91,17 @@ repository. | Field | Required | Description | |----------------------| ------------- |------------- | -| `DoNotMirror` | no | Set to `true` to exclude the entire artifact from regsync.yaml. Alternatively, set to an array of strings to specify tags to exclude from regsync.yaml. +| `DoNotMirror` | no | Set to `true` to exclude the entire `Artifact` from regsync.yaml. Alternatively, set to an array of strings to specify tags to exclude from regsync.yaml. | `SourceArtifact` | yes | The source artifact. If there is no host, the artifact is assumed to be from Docker Hub. | `Tags` | yes | The tags to mirror. | `TargetArtifactName` | no | By default, the target artifact name is derived from the source artifact, and is of the format `mirrored--`. For example, `banzaicloud/logging-operator` becomes `mirrored-banzaicloud-logging-operator`. However, there are some artifacts that do not follow this convention - this field exists for these cases. New artifacts should not set this field. -| `TargetRepositories` | no | Repositories to mirror the artifact to. Repositories are specified via their `BaseUrl` field. If not specified, the artifact is mirrored to all Repositories that have `DefaultTarget` set to true. +| `TargetRepositories` | no | Repositories to mirror the artifact to. Repositories are specified via their `BaseUrl` field. If not specified, the `Artifact` is mirrored to all Repositories that have `DefaultTarget` set to true. ### `autoupdate.yaml` -`autoupdate.yaml` defines configuration for automatically updating image tags +`autoupdate.yaml` defines configuration for automatically updating artifact tags based on various update strategies that monitor sources for new tags. Each -entry specifies a strategy for finding tags of images to potentially add to +entry specifies a strategy for finding tags of artifacts to potentially add to `config.yaml`, which are then submitted as pull requests. | Field | Required | Description | @@ -112,44 +115,44 @@ entry specifies a strategy for finding tags of images to potentially add to #### `GithubRelease` The `GithubRelease` strategy fetches all release tags that matches the VersionConstraint from a GitHub -repository and applies it to the specified images. +repository and applies it to the specified artifacts. If LatestOnly is true, it only fetches from the latest release and does not consider the VersionConstraint. | Field | Required | Description | |---------------------|----------|------------- | | `Owner` | yes | The GitHub repository owner/organization. | `Repository` | yes | The GitHub repository name. -| `Images` | yes | See [`Images`](#images). +| `Artifacts` | yes | See [`Artifacts`](#Artifacts). | `LatestOnly` | no | If true, get only the tag from the latest github release. | `VersionConstraint` | no | A SemVer constraint used to filter the github releases. | `VersionRegex` | no | If specified, only matching release tags will be considered. If a capture group is present, only its contents will be passed on. -##### `Images` +##### `Artifacts` -A list of images to be updated with the latest release tag. Each image will get the same tag as the GitHub release. +A list of artifacts to be updated with the latest release tag. Each artifact will get the same tag as the GitHub release. -| Field | Required | Description | -| ------------- | ------------- |------------- | -| `SourceImage` | yes | The GitHub repository name. -| `TargetImageName` | no | The TargetImageName of the image in `config.yaml` that you want to update. +| Field | Required | Description | +|----------------------| ------------- |------------- | +| `SourceArtifact` | yes | The GitHub repository name. +| `TargetArtifactName` | no | The TargetArtifactName of the artifact in `config.yaml` that you want to update. #### `HelmLatest` The `HelmLatest` strategy templates out the latest version of configured -Helm charts and extracts image references from the rendered manifests. It +Helm charts and extracts artifact references from the rendered manifests. It recursively searches for fields with an "image" key in the templated YAML output. -| Field | Required | Description | -| ------------- | ------------- |------------- | -| `HelmRepo` | yes | The URL of the Helm chart repository. -| `Charts` | yes | A map where keys are the charts to template, and values are another map from environment name to lists of helm values to `--set` in that environment. `helm template` is run once for each environment. -| `Images` | no | Used to map a given update image to an entry in `config.yaml`. There may be multiple entries that have the same `SourceImage`, but different `TargetImageName`s, so we need to choose which one receives the update image. +| Field | Required | Description | +|-----------------| ------------- |------------- | +| `HelmRepo` | yes | The URL of the Helm chart repository. +| `Charts` | yes | A map where keys are the charts to template, and values are another map from environment name to lists of helm values to `--set` in that environment. `helm template` is run once for each environment. +| `Artifacts` | no | Used to map a given update artifact to an entry in `config.yaml`. There may be multiple entries that have the same `SourceArtifact`, but different `TargetArtifactName`s, so we need to choose which one receives the update artifact. | `ImageDenylist` | no | A list of images to exclude from the results. #### `Registry` -The `Registry` strategy fetches all image tags that matches the `VersionFilter` from a registry defined in the Images provided. +The `Registry` strategy fetches all artifact tags that matches the `VersionFilter` from a registry defined in the `Artifacts` provided. Supported registries are: * Suse Container Registry (registry.suse.com) * Docker Hub @@ -160,6 +163,6 @@ Supported registries are: | Field | Required | Description | |-----------------|----------|------------- | -| `Images` | yes | Used to map a given update image to an entry in `config.yaml`. There may be multiple entries that have the same `SourceImage`, but different `TargetImageName`s, so we need to choose which one receives the update image. +| `Artifacts` | yes | Used to map a given update artifact to an entry in `config.yaml`. There may be multiple entries that have the same `SourceArtifact`, but different `TargetArtifactName`s, so we need to choose which one receives the update artifact. | `Latest` | no | A flag to only use the latest tag. This only works if all tags are in semver format. -| `VersionFilter` | no | A regex to match against the image tags fetched from the registry. +| `VersionFilter` | no | A regex to match against the artifact tags fetched from the registry. diff --git a/autoupdate.yaml b/autoupdate.yaml index 302ac2e4ae..44761e4669 100644 --- a/autoupdate.yaml +++ b/autoupdate.yaml @@ -1,20 +1,20 @@ - GithubRelease: - Images: - - SourceImage: quay.io/calico/apiserver - - SourceImage: quay.io/calico/cni - - SourceImage: quay.io/calico/csi - - SourceImage: quay.io/calico/ctl - - SourceImage: quay.io/calico/envoy-gateway - - SourceImage: quay.io/calico/envoy-proxy - - SourceImage: quay.io/calico/envoy-ratelimit - - SourceImage: quay.io/calico/goldmane - - SourceImage: quay.io/calico/kube-controllers - - SourceImage: quay.io/calico/node - - SourceImage: quay.io/calico/node-driver-registrar - - SourceImage: quay.io/calico/pod2daemon-flexvol - - SourceImage: quay.io/calico/typha - - SourceImage: quay.io/calico/whisker - - SourceImage: quay.io/calico/whisker-backend + Artifacts: + - SourceArtifact: quay.io/calico/apiserver + - SourceArtifact: quay.io/calico/cni + - SourceArtifact: quay.io/calico/csi + - SourceArtifact: quay.io/calico/ctl + - SourceArtifact: quay.io/calico/envoy-gateway + - SourceArtifact: quay.io/calico/envoy-proxy + - SourceArtifact: quay.io/calico/envoy-ratelimit + - SourceArtifact: quay.io/calico/goldmane + - SourceArtifact: quay.io/calico/kube-controllers + - SourceArtifact: quay.io/calico/node + - SourceArtifact: quay.io/calico/node-driver-registrar + - SourceArtifact: quay.io/calico/pod2daemon-flexvol + - SourceArtifact: quay.io/calico/typha + - SourceArtifact: quay.io/calico/whisker + - SourceArtifact: quay.io/calico/whisker-backend LatestOnly: true Owner: projectcalico Repository: calico @@ -22,17 +22,28 @@ Reviewers: - rancher/k3s-networking - HelmLatest: + Artifacts: + - SourceArtifact: quay.io/tigera/operator + TargetArtifactName: mirrored-calico-operator Charts: tigera-operator: default: [] HelmRepo: https://docs.tigera.io/calico/charts - Images: - - SourceImage: quay.io/tigera/operator - TargetImageName: mirrored-calico-operator Name: calico-operator Reviewers: - rancher/k3s-networking - HelmLatest: + Artifacts: + - SourceArtifact: quay.io/cilium/certgen + - SourceArtifact: quay.io/cilium/cilium + - SourceArtifact: quay.io/cilium/cilium-envoy + - SourceArtifact: quay.io/cilium/operator-generic + - SourceArtifact: quay.io/cilium/clustermesh-apiserver + - SourceArtifact: quay.io/cilium/hubble-ui + - SourceArtifact: quay.io/cilium/operator-aws + - SourceArtifact: quay.io/cilium/operator-azure + - SourceArtifact: quay.io/cilium/hubble-relay + - SourceArtifact: quay.io/cilium/hubble-ui-backend Charts: cilium: aws: @@ -51,23 +62,12 @@ - quay.io/cilium/startup-script - ghcr.io/spiffe/spire-agent - ghcr.io/spiffe/spire-server - Images: - - SourceImage: quay.io/cilium/certgen - - SourceImage: quay.io/cilium/cilium - - SourceImage: quay.io/cilium/cilium-envoy - - SourceImage: quay.io/cilium/operator-generic - - SourceImage: quay.io/cilium/clustermesh-apiserver - - SourceImage: quay.io/cilium/hubble-ui - - SourceImage: quay.io/cilium/operator-aws - - SourceImage: quay.io/cilium/operator-azure - - SourceImage: quay.io/cilium/hubble-relay - - SourceImage: quay.io/cilium/hubble-ui-backend Name: cilium Reviewers: - rancher/k3s-networking - GithubRelease: - Images: - - SourceImage: coredns/coredns + Artifacts: + - SourceArtifact: coredns/coredns LatestOnly: true Owner: coredns Repository: coredns @@ -77,14 +77,14 @@ - rancher/k3s-networking - Name: cpi-release-manager Registry: - Images: - - SourceImage: registry.k8s.io/cloud-pv-vsphere/cloud-provider-vsphere + Artifacts: + - SourceArtifact: registry.k8s.io/cloud-pv-vsphere/cloud-provider-vsphere VersionFilter: ^v1.([3-9][0-9]).[0-9]+$ Reviewers: - rancher/k3s - GithubRelease: - Images: - - SourceImage: registry.k8s.io/sig-storage/csi-attacher + Artifacts: + - SourceArtifact: registry.k8s.io/sig-storage/csi-attacher LatestOnly: true Owner: kubernetes-csi Repository: external-attacher @@ -92,8 +92,8 @@ Reviewers: - rancher/k3s - GithubRelease: - Images: - - SourceImage: registry.k8s.io/sig-storage/csi-node-driver-registrar + Artifacts: + - SourceArtifact: registry.k8s.io/sig-storage/csi-node-driver-registrar LatestOnly: true Owner: kubernetes-csi Repository: node-driver-registrar @@ -101,8 +101,8 @@ Reviewers: - rancher/k3s - GithubRelease: - Images: - - SourceImage: registry.k8s.io/sig-storage/csi-provisioner + Artifacts: + - SourceArtifact: registry.k8s.io/sig-storage/csi-provisioner LatestOnly: true Owner: kubernetes-csi Repository: external-provisioner @@ -110,8 +110,8 @@ Reviewers: - rancher/k3s - GithubRelease: - Images: - - SourceImage: registry.k8s.io/sig-storage/csi-resizer + Artifacts: + - SourceArtifact: registry.k8s.io/sig-storage/csi-resizer LatestOnly: true Owner: kubernetes-csi Repository: external-resizer @@ -119,8 +119,8 @@ Reviewers: - rancher/k3s - GithubRelease: - Images: - - SourceImage: registry.k8s.io/sig-storage/csi-snapshotter + Artifacts: + - SourceArtifact: registry.k8s.io/sig-storage/csi-snapshotter LatestOnly: true Owner: kubernetes-csi Repository: external-snapshotter @@ -128,8 +128,8 @@ Reviewers: - rancher/k3s - GithubRelease: - Images: - - SourceImage: quay.io/coreos/etcd + Artifacts: + - SourceArtifact: quay.io/coreos/etcd Owner: etcd-io Repository: etcd VersionConstraint: '>3.5.10' @@ -137,8 +137,8 @@ Reviewers: - rancher/k3s - GithubRelease: - Images: - - SourceImage: flannel/flannel + Artifacts: + - SourceArtifact: flannel/flannel LatestOnly: true Owner: flannel-io Repository: flannel @@ -146,8 +146,8 @@ Reviewers: - rancher/k3s-networking - GithubRelease: - Images: - - SourceImage: flannel/flannel-cni-plugin + Artifacts: + - SourceArtifact: flannel/flannel-cni-plugin LatestOnly: true Owner: flannel-io Repository: cni-plugin @@ -155,8 +155,8 @@ Reviewers: - rancher/k3s-networking - GithubRelease: - Images: - - SourceImage: ghcr.io/kube-vip/kube-vip-iptables + Artifacts: + - SourceArtifact: ghcr.io/kube-vip/kube-vip-iptables LatestOnly: true Owner: kube-vip Repository: kube-vip @@ -165,15 +165,15 @@ - rancher/k3s-networking - Name: kube-webhook-certgen Registry: - Images: - - SourceImage: registry.k8s.io/ingress-nginx/kube-webhook-certgen + Artifacts: + - SourceArtifact: registry.k8s.io/ingress-nginx/kube-webhook-certgen VersionFilter: ^v1.[0-9]+.[0-9]+$ Reviewers: - rancher/k3s-networking - GithubRelease: - Images: - - SourceImage: registry.k8s.io/metrics-server/metrics-server - TargetImageName: mirrored-metrics-server + Artifacts: + - SourceArtifact: registry.k8s.io/metrics-server/metrics-server + TargetArtifactName: mirrored-metrics-server LatestOnly: true Owner: kubernetes-sigs Repository: metrics-server @@ -181,8 +181,8 @@ Reviewers: - rancher/k3s - GithubRelease: - Images: - - SourceImage: idealista/prom2teams + Artifacts: + - SourceArtifact: idealista/prom2teams LatestOnly: true Owner: idealista Repository: prom2teams @@ -190,8 +190,8 @@ Reviewers: - rancher/observation-backup - GithubRelease: - Images: - - SourceImage: registry.k8s.io/sig-storage/snapshot-controller + Artifacts: + - SourceArtifact: registry.k8s.io/sig-storage/snapshot-controller LatestOnly: true Owner: kubernetes-csi Repository: external-snapshotter @@ -199,8 +199,8 @@ Reviewers: - rancher/k3s - GithubRelease: - Images: - - SourceImage: sonobuoy/sonobuoy + Artifacts: + - SourceArtifact: sonobuoy/sonobuoy LatestOnly: true Owner: vmware-tanzu Repository: sonobuoy @@ -208,8 +208,8 @@ Reviewers: - rancher/infracloud-team - GithubRelease: - Images: - - SourceImage: registry.k8s.io/sig-storage/livenessprobe + Artifacts: + - SourceArtifact: registry.k8s.io/sig-storage/livenessprobe LatestOnly: true Owner: kubernetes-csi Repository: livenessprobe @@ -218,26 +218,26 @@ - rancher/k3s - Name: traefik Registry: - Images: - - SourceImage: library/traefik + Artifacts: + - SourceArtifact: library/traefik Latest: true VersionFilter: ^2.[0-9]+.[0-9]+$ Reviewers: - rancher/k3s - Name: traefik3 Registry: - Images: - - SourceImage: library/traefik + Artifacts: + - SourceArtifact: library/traefik Latest: true VersionFilter: ^3.[0-9]+.[0-9]+$ Reviewers: - rancher/k3s - GithubRelease: - Images: - - SourceImage: registry.k8s.io/csi-vsphere/driver - TargetImageName: mirrored-cloud-provider-vsphere-csi-release-driver - - SourceImage: registry.k8s.io/csi-vsphere/syncer - TargetImageName: mirrored-cloud-provider-vsphere-csi-release-syncer + Artifacts: + - SourceArtifact: registry.k8s.io/csi-vsphere/driver + TargetArtifactName: mirrored-cloud-provider-vsphere-csi-release-driver + - SourceArtifact: registry.k8s.io/csi-vsphere/syncer + TargetArtifactName: mirrored-cloud-provider-vsphere-csi-release-syncer LatestOnly: true Owner: kubernetes-sigs Repository: vsphere-csi-driver diff --git a/tools/internal/autoupdate/config.go b/tools/internal/autoupdate/config.go index cf74c3b98a..d026a8d34e 100644 --- a/tools/internal/autoupdate/config.go +++ b/tools/internal/autoupdate/config.go @@ -38,12 +38,12 @@ type AutoUpdateOptions struct { GithubClient *github.Client } -// AutoupdateImageRef is used to map a given update image to an entry in config.yaml. -// There may be multiple entries that have the same SourceImage, but different -// TargetImageNames, so we need to choose which one receives the update image. -type AutoupdateImageRef struct { - SourceImage string - TargetImageName string `json:",omitempty"` +// AutoupdateArtifactRef is used to map a given update artifact to an entry in config.yaml. +// There may be multiple entries that have the same SourceArtifact, but different +// TargetArtifactName, so we need to choose which one receives the update artifact. +type AutoupdateArtifactRef struct { + SourceArtifact string + TargetArtifactName string `json:",omitempty"` } func Parse(filePath string) ([]ConfigEntry, error) { @@ -135,53 +135,53 @@ func (entry ConfigEntry) Validate() error { return nil } -// GetUpdateImages returns a slice of Artifacts that depends on the +// GetUpdateArtifacts returns a slice of Artifacts that depends on the // configured update strategy. The returned Artifacts may be from // any source, and they may be gathered in any way. The intention // is that they are new Artifacts (or new tags of existing Artifacts) that // we want to mirror. -func (entry ConfigEntry) GetUpdateImages() ([]*config.Artifact, error) { +func (entry ConfigEntry) GetUpdateArtifacts() ([]*config.Artifact, error) { switch { case entry.GithubRelease != nil: - return entry.GithubRelease.GetUpdateImages() + return entry.GithubRelease.GetUpdateArtifacts() case entry.HelmLatest != nil: - return entry.HelmLatest.GetUpdateImages() + return entry.HelmLatest.GetUpdateArtifacts() case entry.Registry != nil: - return entry.Registry.GetUpdateImages() + return entry.Registry.GetUpdateArtifacts() default: return nil, errors.New("did not find update strategy") } } func (entry ConfigEntry) Run(ctx context.Context, opts AutoUpdateOptions) error { - newImages, err := entry.GetUpdateImages() + newArtifacts, err := entry.GetUpdateArtifacts() if err != nil { - return fmt.Errorf("failed to get latest images for %s: %w", entry.Name, err) + return fmt.Errorf("failed to get latest artifacts for %s: %w", entry.Name, err) } accumulator := config.NewArtifactAccumulator() accumulator.AddArtifacts(opts.ConfigYaml.Artifacts...) - imagesToUpdate := make([]*config.Artifact, 0, len(newImages)) - for _, latestImage := range newImages { - imageToUpdate, err := accumulator.TagDifference(latestImage) + artifactsToUpdate := make([]*config.Artifact, 0, len(newArtifacts)) + for _, latestArtifact := range newArtifacts { + artifactToUpdate, err := accumulator.TagDifference(latestArtifact) if err != nil { - return fmt.Errorf("failed to get tag difference for image %s: %w", latestImage.SourceArtifact, err) + return fmt.Errorf("failed to get tag difference for artifact %s: %w", latestArtifact.SourceArtifact, err) } - if imageToUpdate != nil { - imagesToUpdate = append(imagesToUpdate, imageToUpdate) + if artifactToUpdate != nil { + artifactsToUpdate = append(artifactsToUpdate, artifactToUpdate) } } - if len(imagesToUpdate) == 0 { + if len(artifactsToUpdate) == 0 { fmt.Printf("%s: no updates found\n", entry.Name) return nil } - imageSetHash, err := hashImageSet(imagesToUpdate) + artifactSetHash, err := hashArtifactSet(artifactsToUpdate) if err != nil { - return fmt.Errorf("failed to hash set of images that need updates: %w", err) + return fmt.Errorf("failed to hash set of artifacts that need updates: %w", err) } - branchName := fmt.Sprintf("autoupdate/%s/%s", entry.Name, imageSetHash) + branchName := fmt.Sprintf("autoupdate/%s/%s", entry.Name, artifactSetHash) // When filtering pull requests by head branch, the github API // requires that the head branch is in the format :. @@ -213,30 +213,30 @@ func (entry ConfigEntry) Run(ctx context.Context, opts AutoUpdateOptions) error if opts.DryRun { msg := fmt.Sprintf("%s: would make PR under branch %s that adds:\n", entry.Name, branchName) - for _, imageToUpdate := range imagesToUpdate { - for _, fullImage := range imageToUpdate.CombineSourceArtifactAndTags() { - msg = msg + " - " + fullImage + "\n" + for _, artifactToUpdate := range artifactsToUpdate { + for _, fullArtifact := range artifactToUpdate.CombineSourceArtifactAndTags() { + msg = msg + " - " + fullArtifact + "\n" } } fmt.Print(msg) return nil } - return entry.CreateImageUpdatePullRequest(ctx, opts, branchName, imagesToUpdate) + return entry.CreateArtifactUpdatePullRequest(ctx, opts, branchName, artifactsToUpdate) } -func (entry ConfigEntry) CreateImageUpdatePullRequest(ctx context.Context, opts AutoUpdateOptions, branchName string, imagesToUpdate []*config.Artifact) error { +func (entry ConfigEntry) CreateArtifactUpdatePullRequest(ctx context.Context, opts AutoUpdateOptions, branchName string, artifactsToUpdate []*config.Artifact) error { accumulator := config.NewArtifactAccumulator() accumulator.AddArtifacts(opts.ConfigYaml.Artifacts...) if err := git.CreateAndCheckoutBranch(opts.BaseBranch, branchName); err != nil { return fmt.Errorf("failed to create and checkout branch %s: %w", branchName, err) } - for _, imageToUpdate := range imagesToUpdate { + for _, artifactToUpdate := range artifactsToUpdate { // We can reuse the accumulator here because we are making a sequence - // of commits, each of which makes an addition from imagesToUpdate. + // of commits, each of which makes an addition from artifactsToUpdate. configYaml := opts.ConfigYaml - accumulator.AddArtifacts(imageToUpdate) + accumulator.AddArtifacts(artifactToUpdate) configYaml.Artifacts = accumulator.Artifacts() if err := config.Write(paths.ConfigYaml, configYaml); err != nil { return fmt.Errorf("failed to write %s: %w", paths.ConfigYaml, err) @@ -244,16 +244,16 @@ func (entry ConfigEntry) CreateImageUpdatePullRequest(ctx context.Context, opts regsyncYaml, err := configYaml.ToRegsyncConfig() if err != nil { - return fmt.Errorf("failed to generate regsync config for commit for image %s: %w", imageToUpdate.SourceArtifact, err) + return fmt.Errorf("failed to generate regsync config for commit for artifact %s: %w", artifactToUpdate.SourceArtifact, err) } if err := regsync.WriteConfig(paths.RegsyncYaml, regsyncYaml); err != nil { - return fmt.Errorf("failed to write regsync config for commit for image %s: %w", imageToUpdate.SourceArtifact, err) + return fmt.Errorf("failed to write regsync config for commit for artifact %s: %w", artifactToUpdate.SourceArtifact, err) } - tagString := strings.Join(imageToUpdate.Tags, ", ") - msg := fmt.Sprintf("Add tag(s) %s for image %s", tagString, imageToUpdate.SourceArtifact) + tagString := strings.Join(artifactToUpdate.Tags, ", ") + msg := fmt.Sprintf("Add tag(s) %s for artifact %s", tagString, artifactToUpdate.SourceArtifact) if err := git.Commit(msg); err != nil { - return fmt.Errorf("failed to commit changes for image %s: %w", imageToUpdate.SourceArtifact, err) + return fmt.Errorf("failed to commit changes for artifact %s: %w", artifactToUpdate.SourceArtifact, err) } } if err := git.PushBranch(branchName, "origin"); err != nil { @@ -261,14 +261,14 @@ func (entry ConfigEntry) CreateImageUpdatePullRequest(ctx context.Context, opts } tagCount := 0 - for _, imageToUpdate := range imagesToUpdate { - tagCount = tagCount + len(imageToUpdate.Tags) + for _, artifactToUpdate := range artifactsToUpdate { + tagCount = tagCount + len(artifactToUpdate.Tags) } title := fmt.Sprintf("[autoupdate] Add %d tag(s) for `%s`", tagCount, entry.Name) - body := "This PR was created by the autoupdate workflow.\n\nIt adds the following image tags:" - for _, imageToUpdate := range imagesToUpdate { - for _, fullImage := range imageToUpdate.CombineSourceArtifactAndTags() { - body = body + "\n- `" + fullImage + "`" + body := "This PR was created by the autoupdate workflow.\n\nIt adds the following artifact tags:" + for _, artifactToUpdate := range artifactsToUpdate { + for _, fullArtifact := range artifactToUpdate.CombineSourceArtifactAndTags() { + body = body + "\n- `" + fullArtifact + "`" } } maintainerCanModify := true @@ -292,21 +292,21 @@ func (entry ConfigEntry) CreateImageUpdatePullRequest(ctx context.Context, opts return nil } -// hashImageSet computes a human-readable hash from a passed +// hashArtifactSet computes a human-readable hash from a passed // set of Artifacts. Immune to different order of Artifacts, and -// immune to the order of of the tags in those Artifacts. -func hashImageSet(images []*config.Artifact) (string, error) { - for _, image := range images { - image.Sort() +// immune to the order of the tags in those Artifacts. +func hashArtifactSet(artifacts []*config.Artifact) (string, error) { + for _, artifact := range artifacts { + artifact.Sort() } - slices.SortStableFunc(images, config.CompareArtifacts) + slices.SortStableFunc(artifacts, config.CompareArtifacts) hasher := sha256.New() - for _, image := range images { - for _, fullImage := range image.CombineSourceArtifactAndTags() { - _, err := io.WriteString(hasher, fullImage) + for _, artifact := range artifacts { + for _, fullArtifact := range artifact.CombineSourceArtifactAndTags() { + _, err := io.WriteString(hasher, fullArtifact) if err != nil { - return "", fmt.Errorf("failed to write full image %q: %w", fullImage, err) + return "", fmt.Errorf("failed to write full artifact %q: %w", fullArtifact, err) } } } diff --git a/tools/internal/autoupdate/config_test.go b/tools/internal/autoupdate/config_test.go index 2eea29d976..9197b8be0c 100644 --- a/tools/internal/autoupdate/config_test.go +++ b/tools/internal/autoupdate/config_test.go @@ -23,7 +23,7 @@ func TestConfigEntry(t *testing.T) { GithubRelease: &GithubRelease{ Owner: "test-owner", Repository: "test-repo", - Images: []AutoupdateImageRef{{SourceImage: "rancher/rancher"}}, + Artifacts: []AutoupdateArtifactRef{{SourceArtifact: "rancher/rancher"}}, }, Reviewers: []string{"user", "org/team"}, }, @@ -50,7 +50,7 @@ func TestConfigEntry(t *testing.T) { ConfigEntry: ConfigEntry{ Name: "test-entry", Registry: &Registry{ - Images: []AutoupdateImageRef{{SourceImage: "rancher/rancher"}}, + Artifacts: []AutoupdateArtifactRef{{SourceArtifact: "rancher/rancher"}}, Latest: false, VersionFilter: "^v1\\.([3-9][0-9])\\.[0-9]+$", }, @@ -65,7 +65,7 @@ func TestConfigEntry(t *testing.T) { GithubRelease: &GithubRelease{ Owner: "test-owner", Repository: "test-repo", - Images: []AutoupdateImageRef{{SourceImage: "rancher/rancher"}}, + Artifacts: []AutoupdateArtifactRef{{SourceArtifact: "rancher/rancher"}}, }, }, ExpectedError: "must specify Name", @@ -84,8 +84,8 @@ func TestConfigEntry(t *testing.T) { GithubRelease: &GithubRelease{ Owner: "test-owner", Repository: "test-repo", - Images: []AutoupdateImageRef{{ - SourceImage: "rancher/rancher", + Artifacts: []AutoupdateArtifactRef{{ + SourceArtifact: "rancher/rancher", }}, }, HelmLatest: &HelmLatest{ @@ -106,7 +106,7 @@ func TestConfigEntry(t *testing.T) { GithubRelease: &GithubRelease{ Owner: "test-owner", Repository: "test-repo", - Images: []AutoupdateImageRef{{SourceImage: "rancher/rancher"}}, + Artifacts: []AutoupdateArtifactRef{{SourceArtifact: "rancher/rancher"}}, }, Reviewers: []string{"user", "org/team"}, }, @@ -119,7 +119,7 @@ func TestConfigEntry(t *testing.T) { GithubRelease: &GithubRelease{ Owner: "test-owner", Repository: "test-repo", - Images: []AutoupdateImageRef{{SourceImage: "rancher/rancher"}}, + Artifacts: []AutoupdateArtifactRef{{SourceArtifact: "rancher/rancher"}}, }, Reviewers: []string{"org/team/foo"}, }, @@ -132,7 +132,7 @@ func TestConfigEntry(t *testing.T) { GithubRelease: &GithubRelease{ Owner: "test-owner", Repository: "test-repo", - Images: []AutoupdateImageRef{{SourceImage: "rancher/rancher"}}, + Artifacts: []AutoupdateArtifactRef{{SourceArtifact: "rancher/rancher"}}, }, Reviewers: []string{"org/"}, }, @@ -145,7 +145,7 @@ func TestConfigEntry(t *testing.T) { GithubRelease: &GithubRelease{ Owner: "test-owner", Repository: "test-repo", - Images: []AutoupdateImageRef{{SourceImage: "rancher/rancher"}}, + Artifacts: []AutoupdateArtifactRef{{SourceArtifact: "rancher/rancher"}}, }, Reviewers: []string{"/team"}, }, @@ -158,7 +158,7 @@ func TestConfigEntry(t *testing.T) { GithubRelease: &GithubRelease{ Owner: "test-owner", Repository: "test-repo", - Images: []AutoupdateImageRef{{SourceImage: "rancher/rancher"}}, + Artifacts: []AutoupdateArtifactRef{{SourceArtifact: "rancher/rancher"}}, }, Reviewers: []string{}, }, @@ -171,7 +171,7 @@ func TestConfigEntry(t *testing.T) { GithubRelease: &GithubRelease{ Owner: "test-owner", Repository: "test-repo", - Images: []AutoupdateImageRef{{SourceImage: "rancher/rancher"}}, + Artifacts: []AutoupdateArtifactRef{{SourceArtifact: "rancher/rancher"}}, }, Reviewers: nil, }, @@ -199,11 +199,11 @@ func TestGetBranchHash(t *testing.T) { assert.Nil(t, err) imageSet1 := []*config.Artifact{image1, image2} - hash1, err := hashImageSet(imageSet1) + hash1, err := hashArtifactSet(imageSet1) assert.Nil(t, err) imageSet2 := []*config.Artifact{image2, image1} - hash2, err := hashImageSet(imageSet2) + hash2, err := hashArtifactSet(imageSet2) assert.Nil(t, err) assert.Equal(t, hash1, hash2) @@ -213,13 +213,13 @@ func TestGetBranchHash(t *testing.T) { image1, err := config.NewArtifact("test-org/image", []string{"asdf", "qwer"}, "", nil, nil) assert.Nil(t, err) images1 := []*config.Artifact{image1} - hash1, err := hashImageSet(images1) + hash1, err := hashArtifactSet(images1) assert.Nil(t, err) image2, err := config.NewArtifact("test-org/image", []string{"qwer", "asdf"}, "", nil, nil) assert.Nil(t, err) images2 := []*config.Artifact{image2} - hash2, err := hashImageSet(images2) + hash2, err := hashArtifactSet(images2) assert.Nil(t, err) assert.Equal(t, hash1, hash2) @@ -232,11 +232,11 @@ func TestGetBranchHash(t *testing.T) { assert.Nil(t, err) imageSet1 := []*config.Artifact{image1, image2} - hash1, err := hashImageSet(imageSet1) + hash1, err := hashArtifactSet(imageSet1) assert.Nil(t, err) imageSet2 := []*config.Artifact{image1, image2} - hash2, err := hashImageSet(imageSet2) + hash2, err := hashArtifactSet(imageSet2) assert.Nil(t, err) assert.Equal(t, hash1, hash2) @@ -246,13 +246,13 @@ func TestGetBranchHash(t *testing.T) { image1, err := config.NewArtifact("test-org/image", []string{"asdf", "qwer"}, "", nil, nil) assert.Nil(t, err) images1 := []*config.Artifact{image1} - hash1, err := hashImageSet(images1) + hash1, err := hashArtifactSet(images1) assert.Nil(t, err) image2, err := config.NewArtifact("test-org/image", []string{"asdf", "qwer", "zxcv"}, "", nil, nil) assert.Nil(t, err) images2 := []*config.Artifact{image2} - hash2, err := hashImageSet(images2) + hash2, err := hashArtifactSet(images2) assert.Nil(t, err) assert.NotEqual(t, hash1, hash2) diff --git a/tools/internal/autoupdate/githubrelease.go b/tools/internal/autoupdate/githubrelease.go index 6d2c18502a..c68083964c 100644 --- a/tools/internal/autoupdate/githubrelease.go +++ b/tools/internal/autoupdate/githubrelease.go @@ -13,12 +13,12 @@ import ( // GithubRelease retrieves the tags of all github releases that match the VersionConstraint if // LatestOnly is false, or the tag of the latest release if LatestOnly is true. -// It returns the configured Images with this tag. -// It assumes that the images have the same tags as the github releases. +// It returns the configured Artifacts with this tag. +// It assumes that the artifacts have the same tags as the github releases. type GithubRelease struct { Owner string Repository string - Images []AutoupdateImageRef + Artifacts []AutoupdateArtifactRef LatestOnly bool `json:",omitempty"` VersionConstraint string `json:",omitempty"` compiledVersionConstraint *semver.Constraints `json:"-"` @@ -29,7 +29,7 @@ type GithubRelease struct { compiledVersionRegex *regexp.Regexp `json:"-"` } -func (gr *GithubRelease) GetUpdateImages() ([]*config.Artifact, error) { +func (gr *GithubRelease) GetUpdateArtifacts() ([]*config.Artifact, error) { client := github.NewClient(nil) var tags []string @@ -47,15 +47,15 @@ func (gr *GithubRelease) GetUpdateImages() ([]*config.Artifact, error) { tags = append(tags, ghTags...) } - images := make([]*config.Artifact, 0, len(gr.Images)) - for _, sourceImage := range gr.Images { - image, err := config.NewArtifact(sourceImage.SourceImage, tags, sourceImage.TargetImageName, nil, nil) + artifacts := make([]*config.Artifact, 0, len(gr.Artifacts)) + for _, sourceArtifact := range gr.Artifacts { + artifact, err := config.NewArtifact(sourceArtifact.SourceArtifact, tags, sourceArtifact.TargetArtifactName, nil, nil) if err != nil { - return nil, fmt.Errorf("failed to construct image from source image %q and tags %v: %w", sourceImage, tags, err) + return nil, fmt.Errorf("failed to construct artifact from source artifact %q and tags %v: %w", sourceArtifact, tags, err) } - images = append(images, image) + artifacts = append(artifacts, artifact) } - return images, nil + return artifacts, nil } func (gr *GithubRelease) getVersionsFromAllReleases(client *github.Client) ([]string, error) { @@ -136,9 +136,9 @@ func (gr *GithubRelease) Validate() error { if gr.Repository == "" { return errors.New("must specify Repository") } - if gr.Images == nil { + if gr.Artifacts == nil { return errors.New("must specify Artifacts") - } else if len(gr.Images) == 0 { + } else if len(gr.Artifacts) == 0 { return errors.New("must specify at least one element for Artifacts") } if gr.LatestOnly && gr.VersionConstraint != "" { diff --git a/tools/internal/autoupdate/githubrelease_test.go b/tools/internal/autoupdate/githubrelease_test.go index 6841e2af47..21380f530a 100644 --- a/tools/internal/autoupdate/githubrelease_test.go +++ b/tools/internal/autoupdate/githubrelease_test.go @@ -21,7 +21,7 @@ func TestGithubRelease(t *testing.T) { GithubRelease: &GithubRelease{ Owner: "test-owner", Repository: "test-repo", - Images: []AutoupdateImageRef{{SourceImage: "rancher/rancher"}}, + Artifacts: []AutoupdateArtifactRef{{SourceArtifact: "rancher/rancher"}}, LatestOnly: true, }, ExpectedError: "", @@ -31,7 +31,7 @@ func TestGithubRelease(t *testing.T) { GithubRelease: &GithubRelease{ Owner: "test-owner", Repository: "test-repo", - Images: []AutoupdateImageRef{{SourceImage: "rancher/rancher"}}, + Artifacts: []AutoupdateArtifactRef{{SourceArtifact: "rancher/rancher"}}, VersionConstraint: ">3.5.10", }, ExpectedError: "", @@ -41,7 +41,7 @@ func TestGithubRelease(t *testing.T) { GithubRelease: &GithubRelease{ Owner: "test-owner", Repository: "test-repo", - Images: []AutoupdateImageRef{{SourceImage: "rancher/rancher"}}, + Artifacts: []AutoupdateArtifactRef{{SourceArtifact: "rancher/rancher"}}, LatestOnly: true, VersionConstraint: ">3.5.10", }, @@ -51,15 +51,15 @@ func TestGithubRelease(t *testing.T) { Message: "should return error for empty Owner", GithubRelease: &GithubRelease{ Repository: "test-repo", - Images: []AutoupdateImageRef{{SourceImage: "rancher/rancher"}}, + Artifacts: []AutoupdateArtifactRef{{SourceArtifact: "rancher/rancher"}}, }, ExpectedError: "must specify Owner", }, { Message: "should return error for empty Repository", GithubRelease: &GithubRelease{ - Owner: "test-owner", - Images: []AutoupdateImageRef{{SourceImage: "rancher/rancher"}}, + Owner: "test-owner", + Artifacts: []AutoupdateArtifactRef{{SourceArtifact: "rancher/rancher"}}, }, ExpectedError: "must specify Repository", }, @@ -76,7 +76,7 @@ func TestGithubRelease(t *testing.T) { GithubRelease: &GithubRelease{ Owner: "test-owner", Repository: "test-repo", - Images: []AutoupdateImageRef{}, + Artifacts: []AutoupdateArtifactRef{}, }, ExpectedError: "must specify at least one element for Artifacts", }, @@ -85,7 +85,7 @@ func TestGithubRelease(t *testing.T) { GithubRelease: &GithubRelease{ Owner: "test-owner", Repository: "test-repo", - Images: []AutoupdateImageRef{{SourceImage: "rancher/rancher"}}, + Artifacts: []AutoupdateArtifactRef{{SourceArtifact: "rancher/rancher"}}, VersionConstraint: "InvalidVersionConstraint", }, ExpectedError: "invalid VersionConstraint: improper constraint: InvalidVersionConstraint", @@ -95,7 +95,7 @@ func TestGithubRelease(t *testing.T) { GithubRelease: &GithubRelease{ Owner: "test-owner", Repository: "test-repo", - Images: []AutoupdateImageRef{{SourceImage: "rancher/rancher"}}, + Artifacts: []AutoupdateArtifactRef{{SourceArtifact: "rancher/rancher"}}, VersionRegex: "v[asdf[", }, ExpectedError: "invalid VersionRegex: error parsing regexp: missing closing ]: `[asdf[`", @@ -117,7 +117,7 @@ func TestGithubRelease(t *testing.T) { githubRelease := &GithubRelease{ Owner: "test-owner", Repository: "test-repo", - Images: []AutoupdateImageRef{{SourceImage: "rancher/rancher"}}, + Artifacts: []AutoupdateArtifactRef{{SourceArtifact: "rancher/rancher"}}, VersionConstraint: constraintString, } err := githubRelease.Validate() @@ -132,7 +132,7 @@ func TestGithubRelease(t *testing.T) { githubRelease := &GithubRelease{ Owner: "test-owner", Repository: "test-repo", - Images: []AutoupdateImageRef{{SourceImage: "rancher/rancher"}}, + Artifacts: []AutoupdateArtifactRef{{SourceArtifact: "rancher/rancher"}}, VersionRegex: regexString, } err := githubRelease.Validate() @@ -155,7 +155,7 @@ func TestGithubRelease(t *testing.T) { GithubRelease: &GithubRelease{ Owner: "test-owner", Repository: "test-repo", - Images: []AutoupdateImageRef{{SourceImage: "rancher/rancher"}}, + Artifacts: []AutoupdateArtifactRef{{SourceArtifact: "rancher/rancher"}}, }, Tag: "v1.2.3", ExpectedVersion: "v1.2.3", @@ -165,7 +165,7 @@ func TestGithubRelease(t *testing.T) { GithubRelease: &GithubRelease{ Owner: "test-owner", Repository: "test-repo", - Images: []AutoupdateImageRef{{SourceImage: "rancher/rancher"}}, + Artifacts: []AutoupdateArtifactRef{{SourceArtifact: "rancher/rancher"}}, VersionRegex: "^v(.*)$", }, Tag: "v1.2.3", @@ -176,7 +176,7 @@ func TestGithubRelease(t *testing.T) { GithubRelease: &GithubRelease{ Owner: "test-owner", Repository: "test-repo", - Images: []AutoupdateImageRef{{SourceImage: "rancher/rancher"}}, + Artifacts: []AutoupdateArtifactRef{{SourceArtifact: "rancher/rancher"}}, VersionRegex: "v(asdf)", }, Tag: "v1.2.3", @@ -187,7 +187,7 @@ func TestGithubRelease(t *testing.T) { GithubRelease: &GithubRelease{ Owner: "test-owner", Repository: "test-repo", - Images: []AutoupdateImageRef{{SourceImage: "rancher/rancher"}}, + Artifacts: []AutoupdateArtifactRef{{SourceArtifact: "rancher/rancher"}}, VersionConstraint: ">=1.0.0", }, Tag: "v1.2.3", @@ -198,7 +198,7 @@ func TestGithubRelease(t *testing.T) { GithubRelease: &GithubRelease{ Owner: "test-owner", Repository: "test-repo", - Images: []AutoupdateImageRef{{SourceImage: "rancher/rancher"}}, + Artifacts: []AutoupdateArtifactRef{{SourceArtifact: "rancher/rancher"}}, VersionConstraint: "<1.0.0", }, Tag: "v1.2.3", @@ -209,7 +209,7 @@ func TestGithubRelease(t *testing.T) { GithubRelease: &GithubRelease{ Owner: "test-owner", Repository: "test-repo", - Images: []AutoupdateImageRef{{SourceImage: "rancher/rancher"}}, + Artifacts: []AutoupdateArtifactRef{{SourceArtifact: "rancher/rancher"}}, VersionConstraint: "<1.0.0", VersionRegex: "^v(.*)$", }, diff --git a/tools/internal/autoupdate/helmlatest.go b/tools/internal/autoupdate/helmlatest.go index 533d71d755..3ccdf67693 100644 --- a/tools/internal/autoupdate/helmlatest.go +++ b/tools/internal/autoupdate/helmlatest.go @@ -33,9 +33,9 @@ func (e Environment) ToHelmTemplateArgs() []string { // This is not a perfectly reliable way of finding images that a chart uses, and // does not attempt to be. However, it is probably good enough for simple charts. type HelmLatest struct { - // Images tells the autoupdate code which entry in config.yaml to add the + // Artifacts tells the autoupdate code which entry in config.yaml to add the // update images to. - Images []AutoupdateImageRef `json:",omitempty"` + Artifacts []AutoupdateArtifactRef `json:",omitempty"` // HelmRepo is the URL of the Helm chart repository. HelmRepo string // Charts is a map of chart names to the environments under which we want to @@ -45,13 +45,13 @@ type HelmLatest struct { ImageDenylist []string `json:",omitempty"` } -// GetUpdateImages templates the helm chart and extracts all image references -func (hl *HelmLatest) GetUpdateImages() ([]*config.Artifact, error) { +// GetUpdateArtifacts templates the helm chart and extracts all image references +func (hl *HelmLatest) GetUpdateArtifacts() ([]*config.Artifact, error) { if _, err := exec.LookPath("helm"); err != nil { return nil, fmt.Errorf("helm command not found: %w", err) } - imageMap := make(map[string][]string) + artifactMap := make(map[string][]string) addRepoCmd := exec.Command("helm", "repo", "add", helmRepoName, hl.HelmRepo) if err := addRepoCmd.Run(); err != nil { @@ -87,7 +87,7 @@ func (hl *HelmLatest) GetUpdateImages() ([]*config.Artifact, error) { } else if err != nil { return nil, fmt.Errorf("failed to parse template command output as yaml: %w", err) } - if err := hl.extractImagesFromYaml(templateData, imageMap); err != nil { + if err := hl.extractImagesFromYaml(templateData, artifactMap); err != nil { return nil, fmt.Errorf("failed to extract images from chart %s env %s: %w", chartName, environmentName, err) } } @@ -95,12 +95,12 @@ func (hl *HelmLatest) GetUpdateImages() ([]*config.Artifact, error) { } // Convert the map to a slice of config.Artifact objects - images := make([]*config.Artifact, 0, len(imageMap)) - for sourceImage, tags := range imageMap { + images := make([]*config.Artifact, 0, len(artifactMap)) + for sourceImage, tags := range artifactMap { var foundTargetImageName *string - for _, autoupdateImageRef := range hl.Images { - if sourceImage == autoupdateImageRef.SourceImage { - foundTargetImageName = &autoupdateImageRef.TargetImageName + for _, autoupdateImageRef := range hl.Artifacts { + if sourceImage == autoupdateImageRef.SourceArtifact { + foundTargetImageName = &autoupdateImageRef.TargetArtifactName break } } diff --git a/tools/internal/autoupdate/registries.go b/tools/internal/autoupdate/registries.go index 3e7152d378..6e48311ae6 100644 --- a/tools/internal/autoupdate/registries.go +++ b/tools/internal/autoupdate/registries.go @@ -16,7 +16,7 @@ type DockerHub struct { Repository string } -func (d DockerHub) getImageTags() ([]string, error) { +func (d DockerHub) getArtifactTags() ([]string, error) { var allTags []string page := 1 @@ -72,7 +72,7 @@ type QuayIO struct { Repository string } -func (q QuayIO) getImageTags() ([]string, error) { +func (q QuayIO) getArtifactTags() ([]string, error) { var allTags []string page := 1 @@ -128,7 +128,7 @@ type SUSERegistry struct { Repository string } -func (s SUSERegistry) getImageTags() ([]string, error) { +func (s SUSERegistry) getArtifactTags() ([]string, error) { token, err := s.getSuseAuthToken(s.Namespace, s.Repository) if err != nil { return nil, err @@ -181,7 +181,7 @@ type GitHubRegistry struct { Repository string } -func (g GitHubRegistry) getImageTags() ([]string, error) { +func (g GitHubRegistry) getArtifactTags() ([]string, error) { githubToken := os.Getenv("GITHUB_TOKEN") token := base64.StdEncoding.EncodeToString([]byte(githubToken)) var AllTags []string @@ -238,7 +238,7 @@ type GoogleRegistry struct { Repository string } -func (g GoogleRegistry) getImageTags() ([]string, error) { +func (g GoogleRegistry) getArtifactTags() ([]string, error) { req, err := http.NewRequest(http.MethodGet, fmt.Sprintf("https://gcr.io/v2/%s/%s/tags/list", g.Namespace, g.Repository), nil) if err != nil { return nil, fmt.Errorf("failed to create request: %w", err) @@ -265,7 +265,7 @@ type K8sRegistry struct { Repository string } -func (k K8sRegistry) getImageTags() ([]string, error) { +func (k K8sRegistry) getArtifactTags() ([]string, error) { var registryUrl string if k.Namespace != "" { registryUrl = fmt.Sprintf("https://registry.k8s.io/v2/%s/%s/tags/list", k.Namespace, k.Repository) diff --git a/tools/internal/autoupdate/registry.go b/tools/internal/autoupdate/registry.go index 9abd7eb309..f8008aef0b 100644 --- a/tools/internal/autoupdate/registry.go +++ b/tools/internal/autoupdate/registry.go @@ -16,25 +16,25 @@ import ( ) type Registry struct { - Images []AutoupdateImageRef + Artifacts []AutoupdateArtifactRef Latest bool `json:",omitempty"` VersionFilter string `json:",omitempty"` } var httpClient = &http.Client{Timeout: 10 * time.Second} -type ImageRegistry interface { - getImageTags() ([]string, error) +type ArtifactRegistry interface { + getArtifactTags() ([]string, error) } -func (r *Registry) GetUpdateImages() ([]*config.Artifact, error) { - allTags, err := r.getImageTags() +func (r *Registry) GetUpdateArtifacts() ([]*config.Artifact, error) { + allTags, err := r.getArtifactTags() if err != nil { - return nil, fmt.Errorf("failed to get image tags: %w", err) + return nil, fmt.Errorf("failed to get artifact tags: %w", err) } if len(allTags) == 0 { - return nil, fmt.Errorf("no image tags found") + return nil, fmt.Errorf("no artifact tags found") } var filteredTags []string @@ -69,21 +69,21 @@ func (r *Registry) GetUpdateImages() ([]*config.Artifact, error) { filteredTags = []string{vs[len(vs)-1].String()} // Use the latest version } - images := make([]*config.Artifact, 0, len(r.Images)) - for _, sourceImage := range r.Images { - image, err := config.NewArtifact(sourceImage.SourceImage, filteredTags, sourceImage.TargetImageName, nil, nil) + artifacts := make([]*config.Artifact, 0, len(r.Artifacts)) + for _, sourceArtifact := range r.Artifacts { + artifact, err := config.NewArtifact(sourceArtifact.SourceArtifact, filteredTags, sourceArtifact.TargetArtifactName, nil, nil) if err != nil { return nil, err } - image.SetTargetArtifactName(sourceImage.TargetImageName) - images = append(images, image) + artifact.SetTargetArtifactName(sourceArtifact.TargetArtifactName) + artifacts = append(artifacts, artifact) } - return images, nil + return artifacts, nil } func (r *Registry) Validate() error { - if len(r.Images) == 0 { - return errors.New("must specify at least one image") + if len(r.Artifacts) == 0 { + return errors.New("must specify at least one artifact") } if r.VersionFilter != "" { if _, err := regexp.Compile(r.VersionFilter); err != nil { @@ -93,37 +93,37 @@ func (r *Registry) Validate() error { return nil } -func (r *Registry) getImageTags() ([]string, error) { - registry, err := r.getRegistryInformationFromImage() +func (r *Registry) getArtifactTags() ([]string, error) { + registry, err := r.getRegistryInformationFromArtifact() if err != nil { - return nil, fmt.Errorf("failed to get registry information from image: %w", err) + return nil, fmt.Errorf("failed to get registry information from artifact: %w", err) } - return registry.getImageTags() + return registry.getArtifactTags() } -func (r *Registry) getRegistryInformationFromImage() (ImageRegistry, error) { - image := r.Images[0].SourceImage - splittedImage := strings.Split(image, "/") - if len(splittedImage) < 2 { - return nil, fmt.Errorf("invalid image format: %s", image) +func (r *Registry) getRegistryInformationFromArtifact() (ArtifactRegistry, error) { + artifact := r.Artifacts[0].SourceArtifact + splittedArtifact := strings.Split(artifact, "/") + if len(splittedArtifact) < 2 { + return nil, fmt.Errorf("invalid artifact format: %s", artifact) } var registry, namespace, repository string // Case 1: Handle default Docker Hub images like "flannel/flannel" - if len(splittedImage) == 2 && !strings.Contains(splittedImage[0], ".") { + if len(splittedArtifact) == 2 && !strings.Contains(splittedArtifact[0], ".") { registry = "dockerhub" - namespace = splittedImage[0] - repository = splittedImage[1] - } else if len(splittedImage) == 2 && strings.Contains(splittedImage[0], ".") { + namespace = splittedArtifact[0] + repository = splittedArtifact[1] + } else if len(splittedArtifact) == 2 && strings.Contains(splittedArtifact[0], ".") { // Case 2: Handle images with a registry but no namespace like "k8s.gcr.io/pause" - registry = splittedImage[0] + registry = splittedArtifact[0] namespace = "" - repository = splittedImage[1] + repository = splittedArtifact[1] } else { // Default Case: Handle standard 3-part images like "quay.io/skopeo/stable" // and handle images with long paths like "gcr.io/cloud-provider-vsphere/csi/release/syncer" - registry = splittedImage[0] - namespace = splittedImage[1] - repository = strings.Join(splittedImage[2:], "/") + registry = splittedArtifact[0] + namespace = splittedArtifact[1] + repository = strings.Join(splittedArtifact[2:], "/") } switch registry { case "dockerhub": diff --git a/tools/internal/autoupdate/registry_test.go b/tools/internal/autoupdate/registry_test.go index bd90ff8563..d366205787 100644 --- a/tools/internal/autoupdate/registry_test.go +++ b/tools/internal/autoupdate/registry_test.go @@ -16,25 +16,25 @@ func TestRegistry(t *testing.T) { { Message: "should return nil for a valid Registry strategy", Registry: &Registry{ - Images: []AutoupdateImageRef{{SourceImage: "rancher/rancher"}}, + Artifacts: []AutoupdateArtifactRef{{SourceArtifact: "rancher/rancher"}}, Latest: true, VersionFilter: "^v1\\.([3-9][0-9])\\.[0-9]+$", }, ExpectedError: "", }, { - Message: "should return error if no images provided", + Message: "should return error if no artifacts provided", Registry: &Registry{ - Images: nil, + Artifacts: nil, Latest: true, VersionFilter: "^v1\\.([3-9][0-9])\\.[0-9]+$", }, - ExpectedError: "must specify at least one image", + ExpectedError: "must specify at least one artifact", }, { Message: "should return error the versionFilter provided is invalid", Registry: &Registry{ - Images: []AutoupdateImageRef{{SourceImage: "rancher/rancher"}}, + Artifacts: []AutoupdateArtifactRef{{SourceArtifact: "rancher/rancher"}}, Latest: false, VersionFilter: "[", }, diff --git a/tools/internal/config/artifact.go b/tools/internal/config/artifact.go index 46faf29aee..e0fc1c05a4 100644 --- a/tools/internal/config/artifact.go +++ b/tools/internal/config/artifact.go @@ -11,6 +11,7 @@ import ( ) // Artifact should not be instantiated directly. Instead, use NewArtifact(). +// This represents an OCI artifact type Artifact struct { // If DoNotMirror is a bool and true, the Artifact is not mirrored i.e. // it is not added to the regsync config when the regsync config is