-
Notifications
You must be signed in to change notification settings - Fork 5
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge branch 'main' into more-type-docs
- Loading branch information
Showing
21 changed files
with
243 additions
and
77 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -13,30 +13,67 @@ jobs: | |
| runs-on: ubuntu-latest | ||
| timeout-minutes: 30 | ||
| steps: | ||
| - name: Checkout Repo | ||
| uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # pin@v4 | ||
| - name: Set up Go | ||
| uses: actions/setup-go@41dfa10bad2bb2ae585af6ee5bb4d7d973ad74ed # pin@v5 | ||
| with: | ||
| go-version: "1.22" | ||
| # see https://github.com/actions/setup-go?tab=readme-ov-file#caching-dependency-files-and-build-outputs | ||
| cache-dependency-path: | | ||
| go.sum | ||
| - name: Checkout Repo | ||
| uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # pin@v4 | ||
| - name: Get dependencies | ||
| run: go mod download | ||
| - name: Restore build output from cache | ||
| id: cache-build | ||
| uses: actions/cache@v4 | ||
| with: | ||
| path: bin/manager | ||
| key: ${{ runner.os }}-go-build-${{ hashFiles('**/*.go', 'go.sum') }} | ||
| - name: Build | ||
| if: steps.cache-build.outputs.cache-hit != 'true' | ||
| run: make | ||
| - name: Run Unit-Tests | ||
| run: make test-parallel | ||
| - name: Generate code coverage artifacts | ||
| if: ${{ !cancelled() }} | ||
| uses: actions/upload-artifact@v4 | ||
| with: | ||
| name: code-coverage | ||
| path: cover.out | ||
| - name: Upload code coverage information to codecov.io | ||
| if: ${{ !cancelled() }} | ||
| uses: codecov/[email protected] | ||
| with: | ||
| file: cover.out | ||
| fail_ci_if_error: false | ||
| env: | ||
| CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }} | ||
| CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }} | ||
| - name: Upload test results to codecov.io | ||
| if: ${{ !cancelled() }} | ||
| uses: codecov/test-results-action@v1 | ||
| with: | ||
| token: ${{ secrets.CODECOV_TOKEN }} | ||
| codegen: | ||
| name: Check Codegen | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 30 | ||
| steps: | ||
| - name: Checkout Repo | ||
| uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # pin@v4 | ||
| - name: Set up Go | ||
| uses: actions/setup-go@41dfa10bad2bb2ae585af6ee5bb4d7d973ad74ed # pin@v5 | ||
| with: | ||
| go-version: "1.22" | ||
| - name: go mod tidy | ||
| run: | | ||
| go mod tidy | ||
| if ! diff=$(git diff --exit-code --unified=0 -- go.sum); then | ||
| line=$(echo "$diff" | sed -nr 's/@@ -([0-9]+),.*/\1/p' | head -n 1 | tr -d '\n') | ||
| echo "::error file=go.sum,line=$line::go.sum is out of date. Run 'go mod tidy' and commit the changes." | ||
| exit 1 | ||
| fi | ||
| - name: make build-installer | ||
| run: | | ||
| make build-installer | ||
| if ! git diff --exit-code; then | ||
| echo "::error ::Manifests are out of date. Run 'make build-installer' and commit the changes." | ||
| exit 1 | ||
| fi | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -30,3 +30,5 @@ go.work | |
| # Github Certificates | ||
| *.pem | ||
| secret.yaml | ||
|
|
||
| junit.xml | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Empty file.
Empty file.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Empty file.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,17 +1,93 @@ | ||
| # Welcome to MkDocs | ||
| # GitOps Promoter | ||
|
|
||
| For full documentation visit [mkdocs.org](https://www.mkdocs.org). | ||
| GitOps Promoter facilitates environment promotion for config managed via GitOps. | ||
|
|
||
| ## Commands | ||
| ## Key Features | ||
|
|
||
| * `mkdocs new [dir-name]` - Create a new project. | ||
| * `mkdocs serve` - Start the live-reloading docs server. | ||
| * `mkdocs build` - Build the documentation site. | ||
| * `mkdocs -h` - Print help message and exit. | ||
| * Drift-free promotion process | ||
| * Robust promotion gating system | ||
| * Complete integration with git and SCM tooling | ||
| * No fragile automated changes to user-facing files | ||
|
|
||
| ## Project layout | ||
| ## Key Terms | ||
|
|
||
| mkdocs.yml # The configuration file. | ||
| docs/ | ||
| index.md # The documentation homepage. | ||
| ... # Other markdown pages, images and other files. | ||
| * DRY (don't repeat yourself) Branch: a git branch containing config for all environments, often formatted for "DRY" | ||
| config tools like Helm or Kustomize. | ||
| * Staging Branches: branches for "hydrated," environment-specific config awaiting promotion. | ||
| * Live Branches: branches for "hydrated," environment-specific config changes applied to the live system. | ||
| * Promotion PRs: Pull Requests from the staging branches to the live branches, automatically opened and closed by GitOps | ||
| Promoter during promotion. There is never more than one open PR per environment. | ||
| * Hydrator: a tool that automatically pushes changes from the DRY Branch to each environment's Staging Branch. | ||
| * Hydrated Commit: a git commit pushed to Staging or Live Branches based on a DRY Branch commit. | ||
|
|
||
| ## Why GitOps Promoter? | ||
|
|
||
| ### Minimal Drift | ||
|
|
||
| One of the key features of GitOps is that it minimizes drift: the live state of a running system should always match the | ||
| desired state stored in git. | ||
|
|
||
| Environment promotion systems usually break the GitOps principle of minimal drift. After a user makes a change to desired | ||
| state in git, the promotion system will leave higher environments out of sync while lower environments are validated. | ||
| The state of the promotion system is usually presented via a specialized user interface outside of git. | ||
|
|
||
| GitOps Promoter eliminates drift and ensures that the complete system state is visible in your SCM (such as GitHub). | ||
| When the user changes desired state in git, GitOps Promoter automatically opens Pull Requests corresponding to each | ||
| environment. Once all configured promotion gates pass for the first environment, the PR for that environment will be | ||
| automatically merged. Subsequent checks run, and PRs are merged until the change is live in all environments. | ||
|
|
||
| If a new change arrives during the promotion process, pending PRs are updated to include the new changes, and new PRs | ||
| are opened for any other affected environments. Then the deployment process starts again from the first environment. | ||
|
|
||
| By using branches and PRs to represent environment promotion, we make it easy for users to find what they need to know: | ||
|
|
||
| * The desired state of all environments: look at the DRY branch | ||
| * The live state of an environment: look at the environment's live branch | ||
| * The state of change promotion: look at open PRs | ||
|
|
||
| ### Complete SCM Integration | ||
|
|
||
| Since all promotion operations are managed via git and your SCM, the robust tooling ecosystems of those tools are | ||
| available to customize your promotion system. For example, you can you existing GitHub Actions to gate promotion by | ||
| blocking PR merges, or you could use branch protection rules to require manual approval processes. Anything you can do | ||
| with a git branch or a git PR, you can use to interact with the GitOps Promoter. | ||
|
|
||
| If you choose to use GitOps Promoter's CommitStatus API for promotion gating, the status of your custom checks will | ||
| automatically appear in your SCM user interface, making the user experience feel trustworthy and polished. | ||
|
|
||
| ### No Automated DRY Branch Commits | ||
|
|
||
| Many promotion systems work by automatically pushing changes to a DRY Branch. There are two downsides to this strategy. | ||
|
|
||
| First, automatically committing to the DRY Branch may confuse users. If a user is accustomed to doing their own manual | ||
| work in the DRY Branch (for example, modifying Kubernetes manifests), then they may be surprised when an automation | ||
| pushes a change (such as an image tag bump) or might find the automated interference disruptive to their work. | ||
|
|
||
| Second, automated changes to files in the DRY Branch are often fragile. The promotion system must understand the format | ||
| of the DRY Branch contents and avoid making unsafe changes. The system must also understand subtle differences between | ||
| environments and avoid promoting changes in a way that works for one environment but not for another. For example, a | ||
| Kubernetes manifest promotion system could easily mishandle the promotion of Ingress manifest changes if different | ||
| environments have different route configurations or different Ingress apiVersions. | ||
|
|
||
| Rather than operating on the DRY Branch, GitOps Promoter leaves those concerns entirely to the user. The user makes | ||
| their changes, the hydrator transforms the changes appropriately for each environment, and GitOps Promoter promotes the | ||
| hydrated changes to each environment according to the promotion strategy. | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| GitOps Promoter requires a hydration system to be configured in order to promote your changes. | ||
|
|
||
| A hydration system has two jobs: | ||
|
|
||
| 1. Monitor a DRY Branch for new commits. | ||
| 2. When a commit arrives, push an environment-specific Hydrated Commit to the Staging Branch for each environment. The | ||
| contents of the commit are up to the hydrator. For example, if the DRY Branch contains a Helm chart, the hydrator | ||
| might push a file with the output of `helm template`. | ||
| 3. A `hydrator.metadata` file containing JSON file containing: `{"drySha": "<commit SHA from DRY Branch>"}` | ||
|
|
||
| That's the whole contract! GitOps Promoter will handle opening, updating, and merging PRs as newly-hydrated commits | ||
| arrive. | ||
|
|
||
| "Hydration" could be any transformation you can imagine. For Kubernetes manifests, it could be the output of | ||
| `helm template` or `kustomize build`. For JavaScript, it could be the output of a minifier. For JPG images, it could be | ||
| the output of a compression tool. Or it could just be a simple copy of the source file(s). |
Empty file.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,61 @@ | ||
| # Multi-Tenancy | ||
|
|
||
| ## PromotionStrategy Tenancy | ||
|
|
||
| GitOps Promoter provides namespace-based tenancy for PromotionStrategies. | ||
|
|
||
| To enable environment promotion, a user must install these namespaced resources: | ||
|
|
||
| * PromotionStrategy | ||
| * GitRepository | ||
| * ScmProvider | ||
| * Secret (for SCM access) | ||
|
|
||
| To enable self-service PromotionStrategy management for multiple tenants, a GitOps Promoter admin can give each | ||
| tenant write access to a namespace to manage these resources. As long as the GitOps Promoter controller has access to | ||
| those namespaces, it will reconcile the resources. | ||
|
|
||
| Secrets with SCM credentials may only be referenced by ScmProviders in the same namespace, which in turn may only be | ||
| referenced by GitRepositories in the same namespace, which may only be referenced by PromotionStrategies in the same | ||
| namespace. Limiting these references to a namespace prevents one tenant from referencing a Secret in another tenant's | ||
| namespace and thereby gaining write access to another tenant's repositories. | ||
|
|
||
| **Important**: Provision Secrets securely! | ||
|
|
||
| We recommend using a GitOps-friendly Secret provisioning system that populates the Secret resource on-cluster, such as | ||
| an external secrets operator or sealed secrets. | ||
|
|
||
| If an administrator does not want to use namespace-based tenancy, they must either fully manage GitOps Promoter | ||
| resources themselves or build some other system to regulate Secret access among tenants (for example, by validating | ||
| that one tenant's resources do not reference another tenant's resources within the same namespace). | ||
|
|
||
| If there are no trust boundaries to be enforced among PromotionStrategy users, a GitOps Promoter admin may choose to | ||
| host all resources in a single namespace, keeping in mind the need to avoid resource name collisions. | ||
|
|
||
| ## CommitStatus Tenancy | ||
|
|
||
| As with PromotionStrategies, all references from CommitStatuses (to GitRepositories, then ScmProviders, and finally to | ||
| SCM Secrets) must resolve within the same namespace as the CommitStatus. | ||
|
|
||
| Various actors may want to manage CommitStatuses: | ||
|
|
||
| 1. GitOps Promoter administrators | ||
| 2. Special interest teams (for example, a compliance team) | ||
| 3. PromotionStrategy users | ||
|
|
||
| A given PromotionStrategy may need to reference CommitStatuses from any or all of these actors. | ||
|
|
||
| To facilitate the cross-team communication, _PromotionStrategy references to CommitStatuses are cluster-scoped_. If any | ||
| CommitStatus on a cluster matches the key specified in a PromotionStrategy, then the PromotionStrategy controller will | ||
| take that CommitStatus into account for the promotion process. This allows different actors to host CommitStatuses in | ||
| their own namespaces, using their own SCM credentials. | ||
|
|
||
| This cluster-scoped reference is reasonably safe in a multi-tenant setup because: | ||
|
|
||
| 1. The reference is read-only. When referencing a CommitStatus in another namespace, a PromotionStrategy does not leak | ||
| any information about itself. It just reads the status. | ||
| 2. A CommitStatus's commit SHA must match the SHA of a commit being promoted to affect promotion. In other | ||
| words, the CommitStatus's creator must already have knowledge about the SHAs in the PromotionStrategy's repository. | ||
| 3. The worst a malicious or faulty CommitStatus can do is block an environment's promotion. If a promotion is | ||
| erroneously blocked, the PromotionStrategy user can take advantage of an override mechanism (such as manually | ||
| merging the blocked PR), and the GitOps Promoter's admin can investigate and remediate the faulty blocker. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.