Skip to content

Commit

Permalink
Merge branch 'konflux-ci:main' into KONFLUX-179-caas
Browse files Browse the repository at this point in the history
  • Loading branch information
amisstea authored May 20, 2024
2 parents 9a571c7 + d1a89fa commit a599103
Show file tree
Hide file tree
Showing 48 changed files with 744 additions and 217 deletions.
26 changes: 26 additions & 0 deletions .github/workflows/labeler.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
name: labeler

on: [pull_request]

permissions:
pull-requests: write

jobs:
labeler:
runs-on: ubuntu-latest
name: Label the PR size
steps:
- uses: codelytv/pr-size-labeler@v1
with:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
xs_label: 'size/XS'
xs_max_size: '5'
s_label: 'size/S'
s_max_size: '15'
m_label: 'size/M'
m_max_size: '50'
l_label: 'size/L'
l_max_size: '200'
xl_label: 'size/XL'
fail_if_xl: 'false'
github_api_url: 'https://api.github.com'
25 changes: 25 additions & 0 deletions .github/workflows/stale.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
name: 'Close stale issues and PRs'

on:
schedule:
- cron: '30 1 * * *'
workflow_dispatch: {}

permissions:
issues: write
pull-requests: write

jobs:
stale:
runs-on: ubuntu-latest
steps:
- uses: actions/stale@v9
with:
stale-issue-message: >-
This issue is stale because it has been open 90 days with no
activity. Remove the `state/stale` label or comment, or this
will be closed in 30 days.
days-before-stale: 90
days-before-close: 30
stale-issue-label: 'state/stale'
stale-pr-label: 'state/stale'
4 changes: 2 additions & 2 deletions ADR/0003-interacting-with-internal-services.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,12 +13,12 @@ Accepted
---

Many organizations, including Red Hat, possess numerous internal services that help productize their software.
In many cases, these internal services will continue to play a role in the release workflows used in AppStudio/HACBS.
In many cases, these internal services will continue to play a role in the release workflows used in Konflux.

We originally thought that we should expose access to an organization's internal services by encouraging the use of "[bastion](https://en.wikipedia.org/wiki/Bastion_host)" interfaces that are publicly addressable but which also have some degree of internal network access. On review, we see now that internal network ingress like this opens up unwanted
attack vectors towards an organization's internal networks.

**Problem**: AppStudio/Release pipelines need to **initiate** processes with an organization's internal services which
**Problem**: Konflux Release pipelines need to **initiate** processes with an organization's internal services which
are **not publicly addressable** in a secure fashion and be able to obtain process status and completion results.

## Decision
Expand Down
4 changes: 2 additions & 2 deletions ADR/0006-log-conventions.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,13 +47,13 @@ Optionally, also describe the action in the human-readable `msg`.

**Included in:** _none_

There is no need to identify which AppStudio subsystem the log is coming from (i.e., HAS, SPI, or
There is no need to identify which Konflux subsystem the log is coming from (i.e., HAS, SPI, or
GitOps).

Consider: if an engineer is looking at logs directly in the namespace where a controller is
deployed, then you know which service you are looking at. If an engineer is looking at logs
centralized in cloudwatch or splunk, then the namespace from which the log came will be included
automatically as `namespace_name`, which is sufficient to determine what AppStudio subsystem
automatically as `namespace_name`, which is sufficient to determine what Konflux subsystem
produced the log. See section on automatically added logs below.

### 4. Who was involved?
Expand Down
6 changes: 3 additions & 3 deletions ADR/0008-environment-provisioning.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ In our old KCP architecture, we had [a
design](https://docs.google.com/document/d/1WKd1FVHAxaNQKCIzIW-vUQRgsoOP9T-8rYozAMDpYc0/edit#) for
provisioning a new deployment target in support of new [Environments]. This design was to be
implemented in [GITOPSRVCE-228](https://issues.redhat.com/browse/GITOPSRVCE-228) by an environment
controller that would create and manage sub-workspaces of the user’s main AppStudio workspace, and
controller that would create and manage sub-workspaces of the user’s main Konflux workspace, and
that would provide a serviceaccount to Argo in order to deploy the user’s application to those
sub-workspaces. Now, without KCP, we need a new design.

Expand All @@ -43,7 +43,7 @@ The [Environment] CR serves two purposes:

Some use cases to consider for [Environments]:

1. As a part of the AppStudio workspace initialization process, the user should find that both a **dev and stage environment** with corresponding deployment targets are ready for them ([STONE-180](https://issues.redhat.com/browse/STONE-180)). In our post-KCP architecture, these will be backed by **namespaces** on a devsandbox member cluster.
1. As a part of the Konflux workspace initialization process, the user should find that both a **dev and stage environment** with corresponding deployment targets are ready for them ([STONE-180](https://issues.redhat.com/browse/STONE-180)). In our post-KCP architecture, these will be backed by **namespaces** on a devsandbox member cluster.
2. The user will want to manually create **additional** [Environments] (for example, a prod environment). The user may want to use our compute resources provided in the form of a new **namespace on a devsandbox member cluster** for this ([STONE-183](https://issues.redhat.com/browse/STONE-183)) or they may want to **bring their own cluster** as a target ([STONE-162](https://issues.redhat.com/browse/STONE-162)).
3. The integration-service expects to be able to create **ephemeral** [Environments] for automated testing purposes ([STONE-114](https://issues.redhat.com/browse/STONE-114)). For our short-term goals, the automated testing use case requires the same kind of compute as for the dev and stage [Environments] (devsandbox member cluster namespaces), but will expand to include other kinds of deployment targets in the future - like hypershift clusters ([STONE-185](https://issues.redhat.com/browse/STONE-185)).

Expand Down Expand Up @@ -408,7 +408,7 @@ spec:

### Phase 3

The result of this phase is to automatically provisioning of Hypershift cluster using AppStudio's
The result of this phase is to automatically provisioning of Hypershift cluster using Konflux's
credentials. We call it `provided compute` (compute that we provide, not the user) and it’s included
as part of the offering. This compute can be used for both long lived clusters and for ephemeral
clusters used by the integration service.
Expand Down
8 changes: 4 additions & 4 deletions ADR/0010-namespace-metadata.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,13 +19,13 @@ Accepted

## Context

We need metadata on our namespaces to make AppStudio easier to operate and maintain. Standardizing namespace metadata will make it easier for us to search our logs and metrics across clusters. It will also allow us to enable logging for outgoing network traffic, one of our security requirements.
We need metadata on our namespaces to make Konflux easier to operate and maintain. Standardizing namespace metadata will make it easier for us to search our logs and metrics across clusters. It will also allow us to enable logging for outgoing network traffic, one of our security requirements.

## Namespace labels

We will apply the following labels to AppStudio namespaces, to make them easier to identify programmatically. One namespace can have multiple types/labels:
We will apply the following labels to Konflux namespaces, to make them easier to identify programmatically. One namespace can have multiple types/labels:

- `appstudio.redhat.com/namespacetype: "controller"` for namespaces containing controllers developed for AppStudio. For example, we would annotate the `gitops-service-argocd` namespace but not the `openshift-gitops` namespace.
- `appstudio.redhat.com/namespacetype: "controller"` for namespaces containing controllers developed for Konflux. For example, we would annotate the `gitops-service-argocd` namespace but not the `openshift-gitops` namespace.
- `appstudio.redhat.com/namespacetype: "user-workspace-data"` for User workspaces where Applications, Components, and so on are stored
- `appstudio.redhat.com/namespacetype: "user-deployments"` for the namespaces where GitOps deploys applications for users
- `appstudio.redhat.com/namespacetype: "user-builds"` for the namespaces where the Pipeline Service manages users' PipelineRun resources
Expand All @@ -47,7 +47,7 @@ The following labels are used by required operators:

## Namespace annotations

We will apply the following annotation to namespaces installed and maintained by AppStudio on the clusters that Red Hat manages. This will enable OVN network logging to log outgoing network traffic:
We will apply the following annotation to namespaces installed and maintained by Konflux on the clusters that Red Hat manages. This will enable OVN network logging to log outgoing network traffic:

metadata:
annotations:
Expand Down
6 changes: 3 additions & 3 deletions ADR/0011-roles-and-permissions.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# 11. Roles and Permissions for AppStudio
# 11. Roles and Permissions for Konflux

Date: 2023-01-10

Expand All @@ -7,11 +7,11 @@ Date: 2023-01-10
Accepted

## Context
AppStudio is using Kubernetes as the control plane for managing its resources. We require a system for managing user roles and permissions in this context. We have defined the following roles for our project: "Contributor", "Maintainer", and "Admin". We need to map these roles to specific permissions in the Kubernetes RBAC system, in terms of API groups, verbs, and resources.
Konflux is using Kubernetes as the control plane for managing its resources. We require a system for managing user roles and permissions in this context. We have defined the following roles for our project: "Contributor", "Maintainer", and "Admin". We need to map these roles to specific permissions in the Kubernetes RBAC system, in terms of API groups, verbs, and resources.

## Decision

We will use the built-in Kubernetes RBAC system for AppStudio's role and permissions management, and map the following roles to specific permissions, as described in the table below:
We will use the built-in Kubernetes RBAC system for Konflux's role and permissions management, and map the following roles to specific permissions, as described in the table below:

### Roles
**Contributor:** Members who interact with the workspace mostly through pull requests.
Expand Down
10 changes: 5 additions & 5 deletions ADR/0012-namespace-name-format.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,15 +8,15 @@ Accepted

## Context

The OSD-based control plane provisions one namespace in the target member cluster for every workspace (internally represented by a Space CR) which is created for a AppStudio user. All the namespace names provisioned in this way should have a fixed suffix because of two reasons:
1. Visual separation of the namespaces provisioned for AppStudio workspaces.
The OSD-based control plane provisions one namespace in the target member cluster for every workspace (internally represented by a Space CR) which is created for a Konflux user. All the namespace names provisioned in this way should have a fixed suffix because of two reasons:
1. Visual separation of the namespaces provisioned for Konflux workspaces.
2. Limiting the risk of conflicting with the names used for other namespaces that are present in the cluster - either by default for every OCP/OSD/ROSA cluster or created via other SRE pipelines.

## Decision

Every namespace provisioned for an AppStudio top-level workspace will have a name with the fixed suffix `-tenant`. The complete format will be `<workspace-name>-tenant`.
Every namespace provisioned for an AppStudio environment sub-workspace (created from a `SpaceRequest` CR using `appstudio-env` tier) will have a name with the fixed suffix `-env`. The complete format will be `<sub-workspace-name>-env`.
Every namespace provisioned for an Konflux top-level workspace will have a name with the fixed suffix `-tenant`. The complete format will be `<workspace-name>-tenant`.
Every namespace provisioned for an Konflux environment sub-workspace (created from a `SpaceRequest` CR using `appstudio-env` tier) will have a name with the fixed suffix `-env`. The complete format will be `<sub-workspace-name>-env`.

## Consequences

Any changes in the format of the namespace names cause the deletion of all existing namespaces (provisioned for AppStudio workspaces), followed by the creation of the namespaces which will use the new format. In other words, all data in the old namespaces will be deleted.
Any changes in the format of the namespace names cause the deletion of all existing namespaces (provisioned for Konflux workspaces), followed by the creation of the namespaces which will use the new format. In other words, all data in the old namespaces will be deleted.
18 changes: 9 additions & 9 deletions ADR/0013-integration-service-api-contracts.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# 13. AppStudio Test Stream - API contracts
# 13. Konflux Test Stream - API contracts

Date: 2023-01-30

Expand All @@ -10,9 +10,9 @@ Relates to [ADR 14. Let Pipelines Proceed](0014-let-pipelines-proceed.html)

## Context

The AppStudio project being developed aims to serve Red Hat teams but also partners and customers. This requires a level of adaptability to avoid recreating custom flows and Tasks for each stakeholder.
The Konflux project being developed aims to serve Red Hat teams but also partners and customers. This requires a level of adaptability to avoid recreating custom flows and Tasks for each stakeholder.

In this respect Tasks developed by AppStudio test stream should allow swapping external systems to accommodate different environments. This swap should not induce the complete recreation of pipelines.
In this respect Tasks developed by Konflux test stream should allow swapping external systems to accommodate different environments. This swap should not induce the complete recreation of pipelines.

This and the idea of providing a homogeneous experience, which is easier to comprehend and navigate complex systems, leads to the definition of `API contracts`. These contracts need to be understood as guidance that may evolve with time and experience while keeping the aim of building a flexible homogeneous system.

Expand Down Expand Up @@ -40,7 +40,7 @@ To display count of found vulnerabilities and make it easy to understand and eva

The maximum size of a [Task's Results](https://tekton.dev/vault/pipelines-v0.17.3/tasks/#emitting-results) is limited by the container [termination message](https://kubernetes.io/docs/tasks/debug/debug-application/determine-reason-pod-failure/#customizing-the-termination-message) feature of Kubernetes.

App Studio builds are structured as [a shared Persistent Volume per AppStudio Workspace](https://docs.google.com/document/d/1IPlihVjkJ4Kb9tdhsk7iz3bn5rkT_SvJCNQhyXzK3aI/edit#bookmark=id.gefgys3vno2). This allows teams to share builds, implement caching and other shared volumes. A single persistent volume is mapped to each default build pipeline. Builds are passed a directory specific to their builds.
App Studio builds are structured as [a shared Persistent Volume per Konflux Workspace](https://docs.google.com/document/d/1IPlihVjkJ4Kb9tdhsk7iz3bn5rkT_SvJCNQhyXzK3aI/edit#bookmark=id.gefgys3vno2). This allows teams to share builds, implement caching and other shared volumes. A single persistent volume is mapped to each default build pipeline. Builds are passed a directory specific to their builds.

#### Tekton Result Format for `HACBS_TEST_OUTPUT`

Expand All @@ -57,11 +57,11 @@ The output will provide the following information about the overall test result:

Example contents of the test result output file (**HACBS_TEST_OUTPUT**) for a failed run:
```
{
{
"result": "FAILURE",
"namespace": "image_labels",
"timestamp": "1649148140",
"successes": 12,
"successes": 12,
"note": "Task fbc-related-image-check failed: Command skopeo inspect could not inspect images. For details, check Tekton task log.",
"failures": 2,
"warnings": 0
Expand Down Expand Up @@ -170,7 +170,7 @@ Each test will have a standardized short name in snake case, e.g. `release_label
The output will provide the following information about the overall test result:
- **filename** - Name of the file that was inspected by the Tekton task
- This will reflect the data that the test was executed on, such as `image-inspect.json`, `clair-vulnerabilities.json` etc.
- **namespace** - The rego namespace of the test policy.
- **namespace** - The rego namespace of the test policy.
- **successes** - The number of successful checks in the form of an integer
- **failures** - A JSON list containing objects describing each failure

Expand Down Expand Up @@ -238,7 +238,7 @@ Since tags can be moved from one image to another, they should not be relied on
## Consequences

As a result of the decision here to summarize results in a **HACBS_TEST_OUTPUT** result and store the larger test output as a file named **test_name_output.json**, we should find that:
* Other components in AppStudio can leverage information exposed by TaskRuns - notably the UI (HAC), integration-service, and enterprise-contract - enabling features for the larger system that need to depend on some data from inside a variety of TaskRuns.
* Other components in Konflux can leverage information exposed by TaskRuns - notably the UI (HAC), integration-service, and enterprise-contract - enabling features for the larger system that need to depend on some data from inside a variety of TaskRuns.
* We'll be able to have PipelineRuns that _succeed_ and continue, even if they have tasks whose payloads fail and expose errors. This enables a progressive model where the user can get a build and a functional test and a deployment to their development Environment, even if a linter or scanner in their build pipeline emits an error.
* By having chosen a convention that has "HACBS" in the name, we're going to have trouble integrating third-party Task providers in the future. In order to have its output respected by our system, a hypothetical vendor of a third-party scanner will need to add a "HACBS_TEST_OUTPUT" result on their Task, which is oddly specific to our system. At some point in the future, we should revise this decision to instead align to a common upstream convention that gains traction in the broader tekton ecosystem. See also [HACBS-1563](https://issues.redhat.com/browse/HACBS-1563).

Expand Down Expand Up @@ -269,7 +269,7 @@ Information, which relates to an environment like connection details and credent
## Appendix

- **Task recommendations**: This document focuses (it was at least the original intention) on API contracts but there are more recommendations regarding Task developments. The following ones have been collected by the Tekton project: https://github.com/tektoncd/catalog/blob/main/recommendations.md
- **Programming language**: Tekton supports “choosing the right language for the right task”. That said, from an operational point of view it is beneficial to limit the number of programming languages needed to support AppStudio. Defining “default” languages helps with limiting the skills required to support the platform. This also helps with avoiding knowledge islands where only a few people are able to maintain some Tasks.
- **Programming language**: Tekton supports “choosing the right language for the right task”. That said, from an operational point of view it is beneficial to limit the number of programming languages needed to support Konflux. Defining “default” languages helps with limiting the skills required to support the platform. This also helps with avoiding knowledge islands where only a few people are able to maintain some Tasks.
- **Failure behavior**: A retry mechanism with configurable timeout and exponential backoff needs to be implemented for technical or functional recoverable failures. A scenario example: An image may not have been completely indexed when the result of the security scan is interrogated. In such a case the Task should have a retry mechanism that may wait till completion of the indexing or time out.
- **Repositories**: Whenever there is no sensitive information we aim to have the PoC sources in a public repository. Is there a public GitHub organization for that? Can we use [this](https://github.com/redhat-appstudio)? When a Task is specifically for Red Hat’s infrastructure it should be kept in a private repository. Is there a private GitLab group for that?

Expand Down
2 changes: 1 addition & 1 deletion ADR/0014-let-pipelines-proceed.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
Accepted

Relates to:
* [ADR 13. AppStudio Test Stream - API contracts](0013-integration-service-api-contracts.html)
* [ADR 13. Konflux Test Stream - API contracts](0013-integration-service-api-contracts.html)
* [ADR 30. Tekton Results Naming Convention](0030-tekton-results-naming-convention.html)
* [ADR 32. Decoupling Deployment](0032-decoupling-deployment.html)

Expand Down
Loading

0 comments on commit a599103

Please sign in to comment.