Skip to content

Commit

Permalink
feat!: migrate to ARC (#173)
Browse files Browse the repository at this point in the history
  • Loading branch information
gtsorbo authored Aug 7, 2024
1 parent 3ac2822 commit 0b1299b
Show file tree
Hide file tree
Showing 39 changed files with 761 additions and 779 deletions.
88 changes: 27 additions & 61 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,81 +1,41 @@
# terraform-google-github-actions-runners

[![awesome-runners](https://img.shields.io/badge/listed%20on-awesome--runners-blue.svg)](https://github.com/jonico/awesome-runners)

Using these Terraform modules you can quickly deploy Self Hosted Github Runners for jobs in your GitHub Actions workflows

## [Self Hosted Runners on GKE](modules/gh-runner-gke/README.md)

The `gh-runner-gke` module provisions the resources required to deploy Self Hosted Runners on GCP infrastructure using GKE.

This includes

- Enabling necessary APIs
- VPC
- GKE Cluster
- Kubernetes Secret

Below are some examples:
## Description
Using these Terraform modules you can quickly deploy self-hosted GitHub Runners for jobs in your GitHub Actions workflows

### [Self Hosted runners on GKE that support Docker Workflows](examples/gh-runner-gke-dind/README.md)
## Modules

This example shows how to deploy Self Hosted Runners on GKE that supports Docker Workflows.
### [Self Hosted Runners on GKE](modules/gh-runner-gke/)
The `gh-runner-gke` module provisions the resources required to deploy Self Hosted Runners on GCP infrastructure using GKE. This module follows the guidance for using [Actions Runner Controller](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners-with-actions-runner-controller/quickstart-for-actions-runner-controller#installing-actions-runner-controller) provided by GitHub

### [Simple Self Hosted Runners on GKE](examples/gh-runner-gke-simple/README.md)

This example shows how to deploy a simple GKE Self Hosted Runner.
Examples:
- [Simple Self Hosted Runners on GKE](examples/gh-runner-gke-simple/)
- [Self Hosted runners on GKE that support Docker Workflows](examples/gh-runner-gke-dind/)
- [Self Hosted runners on GKE that support Docker Workflows in rootless configuration](examples/gh-runner-gke-dind-rootless/)

More examples of [Self Hosted Runners on GKE/Anthos](https://github.com/github-developer/self-hosted-runners-anthos).

## [Self Hosted Runners on Managed Instance Groups using VMs](modules/gh-runner-mig-vm/README.md)

### [Self Hosted Runners on Managed Instance Groups using VMs](modules/gh-runner-mig-vm/)
The `gh-runner-mig-vm` module provisions the resources required to deploy Self Hosted Runners on GCP infrastructure using Managed Instance Groups.

This includes

- Enabling necessary APIs
- VPC
- NAT & Cloud Router
- Service Account for MIG
- MIG Instance Template
- MIG Instance Manager
- FW Rules
- Secret Manager Secret

Deployment of Managed Instance Groups requires a [Google VM image](https://cloud.google.com/compute/docs/images) with a startup script that downloads and configures the Runner or a pre-baked image with the runner installed.

Below are some examples:

### [Simple Self Hosted Runner on MIG VMs](examples/gh-runner-mig-native-simple/README.md)

This example shows how to deploy a MIG Self Hosted Runner with startup scripts.

### [Self Hosted Runner on MIG VMs from Packer Image](examples/gh-runner-mig-native-packer/README.md)

This example shows how to deploy a MIG Self Hosted Runner with an image pre-baked using Packer.

## [Self Hosted Runners on Managed Instance Groups using Container VMs](modules/gh-runner-mig-container-vm/README.md)
Examples:
- [Simple Self Hosted Runner on MIG VMs](examples/gh-runner-mig-native-simple/)
- [Self Hosted Runner on MIG VMs from Packer Image](examples/gh-runner-mig-native-packer/)

### [Self Hosted Runners on Managed Instance Groups using Container VMs](modules/gh-runner-mig-container-vm/)
The `gh-runner-mig-container-vm` module provisions the resources required to deploy Self Hosted Runners on GCP infrastructure using Managed Instance Groups.

This includes

- Enabling necessary APIs
- VPC
- NAT & Cloud Router
- MIG Container Instance Template
- MIG Instance Manager
- FW Rules
Examples:
- [Simple Self Hosted Runner on MIG Container VMs](examples/gh-runner-mig-container-vm-simple/)
- [Self Hosted runners on MIG Container VMs that support Docker Workflows](examples/gh-runner-mig-container-vm-dind/)

Below are some examples:
### [GitHub OIDC](modules/gh-oidc/)
This module handles the opinionated creation of infrastructure necessary to configure [Workload Identity pools](https://cloud.google.com/iam/docs/workload-identity-federation#pools) and [providers](https://cloud.google.com/iam/docs/workload-identity-federation#providers) for authenticating to GCP using GitHub Actions OIDC tokens.

### [Self Hosted runners on MIG Container VMs that support Docker Workflows](examples/gh-runner-mig-container-vm-dind/README.md)

This example shows how to deploy a Self Hosted Runner that supports Docker Workflows on MIG Container VMs.

### [Simple Self Hosted Runner on MIG Container VMs](examples/gh-runner-mig-container-vm-simple/README.md)

This example shows how to deploy a Self Hosted Runner on MIG Container VMs.
Examples:
- [Simple Workload Identity configuration for GitHub OIDC](examples/oidc-simple/)


## Requirements
Expand All @@ -94,5 +54,11 @@ The following dependencies must be available:
Refer to the [contribution guidelines](./CONTRIBUTING.md) for
information on contributing to this module.

[iam-module]: https://registry.terraform.io/modules/terraform-google-modules/iam/google
[project-factory-module]: https://registry.terraform.io/modules/terraform-google-modules/project-factory/google
[terraform-provider-gcp]: https://www.terraform.io/docs/providers/google/index.html
[terraform]: https://www.terraform.io/downloads.html

## Security Disclosures

Please see our [security disclosure process](./SECURITY.md).
50 changes: 41 additions & 9 deletions build/int.cloudbuild.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -45,22 +45,54 @@ steps:
args: ['/bin/bash', '-c', 'cft test run TestGHRunnerMIGContainerVM --stage destroy --verbose']
waitFor:
- verify-gh-runner-mig-container-vm
# test/fixtures/gh-runner-gke
- id: apply-gh-runner-gke
# TestAll/examples/gh-runner-gke-simple
- id: apply-gh-runner-gke-simple
name: 'gcr.io/cloud-foundation-cicd/$_DOCKER_IMAGE_DEVELOPER_TOOLS:$_DOCKER_TAG_VERSION_DEVELOPER_TOOLS'
args: ['/bin/bash', '-c', 'cft test run TestGHRunnerGKE --stage apply --verbose']
args: ['/bin/bash', '-c', 'cft test run TestGhRunnerGkeSimple --stage apply --verbose']
waitFor:
- init-all
- id: verify-gh-runner-gke
- id: verify-gh-runner-gke-simple
name: 'gcr.io/cloud-foundation-cicd/$_DOCKER_IMAGE_DEVELOPER_TOOLS:$_DOCKER_TAG_VERSION_DEVELOPER_TOOLS'
args: ['/bin/bash', '-c', 'cft test run TestGHRunnerGKE --stage verify --verbose']
args: ['/bin/bash', '-c', 'cft test run TestGhRunnerGkeSimple --stage verify --verbose']
waitFor:
- apply-gh-runner-gke
- id: destroy-gh-runner-gke
- apply-gh-runner-gke-simple
- id: destroy-gh-runner-gke-simple
name: 'gcr.io/cloud-foundation-cicd/$_DOCKER_IMAGE_DEVELOPER_TOOLS:$_DOCKER_TAG_VERSION_DEVELOPER_TOOLS'
args: ['/bin/bash', '-c', 'cft test run TestGHRunnerGKE --stage destroy --verbose']
args: ['/bin/bash', '-c', 'cft test run TestGhRunnerGkeSimple --stage destroy --verbose']
waitFor:
- verify-gh-runner-gke
- verify-gh-runner-gke-simple
# TestAll/examples/gh-runner-gke-dind
- id: apply-gh-runner-gke-dind
name: 'gcr.io/cloud-foundation-cicd/$_DOCKER_IMAGE_DEVELOPER_TOOLS:$_DOCKER_TAG_VERSION_DEVELOPER_TOOLS'
args: ['/bin/bash', '-c', 'cft test run TestGhRunnerGkeDind --stage apply --verbose']
waitFor:
- init-all
- id: verify-gh-runner-gke-dind
name: 'gcr.io/cloud-foundation-cicd/$_DOCKER_IMAGE_DEVELOPER_TOOLS:$_DOCKER_TAG_VERSION_DEVELOPER_TOOLS'
args: ['/bin/bash', '-c', 'cft test run TestGhRunnerGkeDind --stage verify --verbose']
waitFor:
- apply-gh-runner-gke-dind
- id: destroy-gh-runner-gke-dind
name: 'gcr.io/cloud-foundation-cicd/$_DOCKER_IMAGE_DEVELOPER_TOOLS:$_DOCKER_TAG_VERSION_DEVELOPER_TOOLS'
args: ['/bin/bash', '-c', 'cft test run TestGhRunnerGkeDind --stage destroy --verbose']
waitFor:
- verify-gh-runner-gke-dind
# TestAll/examples/gh-runner-gke-dind-rootless
- id: apply-gh-runner-gke-dind-rootless
name: 'gcr.io/cloud-foundation-cicd/$_DOCKER_IMAGE_DEVELOPER_TOOLS:$_DOCKER_TAG_VERSION_DEVELOPER_TOOLS'
args: ['/bin/bash', '-c', 'cft test run TestGhRunnerGkeDindRootless --stage apply --verbose']
waitFor:
- init-all
- id: verify-gh-runner-gke-dind-rootless
name: 'gcr.io/cloud-foundation-cicd/$_DOCKER_IMAGE_DEVELOPER_TOOLS:$_DOCKER_TAG_VERSION_DEVELOPER_TOOLS'
args: ['/bin/bash', '-c', 'cft test run TestGhRunnerGkeDindRootless --stage verify --verbose']
waitFor:
- apply-gh-runner-gke-dind-rootless
- id: destroy-gh-runner-gke-dind-rootless
name: 'gcr.io/cloud-foundation-cicd/$_DOCKER_IMAGE_DEVELOPER_TOOLS:$_DOCKER_TAG_VERSION_DEVELOPER_TOOLS'
args: ['/bin/bash', '-c', 'cft test run TestGhRunnerGkeDindRootless --stage destroy --verbose']
waitFor:
- verify-gh-runner-gke-dind-rootless
# examples/oidc-simple
- id: apply-oidc
name: 'gcr.io/cloud-foundation-cicd/$_DOCKER_IMAGE_DEVELOPER_TOOLS:$_DOCKER_TAG_VERSION_DEVELOPER_TOOLS'
Expand Down
30 changes: 30 additions & 0 deletions docs/upgrading_to_v4.0.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# Upgrading to v4.0

The v4.0 release of the `gh-runner-gke` module contains breaking changes.
## Migration Instructions

### `gh-runner-gke` module

```diff
module "runner-gke" {
source = "terraform-google-modules/github-actions-runners/google//modules/gh-runner-gke"
- version = "~> 3.0"
+ version = "~> 4.0"

project_id = var.project_id
create_network = true
+ cluster_suffix = "repo"
- repo_name = "repo"
- repo_owner = "repo_owner"
- repo_url = "repo_url"
- gh_token = "gh_token"
+ gh_app_id = "123456"
+ gh_app_installation_id = "12345678"
+ gh_app_private_key = "sample"
+ gh_config_url = "https://github.com/ORGANIZATION"
}
```

- Previously, the name of the created GKE cluster appended `repo_name` as a suffix. Now, the value of the suffix is set via `cluster_suffix`, as runner clusters can be created for workflows more than one GitHub repo. To prevent cluster deletion when upgrading, set the value of `cluster_suffix` to the previous value of `repo_name`.
- The module now prefers authentication via GitHub App installation. Follow the instructions for [authenticating to ARC via GitHub App](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners-with-actions-runner-controller/authenticating-to-the-github-api#authenticating-arc-with-a-github-app), and provide the necessary values to the module instead of the previous repo authentication values. Upgrading will destroy the previously created `runner_k8s_config` kubernetes secret, so previous GitHub Runner images may fail.
- Previously, the GKE cluster service account was provided with the `roles/storage.objectViewer` role to enable pulling images from Container Registry. The module no longer requires access to Container Registry, and so IAM membership has been removed. If your cluster relies on this role, you will need to add it again manually.
41 changes: 41 additions & 0 deletions examples/gh-runner-gke-dind-rootless/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# Self Hosted runners on GKE that support dind-rootless

## Overview

This example shows how to deploy runners that support Docker Workflows on GKE in a rootless configuration. This example is based on the instructions published in [Deploying GitHub Action Runners on GKE with dind-rootless](https://medium.com/google-cloud/github-action-runners-on-gke-with-dind-rootless-bd54e23516c9).

More examples of [Self Hosted Runners on GKE/Anthos](https://github.com/github-developer/self-hosted-runners-anthos).

## Deployment

1. Follow the instructions in the [GitHub documentation](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners-with-actions-runner-controller/authenticating-to-the-github-api#authenticating-arc-with-a-github-app) to create a GitHub App for authenticating ARC

1. Gather the values for your GitHub App ID, GitHub App Installation ID, and GitHub App Private Key from the instructions linked above.

1. Substitute your values into the example [`main.tf`](main.tf). Modify any other values as needed. For a full list of available variables, refer to the [module documentation](../../modules/gh-runner-gke/).

1. Execute Terraform commands to create the required resources.
```sh
terraform init
terraform apply
```

1. Your runners should become active at `https://github.com/organizations/ORGANIZATION/settings/actions/runners`.

<!-- BEGINNING OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
## Inputs

| Name | Description | Type | Default | Required |
|------|-------------|------|---------|:--------:|
| project\_id | The project id to deploy Github Runner MIG | `string` | n/a | yes |

## Outputs

| Name | Description |
|------|-------------|
| cluster\_name | Cluster name |
| location | Cluster location |
| project\_id | The project in which resources are created |
| service\_account | The default service account used for running nodes. |

<!-- END OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
35 changes: 35 additions & 0 deletions examples/gh-runner-gke-dind-rootless/main.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
/**
* Copyright 2020 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

module "runner-gke" {
source = "terraform-google-modules/github-actions-runners/google//modules/gh-runner-gke"
version = "~> 3.0"

project_id = var.project_id
create_network = true
network_name = "runner-network-dind-r"
subnet_name = "runner-subnet-dind-r"
cluster_suffix = "dind-rootless"
gh_app_id = "123456"
gh_app_installation_id = "12345678"
gh_app_private_key = "sample"
gh_config_url = "https://github.com/ORGANIZATION"

# pass values.yaml for dind-rootless runners configuratin
arc_runners_values = [
file("${path.module}/values.yaml")
]
}
Original file line number Diff line number Diff line change
Expand Up @@ -16,28 +16,20 @@

output "service_account" {
description = "The default service account used for running nodes."
value = module.example_gke_runner.service_account
value = module.runner-gke.service_account
}

output "cluster_name" {
description = "Cluster name"
value = module.example_gke_runner.cluster_name
}
output "network_name" {
description = "Name of VPC"
value = module.example_gke_runner.network_name
}

output "subnet_name" {
description = "Name of VPC"
value = module.example_gke_runner.subnet_name
value = module.runner-gke.cluster_name
}

output "location" {
description = "Cluster location"
value = module.example_gke_runner.location
value = module.runner-gke.location
}

output "project_id" {
value = var.project_id_gke
value = var.project_id
description = "The project in which resources are created"
}
40 changes: 40 additions & 0 deletions examples/gh-runner-gke-dind-rootless/providers.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
/**
* Copyright 2020 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

/*****************************************
Kubernetes provider configuration
*****************************************/

data "google_client_config" "default" {
}

provider "kubernetes" {
host = "https://${module.runner-gke.kubernetes_endpoint}"
token = data.google_client_config.default.access_token
cluster_ca_certificate = base64decode(module.runner-gke.ca_certificate)
}

provider "helm" {
kubernetes {
host = "https://${module.runner-gke.kubernetes_endpoint}"
token = data.google_client_config.default.access_token
cluster_ca_certificate = base64decode(module.runner-gke.ca_certificate)
exec {
api_version = "client.authentication.k8s.io/v1beta1"
command = "gke-gcloud-auth-plugin"
}
}
}
Loading

0 comments on commit 0b1299b

Please sign in to comment.