-
Notifications
You must be signed in to change notification settings - Fork 1.2k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
feat: Switch to native Terraform resources for hub registration and ACM
- Loading branch information
Showing
54 changed files
with
900 additions
and
1,161 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,5 +1,4 @@ | ||
# Upgrading to v21.0 | ||
|
||
The v21.0 release of *kubernetes-engine* is a backwards incompatible | ||
release. | ||
|
||
|
@@ -14,3 +13,135 @@ The [Terraform Kubernetes Engine Module](https://github.com/terraform-google-mod | |
### Kubernetes Provider upgrade | ||
The Terraform Kubernetes Engine module now requires version 2.10 or higher of | ||
the Kubernetes Provider. | ||
|
||
### Hub module rewrite | ||
The old [Hub submodule](https://github.com/terraform-google-modules/terraform-google-kubernetes-engine/tree/v20.0.0/modules/hub) | ||
has been renamed to `hub-legacy` and deprecated. It is replaced with a new [fleet membership](https://github.com/terraform-google-modules/terraform-google-kubernetes-engine/tree/master/modules/fleet-membership) | ||
module to handle registering GKE clusters to [fleets](https://cloud.google.com/anthos/multicluster-management/fleets) using the native API. | ||
|
||
The new module relies exclusively on native Terraform resources and should therefore be more robust. | ||
|
||
### Migrating | ||
For GKE clusters, you should update your configuration as follows: | ||
|
||
```diff | ||
module "register" { | ||
- source = "terraform-google-modules/kubernetes-engine/google//modules/hub" | ||
- version = "~> 20.0" | ||
+ source = "terraform-google-modules/kubernetes-engine/google//modules/fleet-membership" | ||
+ version = "~> 21.0" | ||
|
||
project_id = "my-project-id" | ||
cluster_name = "my-cluster-name" | ||
- gke_hub_membership_name = "gke-membership" | ||
+ membership_name = "gke-hub-membership" | ||
location = module.gke.location | ||
- cluster_endpoint = module.gke.endpoint | ||
- gke_hub_sa_name = "sa-for-kind-cluster-membership" | ||
- use_kubeconfig = true | ||
- labels = "testlabel=usekubecontext" | ||
- module_depends_on = [module.gke] | ||
} | ||
``` | ||
|
||
You also need to follow these migration steps: | ||
|
||
1. Remove the old module from your state: | ||
|
||
``` | ||
terraform state rm module.register | ||
``` | ||
2. Remove the cluster from the fleet: | ||
``` | ||
gcloud container fleet memberships delete gke-hub-membership-name | ||
``` | ||
3. Apply the new configuration to re-register the cluster: | ||
``` | ||
terraform apply | ||
``` | ||
#### Legacy module | ||
**The native API only supports registering GKE clusters**. Therefore, the old hub module is preserved as `hub-legacy`. | ||
You can continue using it by updating your configuration to point to the new location. | ||
```diff | ||
module "register" { | ||
- source = "terraform-google-modules/kubernetes-engine/google//modules/hub" | ||
- version = "~> 20.0" | ||
+ source = "terraform-google-modules/kubernetes-engine/google//modules/hub-legacy" | ||
+ version = "~> 21.0" | ||
project_id = "my-project-id" | ||
cluster_name = "my-cluster-name" | ||
location = module.gke.location | ||
cluster_endpoint = module.gke.endpoint | ||
} | ||
``` | ||
|
||
### Anthos Config Management (ACM) and Config Sync Module Rewrite | ||
Together with the rewrite of the Hub module, the ACM module also has been rewritten to use native resources. | ||
|
||
You will need to follow these migration steps: | ||
|
||
1. Update your configuration to use the new module: | ||
|
||
```diff | ||
module "acm" { | ||
source = "terraform-google-modules/kubernetes-engine/google//modules/acm" | ||
- version = "~> 20.0" | ||
+ version = "~> 21.0" | ||
|
||
project_id = "my-project-id" | ||
cluster_name = "simple-zonal-cluster" | ||
location = "us-central1-a" | ||
- cluster_endpoint = module.auth.host | ||
|
||
sync_repo = "[email protected]:GoogleCloudPlatform/csp-config-management.git" | ||
sync_branch = "1.0.0" | ||
policy_dir = "foo-corp" | ||
|
||
secret_type = "ssh" | ||
} | ||
``` | ||
|
||
1. Make sure you have the `kubernetes` provider configured: | ||
|
||
```hcl | ||
provider "kubernetes" { | ||
cluster_ca_certificate = module.auth.cluster_ca_certificate | ||
host = module.auth.host | ||
token = module.auth.token | ||
} | ||
``` | ||
|
||
1. Remove the old module from your state: | ||
|
||
``` | ||
terraform state rm module.acm | ||
``` | ||
|
||
2. Import the old `git-creds` secret into Terraform: | ||
|
||
``` | ||
terraform import 'module.acm.module.acm_operator.kubernetes_secret_v1.creds' 'config-management-system/git-creds' | ||
``` | ||
|
||
3. Apply the new configuration to re-register ACM and confirm everything is working: | ||
|
||
``` | ||
terraform apply | ||
``` | ||
|
||
#### Feature Activation | ||
|
||
Only the first cluster in a fleet should activate the ACM fleet feature. | ||
Other clusters should disable feature activation by setting `enable_fleet_feature = false`. | ||
|
||
#### Config Sync Module Removed | ||
The dedicated Config Sync submodule has been removed. | ||
To use Config Sync, just invoke the ACM module with `enable_policy_controller = false`. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -4,23 +4,38 @@ This example illustrates how to create a simple cluster and install [Anthos Conf | |
|
||
It incorporates the standard cluster module and the [ACM install module](../../modules/acm). | ||
|
||
## Verifying Success | ||
|
||
After applying the Terraform configuration, you can run the following commands to verify that your cluster has synced correctly: | ||
|
||
1. Check ACM install status: | ||
|
||
``` | ||
gcloud config set project $(terraform output --raw project_id) | ||
gcloud alpha container hub config-management status | ||
``` | ||
2. Connect to the cluster: | ||
``` | ||
gcloud container clusters get-credentials $(terraform output --raw cluster_name) --zone=$(terraform output --raw location) | ||
``` | ||
3. Confirm the `shipping-dev` namespace was created: | ||
``` | ||
kubectl describe ns shipping-dev | ||
``` | ||
<!-- BEGINNING OF PRE-COMMIT-TERRAFORM DOCS HOOK --> | ||
## Inputs | ||
| Name | Description | Type | Default | Required | | ||
|------|-------------|------|---------|:--------:| | ||
| acm\_policy\_dir | Subfolder containing configs in ACM Git repo | `string` | `"foo-corp"` | no | | ||
| acm\_sync\_branch | Anthos config management Git branch | `string` | `"1.0.0"` | no | | ||
| acm\_sync\_repo | Anthos config management Git repo | `string` | `"[email protected]:GoogleCloudPlatform/csp-config-management.git"` | no | | ||
| cluster\_name\_suffix | A suffix to append to the default cluster name | `string` | `""` | no | | ||
| ip\_range\_pods | The secondary ip range to use for pods | `any` | n/a | yes | | ||
| ip\_range\_services | The secondary ip range to use for services | `any` | n/a | yes | | ||
| network | The VPC network to host the cluster in | `any` | n/a | yes | | ||
| operator\_path | Path to the operator yaml config. If unset, will download from GCS releases. | `string` | `null` | no | | ||
| project\_id | The project ID to host the cluster in | `any` | n/a | yes | | ||
| region | The region to host the cluster in | `any` | n/a | yes | | ||
| subnetwork | The subnetwork to host the cluster in | `any` | n/a | yes | | ||
| zones | The zone to host the cluster in (required if is a zonal cluster) | `list(string)` | n/a | yes | | ||
| region | The region to host the cluster in | `string` | `"us-central1"` | no | | ||
| zone | The zone to host the cluster in | `string` | `"us-central1-a"` | no | | ||
## Outputs | ||
|
@@ -36,7 +51,7 @@ It incorporates the standard cluster module and the [ACM install module](../../m | |
| location | n/a | | ||
| master\_kubernetes\_version | The master Kubernetes version | | ||
| network | n/a | | ||
| project\_id | n/a | | ||
| project\_id | Standard test outputs | | ||
| region | n/a | | ||
| service\_account | The default service account used for running nodes. | | ||
| subnetwork | n/a | | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -15,13 +15,14 @@ | |
*/ | ||
|
||
module "acm" { | ||
source = "../../modules/acm" | ||
project_id = var.project_id | ||
location = module.gke.location | ||
cluster_name = module.gke.name | ||
sync_repo = var.acm_sync_repo | ||
sync_branch = var.acm_sync_branch | ||
policy_dir = var.acm_policy_dir | ||
cluster_endpoint = module.gke.endpoint | ||
operator_path = var.operator_path | ||
source = "../../modules/acm" | ||
project_id = var.project_id | ||
location = module.gke.location | ||
cluster_name = module.gke.name | ||
|
||
sync_repo = "[email protected]:GoogleCloudPlatform/csp-config-management.git" | ||
sync_branch = "1.0.0" | ||
policy_dir = "foo-corp" | ||
|
||
secret_type = "ssh" | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file was deleted.
Oops, something went wrong.
Oops, something went wrong.