part 5 edits

Author: askmeegs

5-hosted-resources/bigquery/helloworld-dataset.yaml

@@ -2,7 +2,5 @@ apiVersion: bigquery.cnrm.cloud.google.com/v1beta1
 kind: BigQueryDataset
 metadata:
   name: helloworld
-  annotations:
-    configsync.gke.io/cluster-name-selector: cymbal-admin
 spec:
   friendlyName: hello-world

5-hosted-resources/bigquery/mock-dataset.yaml

@@ -2,14 +2,12 @@ apiVersion: bigquery.cnrm.cloud.google.com/v1beta1
 kind: BigQueryJob
 metadata:
   name: cymbal-mock-load-job
-  annotations:
-    configsync.gke.io/cluster-name-selector: cymbal-admin
 spec:
   location: "US"
   jobTimeoutMs: "600000"
   load:
     sourceUris:
-      - "gs://PROJECT_ID/cymbal-mock-transactions.csv"
+      - "gs://krm-test11/cymbal-mock-transactions.csv"
     destinationTable:
       tableRef:
         name: cymbalmocktable
@@ -32,17 +30,13 @@ apiVersion: bigquery.cnrm.cloud.google.com/v1beta1
 kind: BigQueryDataset
 metadata:
   name: cymbalmockdataset
-  annotations:
-    configsync.gke.io/cluster-name-selector: cymbal-admin
 spec:
   friendlyName: cymbal-mock-dataset
 ---
 apiVersion: bigquery.cnrm.cloud.google.com/v1beta1
 kind: BigQueryTable
 metadata:
   name: cymbalmocktable
-  annotations:
-    configsync.gke.io/cluster-name-selector: cymbal-admin
 spec:
   friendlyName: cymbal-mock-table
   datasetRef:

5-hosted-resources/configconnector.yaml

@@ -5,4 +5,4 @@ metadata:
   name: configconnector.core.cnrm.cloud.google.com
 spec:
   mode: cluster
-  googleServiceAccount: "cymbal-admin-kcc@PROJECT_ID.iam.gserviceaccount.com"
+  googleServiceAccount: "cymbal-admin-kcc@krm-test11.iam.gserviceaccount.com"

5-hosted-resources/partA-config-connector.md

@@ -59,7 +59,12 @@ Expected output:
 ✅ Finished installing Config Connector on all clusters.
 ```
 
-This script grants Config Connector (running in the GKE cluster) the IAM permissions it needs to create and update GCP resources in your project, and deploys Config Connector onto the cluster. 
+This script does the following: 
+1. Uninstalls Config Sync from the `cymbal-admin` cluster. (**Note** - currently a bug causes Config Sync and Config Connector to conflict with each other. In the future, you will be able to use Config Sync to deploy your Config Connector resources)
+2. Grants Config Connector IAM permissions to lifecycle GCP resources in your project
+3. Installs Config Connector on the `cymbal-admin` cluster. 
+
+grants Config Connector (running in the GKE cluster) the IAM permissions it needs to create and update GCP resources in your project, and deploys Config Connector onto the cluster. 
 
 ### 5. **Verify that Config Connector is installed on the admin cluster.**
 
@@ -86,6 +91,7 @@ Let's start with a basic example of creating a GCP-hosted resource using Config
 ![screenshot](screenshots/secadmin-gce.jpg)
 
 
+
 ### 1. **View the GCE KRM resources.** 
 
 ```
@@ -119,7 +125,7 @@ spec:
 
 This KRM resource defines one Compute Engine instance, along with a Compute Disk and some networking resources. Notice how the KRM looks a lot like a Deployment YAML - it has a name, metadata with some labels, and a spec, with info specifically about a GCE instance. Config Connector knows how to read this `ComputeEngine` resource type, and take action on it - in this case, create a Compute Engine instance in our GCP project. 
 
-### 2. Apply the Compute Engine resources to the admin cluster. **⚠️ Note** - this demo shows applying the cloud-hosted KRM resources manually with kubectl, due to an ongoing bug between Config Sync and Config Connector. But in an ideal scenario, we use Config Sync to sync the Config Connector KRM just like we did policies. 
+### 2. Apply the Compute Engine resources to the admin cluster. 
 
 ```
 kubectx cymbal-admin
@@ -154,7 +160,9 @@ iamserviceaccount.iam.cnrm.cloud.google.com/inst-dep-cloudmachine   5m58s   True
 
 Note - it may take a few minutes for the resources to be created. In the meantime, you may see `UpdateFailed` or `DependencyNotReady`. This is expected. 
 
-### 4. **Open the Cloud Console and navigate to Compute Engine > VM Instances. Filter on `name:secadmin`. You should see the new GCE instance in the list.** 
+### 4. **Open the Cloud Console and navigate to [Compute Engine > VM Instances](https://console.cloud.google.com/compute/instances). 
+
+Filter on `name:secadmin`. You should see the new GCE instance in the list.** 
 
 ![screenshots](screenshots/secadmin-gce-console.png)
 

5-hosted-resources/partB-cloud-policies.md

@@ -21,10 +21,10 @@ export PROJECT_ID=<your-project-id>
 ### 2. **View the mock transaction dataset.** This is a 1000-line CSV file, whose fields mimic the data currently stored in the Cloud SQL `ledger_db` today. 
 
 ```
-head bigquery/cymbal-mock-transactions.csv 
+cat bigquery/cymbal-mock-transactions.csv 
 ```
 
-Expected output: 
+Expected output (truncated): 
 
 ```
 transaction_id,from_account,to_account,amount,timestamp,user_agent
@@ -39,7 +39,9 @@ transaction_id,from_account,to_account,amount,timestamp,user_agent
 9,145870222,311667375,$39.17,4/15/2021,Chrome/51.0.2704.103
 ```
 
-### 3. **Verify that you have the gsutil tool installed** - this comes bundled with the gcloud command. [Install the tool](https://cloud.google.com/storage/docs/gsutil_install) if it's not in your PATH. 
+### 3. **Verify that you have the gsutil tool installed** - this comes bundled with the gcloud command. 
+
+[Install the tool](https://cloud.google.com/storage/docs/gsutil_install) if it's not in your PATH. 
 
 ```
 gsutil version 
@@ -48,7 +50,7 @@ gsutil version
 Expected output: 
 
 ```
-gsutil version: 4.61
+gsutil version: 4.64
 ```
 
 ### 4. **Create a Cloud Storage bucket in your project, called `datasets`.**
@@ -60,7 +62,7 @@ gsutil mb -c standard gs://$PROJECT_ID-datasets
 Expected output: 
 
 ```
-Creating gs://krm-test-5-datasets/...
+Creating gs://krm-test11-datasets/...
 ```
 
 ### 5. **Upload the mock transaction data to Cloud Storage.**
@@ -85,13 +87,11 @@ cat bigquery/mock-dataset.yaml
 
 Expected output: 
 
-```
+```YAML
 apiVersion: bigquery.cnrm.cloud.google.com/v1beta1
 kind: BigQueryJob
 metadata:
   name: cymbal-mock-load-job
-  annotations:
-    configsync.gke.io/cluster-name-selector: cymbal-admin
 spec:
   location: "US"
   jobTimeoutMs: "600000"
@@ -120,17 +120,13 @@ apiVersion: bigquery.cnrm.cloud.google.com/v1beta1
 kind: BigQueryDataset
 metadata:
   name: cymbalmockdataset
-  annotations:
-    configsync.gke.io/cluster-name-selector: cymbal-admin
 spec:
   friendlyName: cymbal-mock-dataset
 ---
 apiVersion: bigquery.cnrm.cloud.google.com/v1beta1
 kind: BigQueryTable
 metadata:
   name: cymbalmocktable
-  annotations:
-    configsync.gke.io/cluster-name-selector: cymbal-admin
 spec:
   friendlyName: cymbal-mock-table
   datasetRef:
@@ -189,14 +185,15 @@ Now let's come back to the restrictions we outlined at the beginning of this sec
 
 This file defines a constraint template for `BigQueryDatasetAllowName`, and a constraint of type `BigQueryDatasetAllowName`, which together allow only one BigQuery dataset in the policy repo.
 
+View the custom Constraint Template: 
+
 ```
 cat bigquery/constraint-template.yaml
-cat bigquery/constraint.yaml
 ```
 
 Expected output: 
 
-```
+```YAML
 apiVersion: templates.gatekeeper.sh/v1beta1
 kind: ConstraintTemplate
 metadata:
@@ -228,6 +225,24 @@ spec:
     allowedName: cymbalmockdataset
 ```
 
+View the Constraint, using the custom Constraint Template: 
+
+```
+cat bigquery/constraint.yaml 
+```
+
+Expected output: 
+
+```YAML
+apiVersion: constraints.gatekeeper.sh/v1beta1
+kind: BigQueryDatasetAllowName
+metadata:
+  name: bigquery-allow-mock-only
+spec:
+  parameters:
+    allowedName: cymbalmockdatase
+```
+
 ### 12. **Apply the Constraint and Constraint Template** to the admin cluster.
 
 ```

5-hosted-resources/partC-existing-resources.md

@@ -14,12 +14,14 @@ config-connector version
 Expected output: 
 
 ```
-1.46.0
+1.53.0
 ```
 
 ### 2. **View the Cloud SQL KRM export script.** 
 
-This script generates static KRM resource files (YAML) for the Cloud SQL development database. (Although these steps only show KRM for the development DB, we could do the same for the staging and production databases as well.)
+This script generates static KRM resource files (YAML) for the Cloud SQL development database.
+
+**Note**: Although these steps only show KRM for the development DB, we could do the same for the staging and production databases as well.
 
 ```
 cat cloudsql/generate-cloudsql-krm.sh 
@@ -118,11 +120,12 @@ spec:
   resourceID: ledger-db
 ```
 
-These KRM files represent the live state of your Cloud SQL resources, originally created using Terraform. (You will see your PROJECT_ID next to `cnrm.cloud.google.com/project-id`.)
+These KRM files represent the live state of your Cloud SQL resources, originally created using Terraform. (You should see your PROJECT_ID next to `cnrm.cloud.google.com/project-id`.)
 
 ### 5. **Apply the Cloud SQL KRM resources to the cymbal-admin cluster.**
 
 ```
+kubectx cymbal-admin
 kubectl apply -f cloudsql/projects/$PROJECT_ID/SQLInstance/us-east1/cymbal-dev.yaml
 kubectl apply -f cloudsql/projects/$PROJECT_ID/SQLInstance/cymbal-dev/SQLDatabase/accounts-db.yaml
 kubectl apply -f cloudsql/projects/$PROJECT_ID/SQLInstance/cymbal-dev/SQLDatabase/ledger-db.yaml
@@ -157,7 +160,7 @@ NAME                                               AGE   READY   STATUS     STAT
 sqlinstance.sql.cnrm.cloud.google.com/cymbal-dev   42s   True    UpToDate   10s
 ```
 
-### 7. **Open the Cloud Console and navigate to Cloud SQL**. 
+### 7. **Open the Cloud Console and [navigate to Cloud SQL](https://console.cloud.google.com/sql)**. 
 
 Notice how in the list, the `cymbal-dev` cluster now has a new label, `managed-by-cnrm: true`. This indicates that this SQL Instance is now under the management umbrella of Config Connector. 
 
@@ -178,24 +181,25 @@ Notice how in the list, the `cymbal-dev` cluster now has a new label, `managed-b
 
 ## Wrap-up 
 
-If you made it this far, great work - you just completed several challenging demos that explored the Kubernetes Resource Model with multiple angles, developer personas, products, and tools. 
+If you made it this far, **great work**! You just completed several demos that explored the Kubernetes Resource Model with multiple developer personas, products, and tools in mind. 
 
 Let's summarize the key takeaways from all 5 demos: 
 
 - **Building a platform is hard**, especially in the cloud, especially when you have multiple Kubernetes clusters in play, on top of hosted resources.  
-- **KRM is one way to manage your Cloud and Kubernetes config**, but it's not the only way - Demo 1 showed us how to do it with Terraform. 
 - KRM is a great way to manage resources because Kubernetes is constantly running a **control loop** to make sure your **desired** state matches the **actual** cluster state. We saw this in action both for core Kubernetes API resources (Demo 2 / for instance, Deployments that keep Pods alive) and hosted Cloud resources (Demo 5 / via Config Connector)
 - **KRM promotes a "GitOps" model** where you keep all your configuration in Git, and sync it down to multiple clusters at once.  
 - Policy Controller, together with Config Sync, allow you to impose custom policies on your KRM resources, both at deploy-time and during CI/CD (Demo 4). These **policies allow you to set fine-grained controls** on different resource types, to ensure compliance within your org. 
 - **KRM / the Kubernetes API can lifecycle resources that run outside a Kubernetes cluster.** We saw how Config Connector, running inside the admin cluster, created and updated resources in Google Cloud. 
+- **KRM is one way to manage your Cloud and Kubernetes resources**, but it's not the only way - for instance, we set up the initial demo environment with Terraform in part 1. The benefit of putting more and more resources in a KRM format is that you have a single language and toolchain for your infrastructure, in and outside of Kubernetes.  
+
 
-Hopefully you learned a thing or two from these demos- really, we've only just scratched the surface of what KRM can do. All the "learn more" links across Parts 1-5 are available in the [README of this repo](/README.md).
+Hopefully you learned a thing or two from these demos- really, we've only just scratched the surface of what KRM can do. 
 
-And another set of resources to learn more about KRM, its design principles, and other helpful tools, see: **https://github.com/askmeegs/learn-krm**.
+For a list of additional resources to learn the Kubernetes Resource Model, check out: **https://github.com/askmeegs/learn-krm**.
 
 ## ⭐️ We'd love your feedback! ⭐️
 
-### 🗳 [If you have a moment, please fill out this short survey](https://forms.gle/pUX2DPW9fxgDMwEw8) to share your thoughts on this demo! Thank you! 
+### 🗳 [If you have a moment, please fill out this short survey](https://forms.gle/pUX2DPW9fxgDMwEw8) to share your thoughts on these demos. Thank you! 
 
 ## Cleaning up
 

5-hosted-resources/remove-cs-spec.yaml

@@ -0,0 +1,11 @@
+applySpecVersion: 1
+spec:
+  configSync:
+    enabled: false
+    sourceFormat: unstructured
+    syncRepo: https://github.com/GITHUB_USERNAME/cymbalbank-policy
+    syncBranch: main
+    secretType: none
+    policyDir: /
+  policyController:
+    enabled: true

5-hosted-resources/setup-config-connector.sh

@@ -9,44 +9,46 @@ fi
 gcloud config set project $PROJECT_ID
 export SERVICE_ACCOUNT_NAME="cymbal-admin-kcc"
 
-echo "☁️ Creating a Google Service Account (GSA) for Config Connector..."
-gcloud iam service-accounts create $SERVICE_ACCOUNT_NAME
-
-echo "☁️ Granting the GSA cloud resource management permissions..." 
-gcloud projects add-iam-policy-binding ${PROJECT_ID} \
-    --member="serviceAccount:${SERVICE_ACCOUNT_NAME}@${PROJECT_ID}.iam.gserviceaccount.com" \
-    --role="roles/owner"
-
-echo "☁️ Connecting your Google Service Account to the Kubernetes Service Account (KSA) that Config Connector uses..."
-gcloud iam service-accounts add-iam-policy-binding \
-cymbal-admin-kcc@$PROJECT_ID.iam.gserviceaccount.com \
-    --member="serviceAccount:$PROJECT_ID.svc.id.goog[cnrm-system/cnrm-controller-manager]" \
-    --role="roles/iam.workloadIdentityUser"
-
-# Populate configconnector.yaml
-echo "☁️ Populating and deploying configconnector.yaml with your GSA info..."
-sed -i "s/PROJECT_ID/$PROJECT_ID/g" configconnector.yaml
+kcc_project_setup() {
+    echo "☁️ Creating a Google Service Account (GSA) for Config Connector..."
+    gcloud iam service-accounts create $SERVICE_ACCOUNT_NAME
+
+    echo "☁️ Granting the GSA cloud resource management permissions..." 
+    gcloud projects add-iam-policy-binding ${PROJECT_ID} \
+        --member="serviceAccount:${SERVICE_ACCOUNT_NAME}@${PROJECT_ID}.iam.gserviceaccount.com" \
+        --role="roles/owner"
+
+    echo "☁️ Connecting your Google Service Account to the Kubernetes Service Account (KSA) that Config Connector uses..."
+    gcloud iam service-accounts add-iam-policy-binding \
+    cymbal-admin-kcc@$PROJECT_ID.iam.gserviceaccount.com \
+        --member="serviceAccount:$PROJECT_ID.svc.id.goog[cnrm-system/cnrm-controller-manager]" \
+        --role="roles/iam.workloadIdentityUser"
+}
 
+uninstall_config_sync() {
+    CLUSTER_NAME=$1 
+    CLUSTER_ZONE=$2
+    echo "☸️ Uninstalling Config Sync and Policy Controller: $CLUSTER_NAME, zone: $CLUSTER_ZONE"
+    gcloud alpha container hub config-management apply \
+    --membership=$CLUSTER_NAME \
+    --config=remove-cs-spec.yaml \
+    --project=$PROJECT_ID
+}
 
-install_kcc () {
+install_config_connector () {
     CLUSTER_NAME=$1 
     CLUSTER_ZONE=$2 
-    echo "☸️ Installing Config Connector: $CLUSTER_NAME, zone: $CLUSTER_ZONE" \
+    echo "☸️ Installing Config Connector: $CLUSTER_NAME, zone: $CLUSTER_ZONE" 
 
     kubectx $CLUSTER_NAME
 
-    # Apply configconnector.yaml 
-    echo "☁️ Installing the Config Connector controller..." 
     kubectl apply -f configconnector.yaml
     kubectl annotate namespace default cnrm.cloud.google.com/project-id=$PROJECT_ID 
 }
 
-# Note - due to an ongoing bug, Config Sync and Config Connector can't be installed 
-# on GKE at the same time. So dev/staging/prod have Config Sync, and admin has config connector. 
 
-# install_kcc "cymbal-dev" "us-east1-c" 
-# install_kcc "cymbal-staging" "us-central1-a" 
-# install_kcc "cymbal-prod" "us-west1-a"
-install_kcc "cymbal-admin" "us-central1-f"
+kcc_project_setup
+uninstall_config_sync "cymbal-admin" "us-central1-f"
+install_config_connector "cymbal-admin" "us-central1-f"
 
 echo "✅ Finished installing Config Connector on the admin cluster."

cleanup.sh

@@ -26,9 +26,9 @@ echo "🗑 Deleting Config Connector-managed resources (Compute Engine, BigQuery
 kubectx cymbal-admin 
 kubectl delete -f 5-hosted-resources/bigquery/mock-dataset.yaml 
 kubectl delete -f 5-hosted-resources/compute-engine/instance.yaml
-kubectl delete -f cloudsql/projects/$PROJECT_ID/SQLInstance/us-east1/cymbal-dev.yaml
-kubectl delete -f cloudsql/projects/$PROJECT_ID/SQLInstance/cymbal-dev/SQLDatabase/accounts-db.yaml
-kubectl delete -f cloudsql/projects/$PROJECT_ID/SQLInstance/cymbal-dev/SQLDatabase/ledger-db.yaml
+kubectl delete -f 5-hosted-resources/cloudsql/projects/$PROJECT_ID/SQLInstance/us-east1/cymbal-dev.yaml
+kubectl delete -f 5-hosted-resources/cloudsql/projects/$PROJECT_ID/SQLInstance/cymbal-dev/SQLDatabase/accounts-db.yaml
+kubectl delete -f 5-hosted-resources/cloudsql/projects/$PROJECT_ID/SQLInstance/cymbal-dev/SQLDatabase/ledger-db.yaml
 
 echo "💤 Sleeping 30 seconds to allow Config Connector to delete Cloud Resources..."
 sleep 30 
@@ -46,4 +46,4 @@ gcloud iam service-accounts delete cymbal-gsa@$PROJECT_ID.iam.gserviceaccount.co
 # Terraform destroy (GKE clusters, Git repos, Cloud Build permissions, Cloud SQL databases)
 echo "🗑 Running terraform destroy to remove GKE clusters, Cloud SQL databases..." 
 cd 1-setup/
-terraform destroy
+terraform destroy -var-file="base-env/terraform.tfvars" --auto-approve base-env