Skip to content

Commit

Permalink
update run terraform locally
Browse files Browse the repository at this point in the history
  • Loading branch information
@caetano-colin
caetano-colin committed Jul 3, 2024
1 parent ba58c08 commit a737bc0
Showing 1 changed file with 56 additions and 37 deletions.
93 changes: 56 additions & 37 deletions 5-app-infra/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -443,11 +443,23 @@ The pipeline also listens for changes made to `plan`, `development`, `non-produc

#### Artifacts Application

1. The next instructions assume that you are at the same level of the `terraform-google-enterprise-genai` folder. Change into `5-app-infra` folder, copy the Terraform wrapper script and ensure it can be executed.
1. Create `ml-artifact-publish` directory at the same level as `terraform-google-enterprise-genai`.

```bash
cd terraform-google-enterprise-genai/5-app-infra/projects/artifact-publish/
cp ../../../build/tf-wrapper.sh .
mkdir ml-artifact-publish
```

1. Navigate into the repo, change to non-main branch and copy contents of genAI to new repo.
All subsequent steps assume you are running them from the ml-artifact-publish directory.
If you run them from another directory, adjust your copy paths accordingly.

```bash
cd ml-artifact-publish/

cp -RT ../terraform-google-enterprise-genai/5-app-infra/projects/artifact-publish/ .
cp -R ../terraform-google-enterprise-genai/5-app-infra/modules/ ./modules
cp ../terraform-google-enterprise-genai/build/cloudbuild-tf-* .
cp ../terraform-google-enterprise-genai/build/tf-wrapper.sh .
chmod 755 ./tf-wrapper.sh
```

Expand All @@ -462,7 +474,7 @@ The pipeline also listens for changes made to `plan`, `development`, `non-produc
1. Use `terraform output` to get the project backend bucket value from 0-bootstrap.

```bash
export remote_state_bucket=$(terraform -chdir="../../../0-bootstrap/" output -raw projects_gcs_bucket_tfstate)
export remote_state_bucket=$(terraform -chdir="../terraform-google-enterprise-genai/0-bootstrap/" output -raw projects_gcs_bucket_tfstate)
echo "remote_state_bucket = ${remote_state_bucket}"
sed -i "s/REMOTE_STATE_BUCKET/${remote_state_bucket}/" ./common.auto.tfvars
```
Expand All @@ -475,10 +487,10 @@ The pipeline also listens for changes made to `plan`, `development`, `non-produc
member="user:$(gcloud auth list --filter="status=ACTIVE" --format="value(account)")"
echo ${member}

project_id=$(terraform -chdir="../../../4-projects/ml_business_unit/shared/" output -raw cloudbuild_project_id)
project_id=$(terraform -chdir="../terraform-google-enterprise-genai/4-projects/ml_business_unit/shared/" output -raw cloudbuild_project_id)
echo ${project_id}

terraform_sa=$(terraform -chdir="../../../4-projects/ml_business_unit/shared/" output -json terraform_service_accounts | jq '."ml-artifact-publish"' --raw-output)
terraform_sa=$(terraform -chdir="../terraform-google-enterprise-genai/4-projects/ml_business_unit/shared/" output -json terraform_service_accounts | jq '."ml-artifact-publish"' --raw-output)
echo ${terraform_sa}

gcloud iam service-accounts add-iam-policy-binding ${terraform_sa} --project ${project_id} --member="${member}" --role="roles/iam.serviceAccountTokenCreator"
Expand All @@ -487,7 +499,7 @@ The pipeline also listens for changes made to `plan`, `development`, `non-produc
1. Update `backend.tf` with your bucket from the infra pipeline output.

```bash
export backend_bucket=$(terraform -chdir="../../../4-projects/ml_business_unit/shared/" output -json state_buckets | jq '."ml-artifact-publish"' --raw-output)
export backend_bucket=$(terraform -chdir="../terraform-google-enterprise-genai/4-projects/ml_business_unit/shared/" output -json state_buckets | jq '."ml-artifact-publish"' --raw-output)
echo "backend_bucket = ${backend_bucket}"

for i in `find -name 'backend.tf'`; do sed -i "s/UPDATE_APP_INFRA_BUCKET/${backend_bucket}/" $i; done
Expand Down Expand Up @@ -535,14 +547,12 @@ After executing this stage, unset the `GOOGLE_IMPERSONATE_SERVICE_ACCOUNT` envir
unset GOOGLE_IMPERSONATE_SERVICE_ACCOUNT
```

1. `cd` out of the `artifact-publish`.
1. `cd` out of the repository.

```bash
cd
```

1. Navigate to the project that was output from `${ARTIFACT_PROJECT_ID}` in Google's Cloud Console to view the first run of images being built.

#### Configuring Cloud Source Repository of Artifact Application

1. The next instructions assume that you are at the same level of the `terraform-google-enterprise-genai` folder.
Expand Down Expand Up @@ -587,64 +597,73 @@ unset GOOGLE_IMPERSONATE_SERVICE_ACCOUNT

#### Service Catalog Configuration

1. The next instructions assume that you are at the same level of the `terraform-google-enterprise-genai` folder. Change into `5-app-infra` folder, copy the Terraform wrapper script and ensure it can be executed.

1. Create `ml-service-catalog` directory at the same level as `terraform-google-enterprise-genai`.

```bash
cd terraform-google-enterprise-genai/5-app-infra/projects/service-catalog/
cp ../../../build/tf-wrapper.sh .
mkdir ml-service-catalog
```

1. Navigate into the repo, change to non-main branch and copy contents of foundation to new repo.
All subsequent steps assume you are running them from the ml-service-catalog directory.
If you run them from another directory, adjust your copy paths accordingly.

```bash
cd ml-service-catalog

cp -RT ../terraform-google-enterprise-genai/5-app-infra/projects/service-catalog/ .
cp -R ../terraform-google-enterprise-genai/5-app-infra/modules/ ./modules
cp ../terraform-google-enterprise-genai/build/cloudbuild-tf-* .
cp ../terraform-google-enterprise-genai/build/tf-wrapper.sh .
chmod 755 ./tf-wrapper.sh
```

1. Rename `common.auto.example.tfvars` files to `common.auto.tfvars`.
1. Rename `common.auto.example.tfvars` to `common.auto.tfvars`.

```bash
mv common.auto.example.tfvars common.auto.tfvars
```

1. Update `common.auto.tfvars` file with values from your environment.

1. Use `terraform output` to get the project backend bucket value from 0-bootstrap.
1. Update the file with values from your environment and 0-bootstrap. See any of the business unit 1 envs folders [README.md](./ml_business_unit/production/README.md) files for additional information on the values in the `common.auto.tfvars` file.

```bash
export remote_state_bucket=$(terraform -chdir="../../../0-bootstrap/" output -raw projects_gcs_bucket_tfstate)
export remote_state_bucket=$(terraform -chdir="../terraform-google-enterprise-genai/0-bootstrap/" output -raw projects_gcs_bucket_tfstate)
echo "remote_state_bucket = ${remote_state_bucket}"
sed -i "s/REMOTE_STATE_BUCKET/${remote_state_bucket}/" ./common.auto.tfvars
```

1. Provide the user that will be running `./tf-wrapper.sh` the Service Account Token Creator role to the ml Terraform service account.

1. Provide the user permissions to run the terraform locally with the `serviceAccountTokenCreator` permission.
1. Update `backend.tf` with your bucket from the infra pipeline output.

```bash
(cd ../../../4-projects && git checkout production && ./tf-wrapper init shared)

member="user:$(gcloud auth list --filter="status=ACTIVE" --format="value(account)")"
echo ${member}

project_id=$(terraform -chdir="../../../4-projects/ml_business_unit/shared/" output -raw cloudbuild_project_id)
echo ${project_id}

terraform_sa=$(terraform -chdir="../../../4-projects/ml_business_unit/shared/" output -json terraform_service_accounts | jq '."ml-service-catalog"' --raw-output)
echo ${terraform_sa}
export backend_bucket=$(terraform -chdir="../gcp-projects/ml_business_unit/shared/" output -json state_buckets | jq '."ml-service-catalog"' --raw-output)
echo "backend_bucket = ${backend_bucket}"

gcloud iam service-accounts add-iam-policy-binding ${terraform_sa} --project ${project_id} --member="${member}" --role="roles/iam.serviceAccountTokenCreator"
for i in `find -name 'backend.tf'`; do sed -i "s/UPDATE_APP_INFRA_BUCKET/${backend_bucket}/" $i; done
```

1. Update the `log_bucket` variable with the value of the `logs_export_storage_bucket_name`.

```bash
```bash
export log_bucket=$(terraform -chdir="../gcp-org/envs/shared" output -raw logs_export_storage_bucket_name)
echo "log_bucket = ${log_bucket}"
sed -i "s/REPLACE_LOG_BUCKET/${log_bucket}/" ./common.auto.tfvars
```

1. Update `backend.tf` with your bucket from the infra pipeline output.
1. Provide the user permissions to run the terraform locally with the `serviceAccountTokenCreator` permission.

```bash
export backend_bucket=$(terraform -chdir="../../../4-projects/ml_business_unit/shared/" output -json state_buckets | jq '."ml-service-catalog"' --raw-output)
echo "backend_bucket = ${backend_bucket}"
(cd ../terraform-google-enterprise-genai/4-projects && git checkout production && ./tf-wrapper init shared)

for i in `find -name 'backend.tf'`; do sed -i "s/UPDATE_APP_INFRA_BUCKET/${backend_bucket}/" $i; done
member="user:$(gcloud auth list --filter="status=ACTIVE" --format="value(account)")"
echo ${member}

project_id=$(terraform -chdir="../terraform-google-enterprise-genai/4-projects/ml_business_unit/shared/" output -raw cloudbuild_project_id)
echo ${project_id}

terraform_sa=$(terraform -chdir="../terraform-google-enterprise-genai/4-projects/ml_business_unit/shared/" output -json terraform_service_accounts | jq '."ml-service-catalog"' --raw-output)
echo ${terraform_sa}

gcloud iam service-accounts add-iam-policy-binding ${terraform_sa} --project ${project_id} --member="${member}" --role="roles/iam.serviceAccountTokenCreator"
```

We will now deploy each of our environments (development/production/non-production) using this script.
Expand Down

0 comments on commit a737bc0

Please sign in to comment.