Skip to content

Commit

Permalink
Rename bu3 to ml (#63)
Browse files Browse the repository at this point in the history
* first commit

* fix ml_infra_projects name

* restore helper/ and test/ original files

* fix 5-app-infra step description

* fix headers

* fix headers

* fix lint

* fix hardcoded
  • Loading branch information
mariammartins authored Jun 24, 2024
1 parent ba94cb9 commit 468d69c
Show file tree
Hide file tree
Showing 122 changed files with 214 additions and 220 deletions.
4 changes: 2 additions & 2 deletions .github/workflows/lint.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -48,10 +48,10 @@ jobs:
env:
DISABLE_TFLINT: 1
ENABLE_PARALLEL: 0
EXCLUDE_LINT_DIRS: \./examples/machine-learning-pipeline|\./docs/assets/terraform|\./5-app-infra/projects/service-catalog/business_unit_3/shared|\./5-app-infra/projects/artifact-publish/business_unit_3/shared
EXCLUDE_LINT_DIRS: \./examples/machine-learning-pipeline|\./docs/assets/terraform|\./5-app-infra/projects/service-catalog/ml_business_unit/shared|\./5-app-infra/projects/artifact-publish/ml_business_unit/shared

- run: docker run --rm -e DISABLE_TFLINT -e ENABLE_PARALLEL -e EXCLUDE_LINT_DIRS -v ${{ github.workspace }}:/workspace ${{ steps.variables.outputs.dev-tools }} /usr/local/bin/test_lint.sh
env:
DISABLE_TFLINT: 1
ENABLE_PARALLEL: 0
EXCLUDE_LINT_DIRS: \./examples/machine-learning-pipeline|\./docs/assets/terraform|\./5-app-infra/projects/service-catalog/business_unit_3/shared|\./5-app-infra/projects/artifact-publish/business_unit_3/shared
EXCLUDE_LINT_DIRS: \./examples/machine-learning-pipeline|\./docs/assets/terraform|\./5-app-infra/projects/service-catalog/ml_business_unit/shared|\./5-app-infra/projects/artifact-publish/ml_business_unit/shared
6 changes: 3 additions & 3 deletions 0-bootstrap/README-GitHub.md
Original file line number Diff line number Diff line change
Expand Up @@ -843,8 +843,8 @@ An environment variable `GOOGLE_IMPERSONATE_SERVICE_ACCOUNT` will be set with th
mv production.auto.example.tfvars production.auto.tfvars
```
1. See any of the envs folder [README.md](../4-projects/business_unit_1/production/README.md#inputs) files for additional information on the values in the `common.auto.tfvars`, `development.auto.tfvars`, `non-production.auto.tfvars`, and `production.auto.tfvars` files.
1. See any of the shared folder [README.md](../4-projects/business_unit_1/shared/README.md#inputs) files for additional information on the values in the `shared.auto.tfvars` file.
1. See any of the envs folder [README.md](../4-projects/ml_business_unit/production/README.md#inputs) files for additional information on the values in the `common.auto.tfvars`, `development.auto.tfvars`, `non-production.auto.tfvars`, and `production.auto.tfvars` files.
1. See any of the shared folder [README.md](../4-projects/ml_business_unit/shared/README.md#inputs) files for additional information on the values in the `shared.auto.tfvars` file.
1. Use `terraform output` to get the backend bucket value from bootstrap output.
Expand All @@ -862,7 +862,7 @@ An environment variable `GOOGLE_IMPERSONATE_SERVICE_ACCOUNT` will be set with th
git commit -m 'Initialize projects repo'
```
1. You need to manually plan and apply only once the `business_unit_1/shared` and `business_unit_2/shared` environments since `development`, `non-production`, and `production` depend on them.
1. You need to manually plan and apply only once the `ml_business_unit/shared` environments since `development`, `non-production`, and `production` depend on them.
1. Use `terraform output` to get the CI/CD project ID and the projects step Terraform Service Account from gcp-bootstrap output.
1. The CI/CD project ID will be used in the [validation](https://cloud.google.com/docs/terraform/policy-validation/quickstart) of the Terraform configuration
Expand Down
4 changes: 2 additions & 2 deletions 0-bootstrap/README-Jenkins.md
Original file line number Diff line number Diff line change
Expand Up @@ -872,8 +872,8 @@ Here you will configure a VPN Network tunnel to enable connectivity between the
mv production.auto.example.tfvars production.auto.tfvars
```

1. See any of the envs folder [README.md](../4-projects/business_unit_1/production/README.md) files for additional information on the values in the `common.auto.tfvars`, `development.auto.tfvars`, `non-production.auto.tfvars`, and `production.auto.tfvars` files.
1. See any of the shared folder [README.md](../4-projects/business_unit_1/shared/README.md) files for additional information on the values in the `shared.auto.tfvars` file.
1. See any of the envs folder [README.md](../4-projects/ml_business_unit/production/README.md) files for additional information on the values in the `common.auto.tfvars`, `development.auto.tfvars`, `non-production.auto.tfvars`, and `production.auto.tfvars` files.
1. See any of the shared folder [README.md](../4-projects/ml_business_unit/shared/README.md) files for additional information on the values in the `shared.auto.tfvars` file.
1. Use `terraform output` to get the backend bucket value from 0-bootstrap output.

```bash
Expand Down
15 changes: 6 additions & 9 deletions 0-bootstrap/README-Terraform-Cloud.md
Original file line number Diff line number Diff line change
Expand Up @@ -771,16 +771,15 @@ An environment variable `GOOGLE_IMPERSONATE_SERVICE_ACCOUNT` will be set with th
mv production.auto.example.tfvars production.auto.tfvars
```
1. See any of the envs folder [README.md](../4-projects/business_unit_1/production/README.md#inputs) files for additional information on the values in the `common.auto.tfvars`, `development.auto.tfvars`, `non-production.auto.tfvars`, and `production.auto.tfvars` files.
1. See any of the shared folder [README.md](../4-projects/business_unit_1/shared/README.md#inputs) files for additional information on the values in the `shared.auto.tfvars` file.
1. See any of the envs folder [README.md](../4-projects/ml_business_unit/production/README.md#inputs) files for additional information on the values in the `common.auto.tfvars`, `development.auto.tfvars`, `non-production.auto.tfvars`, and `production.auto.tfvars` files.
1. See any of the shared folder [README.md](../4-projects/ml_business_unit/shared/README.md#inputs) files for additional information on the values in the `shared.auto.tfvars` file.
1. You need to manually plan and apply only once the `business_unit_1/shared` and `business_unit_2/shared` environments since `development`, `non-production`, and `production` depend on them.
1. You need to manually plan and apply only once the `ml_business_unit/shared` environments since `development`, `non-production`, and `production` depend on them.
1. In order to manually run the apply for shared workspace from your local we need to temporary unset the TFC backend by renaming `envs/shared/backend.tf` to `envs/shared/backend.tf.temporary_disabled`.
```bash
mv business_unit_1/shared/backend.tf business_unit_1/shared/backend.tf.temporary_disabled
mv business_unit_2/shared/backend.tf business_unit_2/shared/backend.tf.temporary_disabled
mv ml_business_unit/shared/backend.tf ml_business_unit/shared/backend.tf.temporary_disabled
```
1. Use `terraform output` to get the CI/CD project ID and the projects step Terraform Service Account from gcp-bootstrap output.
Expand Down Expand Up @@ -831,10 +830,8 @@ An environment variable `GOOGLE_IMPERSONATE_SERVICE_ACCOUNT` will be set with th
1. In order to set the TFC backend for shared workspace we now can rename `envs/shared/backend.tf.temporary_disabled` to `envs/shared/backend.tf` and run `terraform init`. When you're prompted, agree to copy Terraform state to Terraform Cloud.
```bash
mv business_unit_1/shared/backend.tf.temporary_disabled business_unit_1/shared/backend.tf
mv business_unit_2/shared/backend.tf.temporary_disabled business_unit_2/shared/backend.tf
terraform -chdir="business_unit_1/shared/" init
terraform -chdir="business_unit_2/shared/" init
mv ml_business_unit/shared/backend.tf.temporary_disabled ml_business_unit/shared/backend.tf
terraform -chdir="ml_business_unit/shared/" init
```
1. Commit changes
Expand Down
2 changes: 1 addition & 1 deletion 0-bootstrap/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ Hub and Spoke network model. It also sets up the global DNS hub.</td>
</tr>
<tr>
<td><a href="../5-app-infra">5-app-infra</a></td>
<td>Deploy a <a href="https://cloud.google.com/compute/">Compute Engine</a> instance in one of the business unit projects using the infra pipeline setup in 4-projects.</td>
<td>Deploy a service catalog and artifacts pipeline.</td>
</tr>
</tbody>
</table>
Expand Down
23 changes: 7 additions & 16 deletions 0-bootstrap/terraform_cloud.tf.example
Original file line number Diff line number Diff line change
Expand Up @@ -63,14 +63,10 @@ locals {
"3-shared" = { vcs_branch = "production", directory = "/envs/shared" },
},
"proj" = {
"4-bu1-production" = { vcs_branch = "production", directory = "/business_unit_1/production" },
"4-bu1-non-production" = { vcs_branch = "non-production", directory = "/business_unit_1/non-production" },
"4-bu1-development" = { vcs_branch = "development", directory = "/business_unit_1/development" },
"4-bu1-shared" = { vcs_branch = "production", directory = "/business_unit_1/shared" },
"4-bu2-production" = { vcs_branch = "production", directory = "/business_unit_2/production" },
"4-bu2-non-production" = { vcs_branch = "non-production", directory = "/business_unit_2/non-production" },
"4-bu2-development" = { vcs_branch = "development", directory = "/business_unit_2/development" },
"4-bu2-shared" = { vcs_branch = "production", directory = "/business_unit_2/shared" },
"4-ml-production" = { vcs_branch = "production", directory = "/ml_business_unit/production" },
"4-ml-non-production" = { vcs_branch = "non-production", directory = "/ml_business_unit/non-production" },
"4-ml-development" = { vcs_branch = "development", directory = "/ml_business_unit/development" },
"4-ml-shared" = { vcs_branch = "production", directory = "/ml_business_unit/shared" },

},
}
Expand Down Expand Up @@ -218,14 +214,9 @@ resource "tfe_run_trigger" "networks_shared_production" {
sourceable_id = tfe_workspace.main["3-shared"].id
}

resource "tfe_run_trigger" "projects_bu1_shared_production" {
workspace_id = tfe_workspace.main["4-bu1-production"].id
sourceable_id = tfe_workspace.main["4-bu1-shared"].id
}

resource "tfe_run_trigger" "projects_bu2_shared_production" {
workspace_id = tfe_workspace.main["4-bu2-production"].id
sourceable_id = tfe_workspace.main["4-bu2-shared"].id
resource "tfe_run_trigger" "projects_ml_shared_production" {
workspace_id = tfe_workspace.main["4-ml-production"].id
sourceable_id = tfe_workspace.main["4-ml-shared"].id
}

module "tfc_cicd" {
Expand Down
2 changes: 1 addition & 1 deletion 1-org/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ hub-and-spoke network model. It also sets up the global DNS hub.</td>
</tr>
<tr>
<td><a href="../5-app-infra">5-app-infra</a></td>
<td>Deploy a <a href="https://cloud.google.com/compute/">Compute Engine</a> instance in one of the business unit projects using the infra pipeline set up in 4-projects.</td>
<td>Deploy a service catalog and artifacts pipeline.</td>
</tr>
</tbody>
</table>
Expand Down
3 changes: 2 additions & 1 deletion 2-environments/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,8 @@ Hub and Spoke network model. It also sets up the global DNS hub</td>
</tr>
<tr>
<td><a href="../5-app-infra">5-app-infra</a></td>
<td>Deploy a simple <a href="https://cloud.google.com/compute/">Compute Engine</a> instance in one of the business unit projects using the infra pipeline set up in 4-projects.</td>
<td>Deploy service catalog and artifacts pipeline.</td>
<td>Deploy a service catalog and artifacts pipeline.</td>
</tr>
</tbody>
</table>
Expand Down
3 changes: 2 additions & 1 deletion 3-networks-dual-svpc/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,8 @@ Hub and Spoke network model. It also sets up the global DNS hub</td>
</tr>
<tr>
<td><a href="../5-app-infra">5-app-infra</a></td>
<td>Deploy a simple <a href="https://cloud.google.com/compute/">Compute Engine</a> instance in one of the business unit projects using the infra pipeline set up in 4-projects.</td>
<td>Deploy service catalog and artifacts pipeline.</td>
<td>Deploy a service catalog and artifacts pipeline.</td>
</tr>
</tbody>
</table>
Expand Down
18 changes: 10 additions & 8 deletions 4-projects/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ Hub and Spoke network model. It also sets up the global DNS hub</td>
</tr>
<tr>
<td><a href="../5-app-infra">5-app-infra</a></td>
<td>Deploy a simple <a href="https://cloud.google.com/compute/">Compute Engine</a> instance in one of the business unit projects using the infra pipeline set up in 4-projects.</td>
<td>Deploy service catalog and artifacts pipeline.</td>
</tr>
</tbody>
</table>
Expand All @@ -57,10 +57,9 @@ For an overview of the architecture and the parts, see the

The purpose of this step is to set up the folder structure, projects, and infrastructure pipelines for applications that are connected as service projects to the shared VPC created in the previous stage.

For each business unit, a shared `infra-pipeline` project is created along with Cloud Build triggers, CSRs for application infrastructure code and Google Cloud Storage buckets for state storage.
For machine learning business unit, a shared `infra-pipeline` project is created along with Cloud Build triggers, CSRs for application infrastructure code and Google Cloud Storage buckets for state storage.

This step follows the same [conventions](https://github.com/terraform-google-modules/terraform-google-enterprise-genai#branching-strategy) as the Foundation pipeline deployed in [0-bootstrap](https://github.com/terraform-google-modules/terraform-google-enterprise-genai/blob/master/0-bootstrap/README.md).
A custom [workspace](https://github.com/terraform-google-modules/terraform-google-bootstrap/blob/master/modules/tf_cloudbuild_workspace/README.md) (`bu1-example-app`) is created by this pipeline and necessary roles are granted to the Terraform Service Account of this workspace by enabling variable `sa_roles` as shown in this [example](https://github.com/terraform-google-modules/terraform-google-enterprise-genai/blob/master/4-projects/modules/base_env/example_base_shared_vpc_project.tf).

This pipeline is utilized to deploy resources in projects across development/non-production/production in step [5-app-infra](../5-app-infra/README.md).
Other Workspaces can also be created to isolate deployments if needed.
Expand Down Expand Up @@ -122,8 +121,9 @@ Run `terraform output cloudbuild_project_id` in the `0-bootstrap` folder to get
mv production.auto.example.tfvars production.auto.tfvars
```

1. See any of the envs folder [README.md](./business_unit_1/production/README.md) files for additional information on the values in the `common.auto.tfvars`, `development.auto.tfvars`, `non-production.auto.tfvars`, and `production.auto.tfvars` files.
1. See any of the shared folder [README.md](./business_unit_1/shared/README.md) files for additional information on the values in the `shared.auto.tfvars` file.
1. See any of the envs folder [README.md](./ml_business_unit/production/README.md) files for additional information on the values in the `common.auto.tfvars`, `development.auto.tfvars`, `non-production.auto.tfvars`, and `production.auto.tfvars` files.

1. See any of the shared folder [README.md](./ml_business_unit/shared/README.md) files for additional information on the values in the `shared.auto.tfvars` file.

1. Use `terraform output` to get the backend bucket value from 0-bootstrap output.

Expand All @@ -141,8 +141,10 @@ Run `terraform output cloudbuild_project_id` in the `0-bootstrap` folder to get
git commit -m 'Initialize projects repo'
```

1. You need to manually plan and apply only once the `business_unit_1/shared` and `business_unit_2/shared` environments since `development`, `non-production`, and `production` depend on them.
1. You need to manually plan and apply only once the `ml_business_unit/shared` environments since `development`, `non-production`, and `production` depend on them.

1. To use the `validate` option of the `tf-wrapper.sh` script, please follow the [instructions](https://cloud.google.com/docs/terraform/policy-validation/validate-policies#install) to install the terraform-tools component.

1. Use `terraform output` to get the Cloud Build project ID and the projects step Terraform Service Account from 0-bootstrap output. An environment variable `GOOGLE_IMPERSONATE_SERVICE_ACCOUNT` will be set using the Terraform Service Account to enable impersonation.

```bash
Expand Down Expand Up @@ -253,8 +255,8 @@ See `0-bootstrap` [README-GitHub.md](../0-bootstrap/README-GitHub.md#deploying-s
mv production.auto.example.tfvars production.auto.tfvars
```

1. See any of the envs folder [README.md](./business_unit_1/production/README.md) files for additional information on the values in the `common.auto.tfvars`, `development.auto.tfvars`, `non-production.auto.tfvars`, and `production.auto.tfvars` files.
See any of the shared folder [README.md](./business_unit_1/shared/README.md) files for additional information on the values in the `shared.auto.tfvars` file.
1. See any of the envs folder [README.md](./ml_business_unit/production/README.md) files for additional information on the values in the `common.auto.tfvars`, `development.auto.tfvars`, `non-production.auto.tfvars`, and `production.auto.tfvars` files.
See any of the shared folder [README.md](./ml_business_unit/shared/README.md) files for additional information on the values in the `shared.auto.tfvars` file.
Use `terraform output` to get the remote state bucket (the backend bucket used by previous steps) value from `0-bootstrap` output.

```bash
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,6 @@
terraform {
backend "gcs" {
bucket = "UPDATE_PROJECTS_BACKEND"
prefix = "terraform/projects/business_unit_3/production"
prefix = "terraform/projects/ml_business_unit/development"
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
terraform {
cloud {
workspaces {
name = "4-bu3-production"
name = "4-ml-development"
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
# limitations under the License.
#
locals {
repo_name = "bu3-composer"
business_code = "bu3"
business_unit = "business_unit_3"
repo_name = "ml-composer"
business_code = "ml"
business_unit = "ml_business_unit"
}
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,6 @@
terraform {
backend "gcs" {
bucket = "UPDATE_PROJECTS_BACKEND"
prefix = "terraform/projects/business_unit_3/non-production"
prefix = "terraform/projects/ml_business_unit/non-production"
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
terraform {
cloud {
workspaces {
name = "4-bu3-non-production"
name = "4-ml-non-production"
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
# limitations under the License.
#
locals {
repo_name = "bu3-composer"
business_code = "bu3"
business_unit = "business_unit_3"
repo_name = "ml-composer"
business_code = "ml"
business_unit = "ml_business_unit"
}
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,6 @@
terraform {
backend "gcs" {
bucket = "UPDATE_PROJECTS_BACKEND"
prefix = "terraform/projects/business_unit_3/development"
prefix = "terraform/projects/ml_business_unit/production"
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
terraform {
cloud {
workspaces {
name = "4-bu3-development"
name = "4-ml-production"
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
# limitations under the License.
#
locals {
repo_name = "bu3-composer"
business_code = "bu3"
business_unit = "business_unit_3"
repo_name = "ml-composer"
business_code = "ml"
business_unit = "ml_business_unit"
}
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,6 @@
terraform {
backend "gcs" {
bucket = "UPDATE_PROJECTS_BACKEND"
prefix = "terraform/projects/business_unit_3/shared"
prefix = "terraform/projects/ml_business_unit/shared"
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
terraform {
cloud {
workspaces {
name = "4-bu2-shared"
name = "4-ml-shared"
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,9 @@

locals {
repo_names = [
"bu3-artifact-publish",
"bu3-service-catalog",
"bu3-machine-learning",
"ml-artifact-publish",
"ml-service-catalog",
"ml-machine-learning",
]
}

Expand Down Expand Up @@ -50,7 +50,7 @@ module "app_infra_cloudbuild_project" {
billing_code = "1234"
primary_contact = "[email protected]"
secondary_contact = "[email protected]"
business_code = "bu3"
business_code = "ml"
}

module "infra_pipelines" {
Expand Down
Loading

0 comments on commit 468d69c

Please sign in to comment.