Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rename bu3 to ml #63

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/lint.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -48,10 +48,10 @@ jobs:
env:
DISABLE_TFLINT: 1
ENABLE_PARALLEL: 0
EXCLUDE_LINT_DIRS: \./examples/machine-learning-pipeline|\./docs/assets/terraform|\./5-app-infra/projects/service-catalog/business_unit_3/shared|\./5-app-infra/projects/artifact-publish/business_unit_3/shared
EXCLUDE_LINT_DIRS: \./examples/machine-learning-pipeline|\./docs/assets/terraform|\./5-app-infra/projects/service-catalog/ml_business_unit/shared|\./5-app-infra/projects/artifact-publish/ml_business_unit/shared

- run: docker run --rm -e DISABLE_TFLINT -e ENABLE_PARALLEL -e EXCLUDE_LINT_DIRS -v ${{ github.workspace }}:/workspace ${{ steps.variables.outputs.dev-tools }} /usr/local/bin/test_lint.sh
env:
DISABLE_TFLINT: 1
ENABLE_PARALLEL: 0
EXCLUDE_LINT_DIRS: \./examples/machine-learning-pipeline|\./docs/assets/terraform|\./5-app-infra/projects/service-catalog/business_unit_3/shared|\./5-app-infra/projects/artifact-publish/business_unit_3/shared
EXCLUDE_LINT_DIRS: \./examples/machine-learning-pipeline|\./docs/assets/terraform|\./5-app-infra/projects/service-catalog/ml_business_unit/shared|\./5-app-infra/projects/artifact-publish/ml_business_unit/shared
6 changes: 3 additions & 3 deletions 0-bootstrap/README-GitHub.md
Original file line number Diff line number Diff line change
Expand Up @@ -843,8 +843,8 @@ An environment variable `GOOGLE_IMPERSONATE_SERVICE_ACCOUNT` will be set with th
mv production.auto.example.tfvars production.auto.tfvars
```

1. See any of the envs folder [README.md](../4-projects/business_unit_1/production/README.md#inputs) files for additional information on the values in the `common.auto.tfvars`, `development.auto.tfvars`, `non-production.auto.tfvars`, and `production.auto.tfvars` files.
1. See any of the shared folder [README.md](../4-projects/business_unit_1/shared/README.md#inputs) files for additional information on the values in the `shared.auto.tfvars` file.
1. See any of the envs folder [README.md](../4-projects/ml_business_unit/production/README.md#inputs) files for additional information on the values in the `common.auto.tfvars`, `development.auto.tfvars`, `non-production.auto.tfvars`, and `production.auto.tfvars` files.
1. See any of the shared folder [README.md](../4-projects/ml_business_unit/shared/README.md#inputs) files for additional information on the values in the `shared.auto.tfvars` file.

1. Use `terraform output` to get the backend bucket value from bootstrap output.

Expand All @@ -862,7 +862,7 @@ An environment variable `GOOGLE_IMPERSONATE_SERVICE_ACCOUNT` will be set with th
git commit -m 'Initialize projects repo'
```

1. You need to manually plan and apply only once the `business_unit_1/shared` and `business_unit_2/shared` environments since `development`, `non-production`, and `production` depend on them.
1. You need to manually plan and apply only once the `ml_business_unit/shared` environments since `development`, `non-production`, and `production` depend on them.

1. Use `terraform output` to get the CI/CD project ID and the projects step Terraform Service Account from gcp-bootstrap output.
1. The CI/CD project ID will be used in the [validation](https://cloud.google.com/docs/terraform/policy-validation/quickstart) of the Terraform configuration
Expand Down
4 changes: 2 additions & 2 deletions 0-bootstrap/README-Jenkins.md
Original file line number Diff line number Diff line change
Expand Up @@ -872,8 +872,8 @@ Here you will configure a VPN Network tunnel to enable connectivity between the
mv production.auto.example.tfvars production.auto.tfvars
```

1. See any of the envs folder [README.md](../4-projects/business_unit_1/production/README.md) files for additional information on the values in the `common.auto.tfvars`, `development.auto.tfvars`, `non-production.auto.tfvars`, and `production.auto.tfvars` files.
1. See any of the shared folder [README.md](../4-projects/business_unit_1/shared/README.md) files for additional information on the values in the `shared.auto.tfvars` file.
1. See any of the envs folder [README.md](../4-projects/ml_business_unit/production/README.md) files for additional information on the values in the `common.auto.tfvars`, `development.auto.tfvars`, `non-production.auto.tfvars`, and `production.auto.tfvars` files.
1. See any of the shared folder [README.md](../4-projects/ml_business_unit/shared/README.md) files for additional information on the values in the `shared.auto.tfvars` file.
1. Use `terraform output` to get the backend bucket value from 0-bootstrap output.

```bash
Expand Down
15 changes: 6 additions & 9 deletions 0-bootstrap/README-Terraform-Cloud.md
Original file line number Diff line number Diff line change
Expand Up @@ -771,16 +771,15 @@ An environment variable `GOOGLE_IMPERSONATE_SERVICE_ACCOUNT` will be set with th
mv production.auto.example.tfvars production.auto.tfvars
```

1. See any of the envs folder [README.md](../4-projects/business_unit_1/production/README.md#inputs) files for additional information on the values in the `common.auto.tfvars`, `development.auto.tfvars`, `non-production.auto.tfvars`, and `production.auto.tfvars` files.
1. See any of the shared folder [README.md](../4-projects/business_unit_1/shared/README.md#inputs) files for additional information on the values in the `shared.auto.tfvars` file.
1. See any of the envs folder [README.md](../4-projects/ml_business_unit/production/README.md#inputs) files for additional information on the values in the `common.auto.tfvars`, `development.auto.tfvars`, `non-production.auto.tfvars`, and `production.auto.tfvars` files.
1. See any of the shared folder [README.md](../4-projects/ml_business_unit/shared/README.md#inputs) files for additional information on the values in the `shared.auto.tfvars` file.

1. You need to manually plan and apply only once the `business_unit_1/shared` and `business_unit_2/shared` environments since `development`, `non-production`, and `production` depend on them.
1. You need to manually plan and apply only once the `ml_business_unit/shared` environments since `development`, `non-production`, and `production` depend on them.

1. In order to manually run the apply for shared workspace from your local we need to temporary unset the TFC backend by renaming `envs/shared/backend.tf` to `envs/shared/backend.tf.temporary_disabled`.

```bash
mv business_unit_1/shared/backend.tf business_unit_1/shared/backend.tf.temporary_disabled
mv business_unit_2/shared/backend.tf business_unit_2/shared/backend.tf.temporary_disabled
mv ml_business_unit/shared/backend.tf ml_business_unit/shared/backend.tf.temporary_disabled
```

1. Use `terraform output` to get the CI/CD project ID and the projects step Terraform Service Account from gcp-bootstrap output.
Expand Down Expand Up @@ -831,10 +830,8 @@ An environment variable `GOOGLE_IMPERSONATE_SERVICE_ACCOUNT` will be set with th
1. In order to set the TFC backend for shared workspace we now can rename `envs/shared/backend.tf.temporary_disabled` to `envs/shared/backend.tf` and run `terraform init`. When you're prompted, agree to copy Terraform state to Terraform Cloud.

```bash
mv business_unit_1/shared/backend.tf.temporary_disabled business_unit_1/shared/backend.tf
mv business_unit_2/shared/backend.tf.temporary_disabled business_unit_2/shared/backend.tf
terraform -chdir="business_unit_1/shared/" init
terraform -chdir="business_unit_2/shared/" init
mv ml_business_unit/shared/backend.tf.temporary_disabled ml_business_unit/shared/backend.tf
terraform -chdir="ml_business_unit/shared/" init
```

1. Commit changes
Expand Down
2 changes: 1 addition & 1 deletion 0-bootstrap/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ Hub and Spoke network model. It also sets up the global DNS hub.</td>
</tr>
<tr>
<td><a href="../5-app-infra">5-app-infra</a></td>
<td>Deploy a <a href="https://cloud.google.com/compute/">Compute Engine</a> instance in one of the business unit projects using the infra pipeline setup in 4-projects.</td>
<td>Deploy a service catalog and artifacts pipeline.</td>
</tr>
</tbody>
</table>
Expand Down
23 changes: 7 additions & 16 deletions 0-bootstrap/terraform_cloud.tf.example
Original file line number Diff line number Diff line change
Expand Up @@ -63,14 +63,10 @@ locals {
"3-shared" = { vcs_branch = "production", directory = "/envs/shared" },
},
"proj" = {
"4-bu1-production" = { vcs_branch = "production", directory = "/business_unit_1/production" },
"4-bu1-non-production" = { vcs_branch = "non-production", directory = "/business_unit_1/non-production" },
"4-bu1-development" = { vcs_branch = "development", directory = "/business_unit_1/development" },
"4-bu1-shared" = { vcs_branch = "production", directory = "/business_unit_1/shared" },
"4-bu2-production" = { vcs_branch = "production", directory = "/business_unit_2/production" },
"4-bu2-non-production" = { vcs_branch = "non-production", directory = "/business_unit_2/non-production" },
"4-bu2-development" = { vcs_branch = "development", directory = "/business_unit_2/development" },
"4-bu2-shared" = { vcs_branch = "production", directory = "/business_unit_2/shared" },
"4-ml-production" = { vcs_branch = "production", directory = "/ml_business_unit/production" },
"4-ml-non-production" = { vcs_branch = "non-production", directory = "/ml_business_unit/non-production" },
"4-ml-development" = { vcs_branch = "development", directory = "/ml_business_unit/development" },
"4-ml-shared" = { vcs_branch = "production", directory = "/ml_business_unit/shared" },

},
}
Expand Down Expand Up @@ -218,14 +214,9 @@ resource "tfe_run_trigger" "networks_shared_production" {
sourceable_id = tfe_workspace.main["3-shared"].id
}

resource "tfe_run_trigger" "projects_bu1_shared_production" {
workspace_id = tfe_workspace.main["4-bu1-production"].id
sourceable_id = tfe_workspace.main["4-bu1-shared"].id
}

resource "tfe_run_trigger" "projects_bu2_shared_production" {
workspace_id = tfe_workspace.main["4-bu2-production"].id
sourceable_id = tfe_workspace.main["4-bu2-shared"].id
resource "tfe_run_trigger" "projects_ml_shared_production" {
workspace_id = tfe_workspace.main["4-ml-production"].id
sourceable_id = tfe_workspace.main["4-ml-shared"].id
}

module "tfc_cicd" {
Expand Down
2 changes: 1 addition & 1 deletion 1-org/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ hub-and-spoke network model. It also sets up the global DNS hub.</td>
</tr>
<tr>
<td><a href="../5-app-infra">5-app-infra</a></td>
<td>Deploy a <a href="https://cloud.google.com/compute/">Compute Engine</a> instance in one of the business unit projects using the infra pipeline set up in 4-projects.</td>
<td>Deploy a service catalog and artifacts pipeline.</td>
</tr>
</tbody>
</table>
Expand Down
3 changes: 2 additions & 1 deletion 2-environments/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,8 @@ Hub and Spoke network model. It also sets up the global DNS hub</td>
</tr>
<tr>
<td><a href="../5-app-infra">5-app-infra</a></td>
<td>Deploy a simple <a href="https://cloud.google.com/compute/">Compute Engine</a> instance in one of the business unit projects using the infra pipeline set up in 4-projects.</td>
<td>Deploy service catalog and artifacts pipeline.</td>
<td>Deploy a service catalog and artifacts pipeline.</td>
</tr>
</tbody>
</table>
Expand Down
3 changes: 2 additions & 1 deletion 3-networks-dual-svpc/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,8 @@ Hub and Spoke network model. It also sets up the global DNS hub</td>
</tr>
<tr>
<td><a href="../5-app-infra">5-app-infra</a></td>
<td>Deploy a simple <a href="https://cloud.google.com/compute/">Compute Engine</a> instance in one of the business unit projects using the infra pipeline set up in 4-projects.</td>
<td>Deploy service catalog and artifacts pipeline.</td>
<td>Deploy a service catalog and artifacts pipeline.</td>
</tr>
</tbody>
</table>
Expand Down
18 changes: 10 additions & 8 deletions 4-projects/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ Hub and Spoke network model. It also sets up the global DNS hub</td>
</tr>
<tr>
<td><a href="../5-app-infra">5-app-infra</a></td>
<td>Deploy a simple <a href="https://cloud.google.com/compute/">Compute Engine</a> instance in one of the business unit projects using the infra pipeline set up in 4-projects.</td>
<td>Deploy service catalog and artifacts pipeline.</td>
</tr>
</tbody>
</table>
Expand All @@ -57,10 +57,9 @@ For an overview of the architecture and the parts, see the

The purpose of this step is to set up the folder structure, projects, and infrastructure pipelines for applications that are connected as service projects to the shared VPC created in the previous stage.

For each business unit, a shared `infra-pipeline` project is created along with Cloud Build triggers, CSRs for application infrastructure code and Google Cloud Storage buckets for state storage.
For machine learning business unit, a shared `infra-pipeline` project is created along with Cloud Build triggers, CSRs for application infrastructure code and Google Cloud Storage buckets for state storage.

This step follows the same [conventions](https://github.com/terraform-google-modules/terraform-google-enterprise-genai#branching-strategy) as the Foundation pipeline deployed in [0-bootstrap](https://github.com/terraform-google-modules/terraform-google-enterprise-genai/blob/master/0-bootstrap/README.md).
A custom [workspace](https://github.com/terraform-google-modules/terraform-google-bootstrap/blob/master/modules/tf_cloudbuild_workspace/README.md) (`bu1-example-app`) is created by this pipeline and necessary roles are granted to the Terraform Service Account of this workspace by enabling variable `sa_roles` as shown in this [example](https://github.com/terraform-google-modules/terraform-google-enterprise-genai/blob/master/4-projects/modules/base_env/example_base_shared_vpc_project.tf).

This pipeline is utilized to deploy resources in projects across development/non-production/production in step [5-app-infra](../5-app-infra/README.md).
Other Workspaces can also be created to isolate deployments if needed.
Expand Down Expand Up @@ -122,8 +121,9 @@ Run `terraform output cloudbuild_project_id` in the `0-bootstrap` folder to get
mv production.auto.example.tfvars production.auto.tfvars
```

1. See any of the envs folder [README.md](./business_unit_1/production/README.md) files for additional information on the values in the `common.auto.tfvars`, `development.auto.tfvars`, `non-production.auto.tfvars`, and `production.auto.tfvars` files.
1. See any of the shared folder [README.md](./business_unit_1/shared/README.md) files for additional information on the values in the `shared.auto.tfvars` file.
1. See any of the envs folder [README.md](./ml_business_unit/production/README.md) files for additional information on the values in the `common.auto.tfvars`, `development.auto.tfvars`, `non-production.auto.tfvars`, and `production.auto.tfvars` files.

1. See any of the shared folder [README.md](./ml_business_unit/shared/README.md) files for additional information on the values in the `shared.auto.tfvars` file.

1. Use `terraform output` to get the backend bucket value from 0-bootstrap output.

Expand All @@ -141,8 +141,10 @@ Run `terraform output cloudbuild_project_id` in the `0-bootstrap` folder to get
git commit -m 'Initialize projects repo'
```

1. You need to manually plan and apply only once the `business_unit_1/shared` and `business_unit_2/shared` environments since `development`, `non-production`, and `production` depend on them.
1. You need to manually plan and apply only once the `ml_business_unit/shared` environments since `development`, `non-production`, and `production` depend on them.

1. To use the `validate` option of the `tf-wrapper.sh` script, please follow the [instructions](https://cloud.google.com/docs/terraform/policy-validation/validate-policies#install) to install the terraform-tools component.

1. Use `terraform output` to get the Cloud Build project ID and the projects step Terraform Service Account from 0-bootstrap output. An environment variable `GOOGLE_IMPERSONATE_SERVICE_ACCOUNT` will be set using the Terraform Service Account to enable impersonation.

```bash
Expand Down Expand Up @@ -253,8 +255,8 @@ See `0-bootstrap` [README-GitHub.md](../0-bootstrap/README-GitHub.md#deploying-s
mv production.auto.example.tfvars production.auto.tfvars
```

1. See any of the envs folder [README.md](./business_unit_1/production/README.md) files for additional information on the values in the `common.auto.tfvars`, `development.auto.tfvars`, `non-production.auto.tfvars`, and `production.auto.tfvars` files.
See any of the shared folder [README.md](./business_unit_1/shared/README.md) files for additional information on the values in the `shared.auto.tfvars` file.
1. See any of the envs folder [README.md](./ml_business_unit/production/README.md) files for additional information on the values in the `common.auto.tfvars`, `development.auto.tfvars`, `non-production.auto.tfvars`, and `production.auto.tfvars` files.
See any of the shared folder [README.md](./ml_business_unit/shared/README.md) files for additional information on the values in the `shared.auto.tfvars` file.
Use `terraform output` to get the remote state bucket (the backend bucket used by previous steps) value from `0-bootstrap` output.

```bash
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,6 @@
terraform {
backend "gcs" {
bucket = "UPDATE_PROJECTS_BACKEND"
prefix = "terraform/projects/business_unit_3/production"
prefix = "terraform/projects/ml_business_unit/development"
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
terraform {
cloud {
workspaces {
name = "4-bu3-production"
name = "4-ml-development"
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
# limitations under the License.
#
locals {
repo_name = "bu3-composer"
business_code = "bu3"
business_unit = "business_unit_3"
repo_name = "ml-composer"
business_code = "ml"
business_unit = "ml_business_unit"
}
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,6 @@
terraform {
backend "gcs" {
bucket = "UPDATE_PROJECTS_BACKEND"
prefix = "terraform/projects/business_unit_3/non-production"
prefix = "terraform/projects/ml_business_unit/non-production"
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
terraform {
cloud {
workspaces {
name = "4-bu3-non-production"
name = "4-ml-non-production"
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
# limitations under the License.
#
locals {
repo_name = "bu3-composer"
business_code = "bu3"
business_unit = "business_unit_3"
repo_name = "ml-composer"
business_code = "ml"
business_unit = "ml_business_unit"
}
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,6 @@
terraform {
backend "gcs" {
bucket = "UPDATE_PROJECTS_BACKEND"
prefix = "terraform/projects/business_unit_3/development"
prefix = "terraform/projects/ml_business_unit/production"
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
terraform {
cloud {
workspaces {
name = "4-bu3-development"
name = "4-ml-production"
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
# limitations under the License.
#
locals {
repo_name = "bu3-composer"
business_code = "bu3"
business_unit = "business_unit_3"
repo_name = "ml-composer"
business_code = "ml"
business_unit = "ml_business_unit"
}
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,6 @@
terraform {
backend "gcs" {
bucket = "UPDATE_PROJECTS_BACKEND"
prefix = "terraform/projects/business_unit_3/shared"
prefix = "terraform/projects/ml_business_unit/shared"
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
terraform {
cloud {
workspaces {
name = "4-bu2-shared"
name = "4-ml-shared"
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,9 @@

locals {
repo_names = [
"bu3-artifact-publish",
"bu3-service-catalog",
"bu3-machine-learning",
"ml-artifact-publish",
"ml-service-catalog",
"ml-machine-learning",
]
}

Expand Down Expand Up @@ -50,7 +50,7 @@ module "app_infra_cloudbuild_project" {
billing_code = "1234"
primary_contact = "[email protected]"
secondary_contact = "[email protected]"
business_code = "bu3"
business_code = "ml"
}

module "infra_pipelines" {
Expand Down
Loading