chore(resourcehierarchy): disambiguate prj-c-logging and prj-c-billin…

…g-logs (#1197)

1-org/envs/shared/README.md

@@ -18,7 +18,7 @@
 | log\_export\_storage\_location | The location of the storage bucket used to export logs. | `string` | `null` | no |
 | log\_export\_storage\_retention\_policy | Configuration of the bucket's data retention policy for how long objects in the bucket should be retained. | <pre>object({<br>    is_locked             = bool<br>    retention_period_days = number<br>  })</pre> | `null` | no |
 | log\_export\_storage\_versioning | (Optional) Toggles bucket versioning, ability to retain a non-current object version when the live object version gets replaced or deleted. | `bool` | `false` | no |
-| project\_budget | Budget configuration for projects.<br>  budget\_amount: The amount to use as the budget.<br>  alert\_spent\_percents: A list of percentages of the budget to alert on when threshold is exceeded.<br>  alert\_pubsub\_topic: The name of the Cloud Pub/Sub topic where budget related messages will be published, in the form of `projects/{project_id}/topics/{topic_id}`.<br>  alert\_spend\_basis: The type of basis used to determine if spend has passed the threshold. Possible choices are `CURRENT_SPEND` or `FORECASTED_SPEND` (default). | <pre>object({<br>    dns_hub_budget_amount                       = optional(number, 1000)<br>    dns_hub_alert_spent_percents                = optional(list(number), [1.2])<br>    dns_hub_alert_pubsub_topic                  = optional(string, null)<br>    dns_hub_budget_alert_spend_basis            = optional(string, "FORECASTED_SPEND")<br>    base_net_hub_budget_amount                  = optional(number, 1000)<br>    base_net_hub_alert_spent_percents           = optional(list(number), [1.2])<br>    base_net_hub_alert_pubsub_topic             = optional(string, null)<br>    base_net_hub_budget_alert_spend_basis       = optional(string, "FORECASTED_SPEND")<br>    base_network_budget_amount                  = optional(number, 1000)<br>    base_network_alert_spent_percents           = optional(list(number), [1.2])<br>    base_network_alert_pubsub_topic             = optional(string, null)<br>    base_network_budget_alert_spend_basis       = optional(string, "FORECASTED_SPEND")<br>    restricted_net_hub_budget_amount            = optional(number, 1000)<br>    restricted_net_hub_alert_spent_percents     = optional(list(number), [1.2])<br>    restricted_net_hub_alert_pubsub_topic       = optional(string, null)<br>    restricted_net_hub_budget_alert_spend_basis = optional(string, "FORECASTED_SPEND")<br>    restricted_network_budget_amount            = optional(number, 1000)<br>    restricted_network_alert_spent_percents     = optional(list(number), [1.2])<br>    restricted_network_alert_pubsub_topic       = optional(string, null)<br>    restricted_network_budget_alert_spend_basis = optional(string, "FORECASTED_SPEND")<br>    interconnect_budget_amount                  = optional(number, 1000)<br>    interconnect_alert_spent_percents           = optional(list(number), [1.2])<br>    interconnect_alert_pubsub_topic             = optional(string, null)<br>    interconnect_budget_alert_spend_basis       = optional(string, "FORECASTED_SPEND")<br>    org_secrets_budget_amount                   = optional(number, 1000)<br>    org_secrets_alert_spent_percents            = optional(list(number), [1.2])<br>    org_secrets_alert_pubsub_topic              = optional(string, null)<br>    org_secrets_budget_alert_spend_basis        = optional(string, "FORECASTED_SPEND")<br>    org_billing_logs_budget_amount              = optional(number, 1000)<br>    org_billing_logs_alert_spent_percents       = optional(list(number), [1.2])<br>    org_billing_logs_alert_pubsub_topic         = optional(string, null)<br>    org_billing_logs_budget_alert_spend_basis   = optional(string, "FORECASTED_SPEND")<br>    org_audit_logs_budget_amount                = optional(number, 1000)<br>    org_audit_logs_alert_spent_percents         = optional(list(number), [1.2])<br>    org_audit_logs_alert_pubsub_topic           = optional(string, null)<br>    org_audit_logs_budget_alert_spend_basis     = optional(string, "FORECASTED_SPEND")<br>    org_kms_budget_amount                       = optional(number, 1000)<br>    org_kms_alert_spent_percents                = optional(list(number), [1.2])<br>    org_kms_alert_pubsub_topic                  = optional(string, null)<br>    org_kms_budget_alert_spend_basis            = optional(string, "FORECASTED_SPEND")<br>    scc_notifications_budget_amount             = optional(number, 1000)<br>    scc_notifications_alert_spent_percents      = optional(list(number), [1.2])<br>    scc_notifications_alert_pubsub_topic        = optional(string, null)<br>    scc_notifications_budget_alert_spend_basis  = optional(string, "FORECASTED_SPEND")<br>  })</pre> | `{}` | no |
+| project\_budget | Budget configuration for projects.<br>  budget\_amount: The amount to use as the budget.<br>  alert\_spent\_percents: A list of percentages of the budget to alert on when threshold is exceeded.<br>  alert\_pubsub\_topic: The name of the Cloud Pub/Sub topic where budget related messages will be published, in the form of `projects/{project_id}/topics/{topic_id}`.<br>  alert\_spend\_basis: The type of basis used to determine if spend has passed the threshold. Possible choices are `CURRENT_SPEND` or `FORECASTED_SPEND` (default). | <pre>object({<br>    dns_hub_budget_amount                       = optional(number, 1000)<br>    dns_hub_alert_spent_percents                = optional(list(number), [1.2])<br>    dns_hub_alert_pubsub_topic                  = optional(string, null)<br>    dns_hub_budget_alert_spend_basis            = optional(string, "FORECASTED_SPEND")<br>    base_net_hub_budget_amount                  = optional(number, 1000)<br>    base_net_hub_alert_spent_percents           = optional(list(number), [1.2])<br>    base_net_hub_alert_pubsub_topic             = optional(string, null)<br>    base_net_hub_budget_alert_spend_basis       = optional(string, "FORECASTED_SPEND")<br>    base_network_budget_amount                  = optional(number, 1000)<br>    base_network_alert_spent_percents           = optional(list(number), [1.2])<br>    base_network_alert_pubsub_topic             = optional(string, null)<br>    base_network_budget_alert_spend_basis       = optional(string, "FORECASTED_SPEND")<br>    restricted_net_hub_budget_amount            = optional(number, 1000)<br>    restricted_net_hub_alert_spent_percents     = optional(list(number), [1.2])<br>    restricted_net_hub_alert_pubsub_topic       = optional(string, null)<br>    restricted_net_hub_budget_alert_spend_basis = optional(string, "FORECASTED_SPEND")<br>    restricted_network_budget_amount            = optional(number, 1000)<br>    restricted_network_alert_spent_percents     = optional(list(number), [1.2])<br>    restricted_network_alert_pubsub_topic       = optional(string, null)<br>    restricted_network_budget_alert_spend_basis = optional(string, "FORECASTED_SPEND")<br>    interconnect_budget_amount                  = optional(number, 1000)<br>    interconnect_alert_spent_percents           = optional(list(number), [1.2])<br>    interconnect_alert_pubsub_topic             = optional(string, null)<br>    interconnect_budget_alert_spend_basis       = optional(string, "FORECASTED_SPEND")<br>    org_secrets_budget_amount                   = optional(number, 1000)<br>    org_secrets_alert_spent_percents            = optional(list(number), [1.2])<br>    org_secrets_alert_pubsub_topic              = optional(string, null)<br>    org_secrets_budget_alert_spend_basis        = optional(string, "FORECASTED_SPEND")<br>    org_billing_export_budget_amount            = optional(number, 1000)<br>    org_billing_export_alert_spent_percents     = optional(list(number), [1.2])<br>    org_billing_export_alert_pubsub_topic       = optional(string, null)<br>    org_billing_export_budget_alert_spend_basis = optional(string, "FORECASTED_SPEND")<br>    org_audit_logs_budget_amount                = optional(number, 1000)<br>    org_audit_logs_alert_spent_percents         = optional(list(number), [1.2])<br>    org_audit_logs_alert_pubsub_topic           = optional(string, null)<br>    org_audit_logs_budget_alert_spend_basis     = optional(string, "FORECASTED_SPEND")<br>    org_kms_budget_amount                       = optional(number, 1000)<br>    org_kms_alert_spent_percents                = optional(list(number), [1.2])<br>    org_kms_alert_pubsub_topic                  = optional(string, null)<br>    org_kms_budget_alert_spend_basis            = optional(string, "FORECASTED_SPEND")<br>    scc_notifications_budget_amount             = optional(number, 1000)<br>    scc_notifications_alert_spent_percents      = optional(list(number), [1.2])<br>    scc_notifications_alert_pubsub_topic        = optional(string, null)<br>    scc_notifications_budget_alert_spend_basis  = optional(string, "FORECASTED_SPEND")<br>  })</pre> | `{}` | no |
 | remote\_state\_bucket | Backend bucket to load Terraform Remote State Data from previous steps. | `string` | n/a | yes |
 | scc\_notification\_filter | Filter used to create the Security Command Center Notification, you can see more details on how to create filters in https://cloud.google.com/security-command-center/docs/how-to-api-filter-notifications#create-filter | `string` | `"state = \"ACTIVE\""` | no |
 | scc\_notification\_name | Name of the Security Command Center Notification. It must be unique in the organization. Run `gcloud scc notifications describe <scc_notification_name> --organization=org_id` to check if it already exists. | `string` | n/a | yes |
@@ -45,7 +45,7 @@
 | logs\_export\_storage\_bucket\_name | The storage bucket for destination of log exports |
 | network\_folder\_name | The network folder name. |
 | org\_audit\_logs\_project\_id | The org audit logs project ID. |
-| org\_billing\_logs\_project\_id | The org billing logs project ID |
+| org\_billing\_export\_project\_id | The org billing export project ID |
 | org\_id | The organization id |
 | org\_kms\_project\_id | The org Cloud Key Management Service (KMS) project ID |
 | org\_secrets\_project\_id | The org secrets project ID |

1-org/envs/shared/iam.tf

@@ -87,13 +87,13 @@ resource "google_project_iam_member" "audit_log_bq_data_viewer" {
 *****************************************/
 
 resource "google_project_iam_member" "billing_bq_user" {
-  project = module.org_billing_logs.project_id
+  project = module.org_billing_export.project_id
   role    = "roles/bigquery.user"
   member  = "group:${local.required_groups["billing_data_users"]}"
 }
 
 resource "google_project_iam_member" "billing_bq_viewer" {
-  project = module.org_billing_logs.project_id
+  project = module.org_billing_export.project_id
   role    = "roles/bigquery.dataViewer"
   member  = "group:${local.required_groups["billing_data_users"]}"
 }

1-org/envs/shared/log_sinks.tf

@@ -91,7 +91,7 @@ module "logs_export" {
 
 resource "google_bigquery_dataset" "billing_dataset" {
   dataset_id    = "billing_data"
-  project       = module.org_billing_logs.project_id
+  project       = module.org_billing_export.project_id
   friendly_name = "GCP Billing Data"
   location      = coalesce(var.billing_export_dataset_location, local.default_region)
 }

1-org/envs/shared/outputs.tf

@@ -49,9 +49,9 @@ output "org_audit_logs_project_id" {
   description = "The org audit logs project ID."
 }
 
-output "org_billing_logs_project_id" {
-  value       = module.org_billing_logs.project_id
-  description = "The org billing logs project ID"
+output "org_billing_export_project_id" {
+  value       = module.org_billing_export.project_id
+  description = "The org billing export project ID"
 }
 
 output "org_secrets_project_id" {

1-org/envs/shared/projects.tf

@@ -29,7 +29,7 @@ locals {
 }
 
 /******************************************
-  Projects for log sinks
+  Project for log sinks
 *****************************************/
 
 module "org_audit_logs" {
@@ -61,33 +61,37 @@ module "org_audit_logs" {
   budget_alert_spend_basis    = var.project_budget.org_audit_logs_budget_alert_spend_basis
 }
 
-module "org_billing_logs" {
+/******************************************
+  Project for billing export
+*****************************************/
+
+module "org_billing_export" {
   source  = "terraform-google-modules/project-factory/google"
   version = "~> 15.0"
 
   random_project_id        = true
   random_project_id_length = 4
   default_service_account  = "deprivilege"
-  name                     = "${local.project_prefix}-c-billing-logs"
+  name                     = "${local.project_prefix}-c-billing-export"
   org_id                   = local.org_id
   billing_account          = local.billing_account
   folder_id                = google_folder.common.id
   activate_apis            = ["logging.googleapis.com", "bigquery.googleapis.com", "billingbudgets.googleapis.com"]
 
   labels = {
     environment       = "common"
-    application_name  = "org-billing-logs"
+    application_name  = "org-billing-export"
     billing_code      = "1234"
     primary_contact   = "example1"
     secondary_contact = "example2"
     business_code     = "shared"
     env_code          = "c"
     vpc               = "none"
   }
-  budget_alert_pubsub_topic   = var.project_budget.org_billing_logs_alert_pubsub_topic
-  budget_alert_spent_percents = var.project_budget.org_billing_logs_alert_spent_percents
-  budget_amount               = var.project_budget.org_billing_logs_budget_amount
-  budget_alert_spend_basis    = var.project_budget.org_billing_logs_budget_alert_spend_basis
+  budget_alert_pubsub_topic   = var.project_budget.org_billing_export_alert_pubsub_topic
+  budget_alert_spent_percents = var.project_budget.org_billing_export_alert_spent_percents
+  budget_amount               = var.project_budget.org_billing_export_budget_amount
+  budget_alert_spend_basis    = var.project_budget.org_billing_export_budget_alert_spend_basis
 }
 
 /******************************************

1-org/envs/shared/variables.tf

@@ -125,10 +125,10 @@ variable "project_budget" {
     org_secrets_alert_spent_percents            = optional(list(number), [1.2])
     org_secrets_alert_pubsub_topic              = optional(string, null)
     org_secrets_budget_alert_spend_basis        = optional(string, "FORECASTED_SPEND")
-    org_billing_logs_budget_amount              = optional(number, 1000)
-    org_billing_logs_alert_spent_percents       = optional(list(number), [1.2])
-    org_billing_logs_alert_pubsub_topic         = optional(string, null)
-    org_billing_logs_budget_alert_spend_basis   = optional(string, "FORECASTED_SPEND")
+    org_billing_export_budget_amount            = optional(number, 1000)
+    org_billing_export_alert_spent_percents     = optional(list(number), [1.2])
+    org_billing_export_alert_pubsub_topic       = optional(string, null)
+    org_billing_export_budget_alert_spend_basis = optional(string, "FORECASTED_SPEND")
     org_audit_logs_budget_amount                = optional(number, 1000)
     org_audit_logs_alert_spent_percents         = optional(list(number), [1.2])
     org_audit_logs_alert_pubsub_topic           = optional(string, null)

README.md

@@ -64,7 +64,7 @@ This will create the following folder and project structure:
 example-organization
 └── fldr-common
     ├── prj-c-logging
-    ├── prj-c-billing-logs
+    ├── prj-c-billing-export
     ├── prj-c-scc
     ├── prj-c-kms
     └── prj-c-secrets
@@ -83,9 +83,9 @@ example-organization
 
 #### Logs
 
-Among the four projects created under the common folder, two projects (`prj-c-logging`, `prj-c-billing-logs`) are used for logging.
-The first one is for organization-wide audit logs, and the second one is for billing logs.
-In both cases, the logs are collected into BigQuery datasets which you can then use for general querying, dashboarding, and reporting. Logs are also exported to Pub/Sub, a Cloud Storage bucket, and a log bucket.
+Under the common folder, a project `prj-c-logging` is used as the destination for organization wide sinks. This includes admin activity audit logs from all projects in your organization and the billing account.
+
+Logs are collected into a logging bucket with a linked BigQuery dataset, which can be used for ad-hoc log investigations, querying, or reporting. Log sinks can also be configured to export to Pub/Sub for exporting to external systems or Cloud Storage for long-term storage.
 
 **Notes**:
 
@@ -248,7 +248,7 @@ After all steps above have been executed, your Google Cloud organization should
 example-organization
 └── fldr-common
     ├── prj-c-logging
-    ├── prj-c-billing-logs
+    ├── prj-c-billing-export
     ├── prj-c-scc
     ├── prj-c-kms
     ├── prj-c-secrets

test/integration/org/org_test.go

@@ -214,7 +214,7 @@ func TestOrg(t *testing.T) {
 			assert.Subset(listCategories, expectedCategories, "notification category subscriptions should be the same")
 
 			//logging
-			billingLogsProjectID := org.GetStringOutput("org_billing_logs_project_id")
+			billingLogsProjectID := org.GetStringOutput("org_billing_export_project_id")
 			billingDatasetName := "billing_data"
 			billingDatasetFullName := fmt.Sprintf("%s:%s", billingLogsProjectID, billingDatasetName)
 			billingDataset := gcloud.Runf(t, "alpha bq datasets describe %s --project %s", billingDatasetName, billingLogsProjectID)
@@ -409,7 +409,7 @@ func TestOrg(t *testing.T) {
 					},
 				},
 				{
-					output: "org_billing_logs_project_id",
+					output: "org_billing_export_project_id",
 					apis: []string{
 						"logging.googleapis.com",
 						"bigquery.googleapis.com",