TEF upstream git resync and terraform provider updates cadence #387

fmichaelobrien · 2024-04-16T13:16:40Z

TEF V5 tracking https://github.com/terraform-google-modules/terraform-example-foundation/labels/v5.0

see fork reference
https://github.com/CloudLandingZone/terraform-example-foundation

Branch: https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/tree/gh387-upstream-sync

PRs merged to the terraform-example-foundation upstream repo

20240426 fix: Add roles and service enablements to bootstrap project terraform-google-modules/terraform-example-foundation#1175

terraform-example-foundation upstream resync procedure - copy not fork

working out a solution - which is like the ADO sync in Add Azure DevOps ADO as build and repository option #399 (comment)
branch: gh387-upstream-sync https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/tree/gh387-upstream-sync

cd _upstream
git clone https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding.git
cd pbmm-on-gcp-onboarding 
git checkout gh387-upstream-sync 

michaelobrien@mbp7 pbmm-on-gcp-onboarding % git status
On branch gh387-upstream-sync
Your branch is up to date with 'origin/gh387-upstream-sync'.

git remote add upstream https://github.com/terraform-google-modules/terraform-example-foundation.git
git fetch upstream
git merge upstream/master gh387-upstream-sync
fatal: refusing to merge unrelated histories

Indirect terraform provider updates

@nbugden thank you for the bucket names fix
Merge in 63 character bucket naming standard fix indirectly via the next terraform init - that pull in in the latest generated terraform-google-provider via the magic modules compile

GoogleCloudPlatform/magic-modules#10426
for
hashicorp/terraform-provider-google#17831

following
previous pr terraform-google-modules/terraform-google-cloud-storage#308

closes
terraform-google-modules/terraform-example-foundation#1169
terraform-google-modules/terraform-google-cloud-storage#307

upgrade the provider on the next init (1.3.10) and 1.6 (we are not using the default 1.7.5 yet)
https://github.com/hashicorp/terraform-provider-google?tab=readme-ov-file#upgrading-the-provider

terraform init -upgrade

Sync procedure jira needed for #376

The text was updated successfully, but these errors were encountered:

fmichaelobrien · 2024-04-24T12:17:23Z

Indirect

Direct

https://github.com/terraform-google-modules/terraform-example-foundation/pull/1181/files
https://github.com/terraform-google-modules/terraform-example-foundation/pull/1196/files
nonproduction labels chore: Fix project labels terraform-google-modules/terraform-example-foundation#1107
VPCSC dry run mode enhancement (with 3-networks-hub-and-spoke pending) - fix(VPCSC): enable dryrun mode terraform-google-modules/terraform-example-foundation#1210
https://github.com/terraform-google-modules/terraform-example-foundation/pull/1199/files

obriensystems · 2024-04-24T15:57:26Z

Todo: keep the repos relatively in sync
nonproduction vs non-production for example

obriensystems · 2024-04-25T19:53:04Z

  nonproduction_folder_name         = data.terraform_remote_state.env_nonproduction.outputs.env_folder

resource "gitlab_branch_protection" "non_production" {
 tf_apply_branches = ["development", "non\\-production", "production"]

Align with PR from 3 weeks ago upstream
terraform-google-modules/terraform-example-foundation#1107
terraform-google-modules/terraform-example-foundation@eb28ecb#diff-446d88326ec6ad0370d30957f6546694390e92cc997a810e60783bfeb016efc1L308

obriensystems · 2024-05-03T14:20:16Z

prepare for TEF to PBMM sync
fix the main branch first - by re-adding cloud build support
#421

michaelobrien@mbp7 terraform-example-foundation % git pull
remote: Enumerating objects: 120, done.
remote: Counting objects: 100% (119/119), done.
remote: Compressing objects: 100% (80/80), done.
remote: Total 120 (delta 52), reused 78 (delta 33), pack-reused 1
Receiving objects: 100% (120/120), 103.94 KiB | 1.07 MiB/s, done.
Resolving deltas: 100% (52/52), completed with 7 local objects.
From https://github.com/terraform-google-modules/terraform-example-foundation
   a63dc88..034b995  master                                                         -> origin/master
 * [new branch]      dependabot/go_modules/helpers/foundation-deployer/github.com/hashicorp/go-getter-1.7.4 -> origin/dependabot/go_modules/helpers/foundation-deployer/github.com/hashicorp/go-getter-1.7.4
 + 5653048...218bb21 dependabot/go_modules/helpers/foundation-deployer/golang.org/x/net-0.23.0 -> origin/dependabot/go_modules/helpers/foundation-deployer/golang.org/x/net-0.23.0  (forced update)
 * [new branch]      dependabot/go_modules/test/integration/github.com/hashicorp/go-getter-1.7.4 -> origin/dependabot/go_modules/test/integration/github.com/hashicorp/go-getter-1.7.4
 + 4d6d15d...79f5c67 dependabot/go_modules/test/integration/golang.org/x/net-0.23.0 -> origin/dependabot/go_modules/test/integration/golang.org/x/net-0.23.0  (forced update)
 + d4c27d3...4318c63 release-please--branches--master                               -> origin/release-please--branches--master  (forced update)
   ddeae49..1c2e4a8  renovate/go-modules-andor-dev-tools                            -> origin/renovate/go-modules-andor-dev-tools
 + 38e9441...7a1e347 renovate/google-cloud-security-center-8.x                      -> origin/renovate/google-cloud-security-center-8.x  (forced update)
Updating a63dc88..034b995
Fast-forward
 .github/workflows/go-lint.yaml                                     |  6 +++---
 0-bootstrap/README.md                                              | 14 ++++++++++++++
 1-org/envs/shared/terraform.example.tfvars                         |  4 ----
 2-environments/envs/nonproduction/terraform.tfvars                 |  2 +-
 4-projects/business_unit_1/nonproduction/nonproduction.auto.tfvars |  2 +-
 4-projects/business_unit_2/nonproduction/nonproduction.auto.tfvars |  2 +-
 README.md                                                          |  2 +-
 go.work                                                            |  2 +-
 helpers/foundation-deployer/README.md                              |  2 +-
 helpers/foundation-deployer/go.mod                                 |  2 +-
 test/integration/go.mod                                            |  2 +-
 test/integration/networks/networks_test.go                         |  2 +-
 test/integration/testutils/retry.go                                |  3 +++
 13 files changed, 29 insertions(+), 16 deletions(-)

keep

ssc-notification randomization for multi-tenant deployment https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/pull/385/files
bucket name length - https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/pull/386/files

Changes

Adjust

Help with upstream sync

revert non-production back to tef nonproduction
0-bootstrap/

Sync

michaelobrien@mbp7 _upstream % cd pbmm-on-gcp-onboarding 
michaelobrien@mbp7 pbmm-on-gcp-onboarding % git checkout main
Already on 'main'
Your branch is up to date with 'origin/main'.
michaelobrien@mbp7 pbmm-on-gcp-onboarding % git checkout gh387-upstream-sync
Switched to branch 'gh387-upstream-sync'
Your branch is up to date with 'origin/gh387-upstream-sync'.
michaelobrien@mbp7 pbmm-on-gcp-onboarding % git merge main
Updating b9fe8d39..d7344f5f
Fast-forward
 README.md | 21 ++++++++++++++++++++-
 1 file changed, 20 insertions(+), 1 deletion(-)
michaelobrien@mbp7 pbmm-on-gcp-onboarding % 
ichaelobrien@mbp7 pbmm-on-gcp-onboarding % git push origin gh387-upstream-sync
Total 0 (delta 0), reused 0 (delta 0), pack-reused 0
remote: 
remote: GitHub found 5 vulnerabilities on GoogleCloudPlatform/pbmm-on-gcp-onboarding's default branch (2 critical, 3 moderate). To find out more, visit:
remote:      https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/security/dependabot
remote: 
To https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding.git
   b9fe8d39..d7344f5f  gh387-upstream-sync -> gh387-upstream-sync

fmichaelobrien · 2024-05-04T23:20:06Z

20240504: Repo state: CB/CSR are the default for the main branch

The main branch is ready for Cloud Build / Cloud Source Repositories out of the box. For ADO support this is in queue via #399 after a TEF upstream merge via #387

obriensystems · 2024-05-11T20:04:47Z

Sync 20240511 version of the following master branch
https://github.com/terraform-google-modules/terraform-example-foundation/
via branch terraform-google-modules/terraform-example-foundation#1199
with last PR
terraform-google-modules/terraform-example-foundation#1199

into
https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/tree/gh387-upstream-sync

0-bootstrap, 1-org done

tef minor bug for label changes in https://github.com/terraform-google-modules/terraform-example-foundation/pull/1199/files#diff-d6697e7c916ba73d6ae87ff4b1ce67cabc9b9738ab31c9ba582e2a3218982838L279

https://github.com/terraform-google-modules/terraform-example-foundation/blob/master/1-org/envs/shared/projects.tf#L253
match
https://github.com/terraform-google-modules/terraform-example-foundation/blob/master/1-org/envs/shared/projects.tf#L237

application_name = "org-dns-hub"

application_name = "org-net-dns"

and
-net-interconnect

just like in
https://github.com/terraform-google-modules/terraform-example-foundation/blob/master/1-org/envs/shared/projects.tf#L295

3-networks-hub-and-spoke

2 symlinks modified in error during the non-production to nonproduction rename/refactor - the change is to ../common - which itself is a symlink so the double symlink works. The other 6 of the 2x4 symlinks are unmodified as ../../
see change in #1107
https://github.com/terraform-google-modules/terraform-example-foundation/pull/1107/files#diff-bcfdfb477dea3f75d9d515aa61ce4f3836b223f11d6e4234122778963fbfef4a

https://github.com/terraform-google-modules/terraform-example-foundation/blob/master/3-networks-hub-and-spoke/envs/nonproduction/access_context.auto.tfvars

-../../access_context.auto.tfvars
+../shared/access_context.auto.tfvars

there is a 2nd symlink regression in the same directory in
https://github.com/terraform-google-modules/terraform-example-foundation/blob/master/3-networks-hub-and-spoke/envs/nonproduction/common.auto.tfvars

-../../common.auto.tfvars
+../shared/common.auto.tfvars

I'll raise an issue shortly with the fix after I test a full upstream sync of the TEF.

obriensystems · 2024-05-12T21:06:43Z

130 files - ignore the 2nd email CLA, now testing the branch before prepping for main to master - after a couple PRs in queue
#435

fmichaelobrien · 2024-05-12T21:18:11Z

I completed the TEF upstream sync - 130 changes to get up to 20240511 main branch status.
Details in the PR below
#435
I will hold off on merging the branch below until I test a clean deploy - and also likely wait for your PR - so I will take the merge of your changes - instead of the other way around.
https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/tree/gh387-upstream-sync
for
#387

fmichaelobrien · 2024-05-13T01:59:59Z

as part of terraform-google-modules/terraform-example-foundation#1229
fix symlinks

fmichaelobrien-macbookpro:terraform-example-foundation fmichaelobrien$ git status
On branch gh1229-fix-symlinks
Your branch is up to date with 'origin/gh1229-fix-symlinks'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
	modified:   3-networks-hub-and-spoke/envs/nonproduction/access_context.auto.tfvars
	modified:   3-networks-hub-and-spoke/envs/nonproduction/common.auto.tfvars

no changes added to commit (use "git add" and/or "git commit -a")
fmichaelobrien-macbookpro:terraform-example-foundation fmichaelobrien$ git diff
diff --git a/3-networks-hub-and-spoke/envs/nonproduction/access_context.auto.tfvars b/3-networks-hub-and-spoke/envs/nonproduction/access_context.auto.tfvars
index 59c7aa1..b0cccce 120000
--- a/3-networks-hub-and-spoke/envs/nonproduction/access_context.auto.tfvars
+++ b/3-networks-hub-and-spoke/envs/nonproduction/access_context.auto.tfvars
@@ -1 +1 @@
-../shared/access_context.auto.tfvars
\ No newline at end of file
+../../access_context.auto.tfvars
\ No newline at end of file
diff --git a/3-networks-hub-and-spoke/envs/nonproduction/common.auto.tfvars b/3-networks-hub-and-spoke/envs/nonproduction/common.auto.tfvars
index 3316647..39aaa46 120000
--- a/3-networks-hub-and-spoke/envs/nonproduction/common.auto.tfvars
+++ b/3-networks-hub-and-spoke/envs/nonproduction/common.auto.tfvars
@@ -1 +1 @@
-../shared/common.auto.tfvars
\ No newline at end of file
+../../common.auto.tfvars
\ No newline at end of file

fmichaelobrien · 2024-05-14T12:33:13Z

pending June upstream sync

Direct

fix!: use bootstrap.outputs.common_config as default region terraform-google-modules/terraform-example-foundation#1181

Indirect

fmichaelobrien · 2024-05-17T12:26:35Z

missing
https://github.com/terraform-google-modules/terraform-example-foundation/blob/master/test/restore_tf_files.sh
architecture.md

fmichaelobrien · 2024-05-17T18:54:48Z

Merge duplicate environment removal in a patch just after sunday's pull
terraform-google-modules/terraform-example-foundation@061e769

checked - already removed
https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/blob/gh387-upstream-sync/1-org/envs/shared/projects.tf#L294

obriensystems · 2024-05-17T19:00:25Z

pull 20240517:1500

chaelobrien@mbp7 terraform-example-foundation % git pull
remote: Enumerating objects: 180, done.
remote: Counting objects: 100% (180/180), done.
remote: Compressing objects: 100% (93/93), done.
remote: Total 180 (delta 96), reused 148 (delta 78), pack-reused 0
Receiving objects: 100% (180/180), 116.39 KiB | 3.75 MiB/s, done.
Resolving deltas: 100% (96/96), completed with 15 local objects.
From https://github.com/terraform-google-modules/terraform-example-foundation
   034b995..0a69eac  master                                                        -> origin/master
 + 4318c63...e686f63 release-please--branches--master                              -> origin/release-please--branches--master  (forced update)
 + 1c2e4a8...64f8d52 renovate/go-modules-andor-dev-tools                           -> origin/renovate/go-modules-andor-dev-tools  (forced update)
 * [new branch]      renovate/googlecloudplatform-lb-internal-google-6.x           -> origin/renovate/googlecloudplatform-lb-internal-google-6.x
 * [new branch]      renovate/terraform-google-modules-cloud-storage-google-6.x    -> origin/renovate/terraform-google-modules-cloud-storage-google-6.x
 * [new branch]      renovate/terraform-google-modules-project-factory-google-15.x -> origin/renovate/terraform-google-modules-project-factory-google-15.x
Updating 034b995..0a69eac
Fast-forward
 .github/workflows/go-lint.yaml                                       |  8 ++++----
 .github/workflows/go-test.yaml                                       | 51 +++++++++++++++++++++++++++++++++++++++++++++++++++
 1-org/envs/shared/projects.tf                                        | 13 ++++++-------
 3-networks-dual-svpc/envs/shared/dns-hub.tf                          | 16 ++++++++--------
 3-networks-dual-svpc/envs/shared/interconnect.tf.example             |  2 +-
 3-networks-dual-svpc/envs/shared/partner_interconnect.tf.example     |  2 +-
 3-networks-dual-svpc/modules/base_shared_vpc/dns.tf                  |  2 +-
 3-networks-dual-svpc/modules/partner_interconnect/README.md          |  2 +-
 3-networks-dual-svpc/modules/restricted_shared_vpc/dns.tf            |  2 +-
 3-networks-hub-and-spoke/envs/shared/dns-hub.tf                      | 16 ++++++++--------
 3-networks-hub-and-spoke/envs/shared/interconnect.tf.example         |  2 +-
 3-networks-hub-and-spoke/envs/shared/partner_interconnect.tf.example |  2 +-
 3-networks-hub-and-spoke/modules/base_shared_vpc/dns.tf              |  2 +-
 3-networks-hub-and-spoke/modules/partner_interconnect/README.md      |  2 +-
 3-networks-hub-and-spoke/modules/restricted_shared_vpc/dns.tf        |  2 +-
 Makefile                                                             |  2 +-
 README.md                                                            | 16 ++++++++--------
 build/int.cloudbuild.yaml                                            |  2 +-
 build/lint.cloudbuild.yaml                                           |  2 +-
 docs/upgrading_to_v4.0.md                                            |  8 ++++----
 helpers/foundation-deployer/README.md                                |  2 +-
 helpers/foundation-deployer/go.mod                                   | 27 ++++++++++++++-------------
 helpers/foundation-deployer/go.sum                                   | 40 ++++++++++++++++++++++++++++++----------
 test/integration/go.mod                                              | 34 ++++++++++++++++------------------
 test/integration/go.sum                                              | 70 ++++++++++++++++++++++++++++++++++++----------------------------------
 test/integration/shared/shared_test.go                               | 18 +++++++++---------
 26 files changed, 208 insertions(+), 137 deletions(-)
 create mode 100644 .github/workflows/go-test.yaml

fmichaelobrien · 2024-05-24T03:42:04Z

Merge Marian's single yaml config with the removal of BU2 - (reduces project count from 51)
terraform-google-modules/terraform-example-foundation#1241
as part of terraform-google-modules/terraform-example-foundation#1238

github-actions · 2024-08-15T23:21:44Z

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 7 days

fmichaelobrien added the tef label Apr 16, 2024

fmichaelobrien self-assigned this Apr 16, 2024

fmichaelobrien mentioned this issue Apr 16, 2024

TEF issues to migrate over to this staging copy of the TEF #376

Closed

obriensystems changed the title ~~TEF resync and terraform provider updates cadence~~ TEF upstream git resync and terraform provider updates cadence Apr 24, 2024

obriensystems added the upstream label May 3, 2024

This was referenced May 3, 2024

Replace/revert-back CB/CSR options as we add local terraform and ado (399) options - to be able to use CB/CSR as a current option while we retrofit #421

Closed

#421 - prepare main for dual CB/CSR and terraform local deployment modes #423

Merged

obriensystems pushed a commit that referenced this issue May 11, 2024

#387 - sync 0-b tef 20240511

92ad88d

obriensystems mentioned this issue May 11, 2024

#387 - upstream TEF sync 20240511 - to PR 1199 #435

Closed

obriensystems added a commit that referenced this issue May 11, 2024

#387 - sync 1-org tef 20240511

105ae82

obriensystems added a commit that referenced this issue May 11, 2024

#387 - sync - rename non-p to nonp - tef 20240511

04c3e14

obriensystems added a commit that referenced this issue May 11, 2024

#387 - sync - 2-env - tef 20240511

ae1a9fe

obriensystems added a commit that referenced this issue May 11, 2024

#387 - sync - 3-n-d-s - tef 20240511

d1ec446

obriensystems added a commit that referenced this issue May 11, 2024

#387 - 3-nhas symlink - tef 20240511

c74d6b0

obriensystems added a commit that referenced this issue May 11, 2024

#387 - 3-nhas finish - tef 20240511

e5cf808

obriensystems mentioned this issue May 11, 2024

Recurring error "400 route operation in progress" when applying 3-network-hub-and-spoke terraform-google-modules/terraform-example-foundation#1228

Closed

obriensystems added a commit that referenced this issue May 12, 2024

#387 - 4-p nonp rename fist - tef 20240511

9e2270e

obriensystems added a commit that referenced this issue May 12, 2024

#387 - 4-p finished - tef 20240511

38bc820

obriensystems added a commit that referenced this issue May 12, 2024

#387 - 5-af - tef 20240511

6fbaf12

obriensystems added a commit that referenced this issue May 12, 2024

#387 - build - tef 20240511

3f3b646

obriensystems added a commit that referenced this issue May 12, 2024

#387 - docs - tef 20240511

be6c121

obriensystems added a commit that referenced this issue May 12, 2024

#387 - helpers - tef 20240511

64c0b18

obriensystems added a commit that referenced this issue May 12, 2024

#387 - root - tef 20240511

ed1e43d

fmichaelobrien added a commit that referenced this issue May 13, 2024

#387 - fix symlink - tef 1231

166e887

fmichaelobrien mentioned this issue May 28, 2024

Merge Marian's latest dev branch - incorporating 20240520 TEF upstream sync #448

Closed

github-actions bot added the Stale label Aug 15, 2024

github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Aug 25, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

TEF upstream git resync and terraform provider updates cadence #387

TEF upstream git resync and terraform provider updates cadence #387

fmichaelobrien commented Apr 16, 2024 •

edited

Loading

fmichaelobrien commented Apr 24, 2024 •

edited

Loading

obriensystems commented Apr 24, 2024

obriensystems commented Apr 25, 2024

obriensystems commented May 3, 2024 •

edited

Loading

fmichaelobrien commented May 4, 2024

obriensystems commented May 11, 2024 •

edited

Loading

obriensystems commented May 12, 2024

fmichaelobrien commented May 12, 2024

fmichaelobrien commented May 13, 2024

fmichaelobrien commented May 14, 2024

fmichaelobrien commented May 17, 2024

fmichaelobrien commented May 17, 2024 •

edited

Loading

obriensystems commented May 17, 2024

fmichaelobrien commented May 24, 2024 •

edited

Loading

github-actions bot commented Aug 15, 2024

TEF upstream git resync and terraform provider updates cadence #387

TEF upstream git resync and terraform provider updates cadence #387

Comments

fmichaelobrien commented Apr 16, 2024 • edited Loading

PRs merged to the terraform-example-foundation upstream repo

terraform-example-foundation upstream resync procedure - copy not fork

Indirect terraform provider updates

fmichaelobrien commented Apr 24, 2024 • edited Loading

obriensystems commented Apr 24, 2024

obriensystems commented Apr 25, 2024

obriensystems commented May 3, 2024 • edited Loading

keep

Changes

Adjust

Help with upstream sync

Sync

fmichaelobrien commented May 4, 2024

20240504: Repo state: CB/CSR are the default for the main branch

obriensystems commented May 11, 2024 • edited Loading

0-bootstrap, 1-org done

3-networks-hub-and-spoke

obriensystems commented May 12, 2024

fmichaelobrien commented May 12, 2024

fmichaelobrien commented May 13, 2024

fmichaelobrien commented May 14, 2024

pending June upstream sync

Direct

Indirect

fmichaelobrien commented May 17, 2024

fmichaelobrien commented May 17, 2024 • edited Loading

obriensystems commented May 17, 2024

fmichaelobrien commented May 24, 2024 • edited Loading

github-actions bot commented Aug 15, 2024

fmichaelobrien commented Apr 16, 2024 •

edited

Loading

fmichaelobrien commented Apr 24, 2024 •

edited

Loading

obriensystems commented May 3, 2024 •

edited

Loading

obriensystems commented May 11, 2024 •

edited

Loading

fmichaelobrien commented May 17, 2024 •

edited

Loading

fmichaelobrien commented May 24, 2024 •

edited

Loading