fix: Phasing out CSR dependency #1315

lpezet · 2024-08-02T04:41:13Z

PR for issue #1309

lpezet · 2024-08-02T14:08:44Z

Shoot...exceeded project quota limit (even after the increase to 50 as suggested in documentation). No local testing for me for a while then lol.

eeaton · 2024-08-02T14:41:46Z

You might be able to make another incremental quota increase request... there is some secret sauce I don't understand for how it works, but give it a try. Be aware however that Projects in the state where they're deleted but recoverable also count against your project quota, so it might take some time to reset the quota.

I'll chip away at the review of the other PR you've raised recently, should be able to make some progress with the automatic CI we have on our end.

eeaton · 2024-08-02T14:42:39Z

/gcbrun

daniel-cit · 2024-08-02T15:28:26Z

build error

Step #3 - "converge-bootstrap": TestBootstrap 2024-08-02T15:15:18Z command.go:185: 
Step #3 - "converge-bootstrap": TestBootstrap 2024-08-02T15:15:18Z command.go:185: Error: Error creating Workflow: googleapi: Error 400: Workflows service agent does not exist
Step #3 - "converge-bootstrap": TestBootstrap 2024-08-02T15:15:18Z command.go:185: 
Step #3 - "converge-bootstrap": TestBootstrap 2024-08-02T15:15:18Z command.go:185:   with module.tf_cloud_builder.google_workflows_workflow.builder,
Step #3 - "converge-bootstrap": TestBootstrap 2024-08-02T15:15:18Z command.go:185:   on .terraform/modules/tf_cloud_builder/modules/tf_cloudbuild_builder/workflow.tf line 36, in resource "google_workflows_workflow" "builder":
Step #3 - "converge-bootstrap": TestBootstrap 2024-08-02T15:15:18Z command.go:185:   36: resource "google_workflows_workflow" "builder" {
Step #3 - "converge-bootstrap": TestBootstrap 2024-08-02T15:15:18Z command.go:185: 
Step #3 - "converge-bootstrap": TestBootstrap 2024-08-02T15:15:18Z retry.go:99: Returning due to fatal error: FatalError{Underlying: error while running command: exit status 1; 
Step #3 - "converge-bootstrap": Error: Error creating Workflow: googleapi: Error 400: Workflows service agent does not exist
Step #3 - "converge-bootstrap": 
Step #3 - "converge-bootstrap":   with module.tf_cloud_builder.google_workflows_workflow.builder,
Step #3 - "converge-bootstrap":   on .terraform/modules/tf_cloud_builder/modules/tf_cloudbuild_builder/workflow.tf line 36, in resource "google_workflows_workflow" "builder":
Step #3 - "converge-bootstrap":   36: resource "google_workflows_workflow" "builder" {
Step #3 - "converge-bootstrap": }
Step #3 - "converge-bootstrap":     apply.go:34: 
Step #3 - "converge-bootstrap":         	Error Trace:	/builder/home/go/pkg/mod/github.com/gruntwork-io/[email protected]/modules/terraform/apply.go:34
Step #3 - "converge-bootstrap":         	            				/builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/[email protected]/pkg/tft/terraform.go:571
Step #3 - "converge-bootstrap":         	            				/workspace/test/integration/bootstrap/bootstrap_test.go:126
Step #3 - "converge-bootstrap":         	            				/builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/[email protected]/pkg/tft/terraform.go:630
Step #3 - "converge-bootstrap":         	            				/builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/[email protected]/pkg/tft/terraform.go:669
Step #3 - "converge-bootstrap":         	            				/builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/[email protected]/pkg/utils/stages.go:31
Step #3 - "converge-bootstrap":         	            				/builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/[email protected]/pkg/tft/terraform.go:669
Step #3 - "converge-bootstrap":         	Error:      	Received unexpected error:
Step #3 - "converge-bootstrap":         	            	FatalError{Underlying: error while running command: exit status 1; 
Step #3 - "converge-bootstrap":         	            	Error: Error creating Workflow: googleapi: Error 400: Workflows service agent does not exist
Step #3 - "converge-bootstrap":         	            	
Step #3 - "converge-bootstrap":         	            	  with module.tf_cloud_builder.google_workflows_workflow.builder,
Step #3 - "converge-bootstrap":         	            	  on .terraform/modules/tf_cloud_builder/modules/tf_cloudbuild_builder/workflow.tf line 36, in resource "google_workflows_workflow" "builder":
Step #3 - "converge-bootstrap":         	            	  36: resource "google_workflows_workflow" "builder" {
Step #3 - "converge-bootstrap":         	            	}
Step #3 - "converge-bootstrap":         	Test:       	TestBootstrap
Step #3 - "converge-bootstrap": 2024/08/02 15:15:18 RUN_STAGE env var set to apply
Step #3 - "converge-bootstrap": 2024/08/02 15:15:18 Skipping stage teardown
Step #3 - "converge-bootstrap": --- FAIL: TestBootstrap (915.97s)

I think we may need to use a flag to toggle usage of CSR for now, both in the code and in the tests

lpezet · 2024-08-03T01:38:29Z

@daniel-cit What's the relationship between CSR and Workflows (genuinely curious)? Do you mean CSR would be enabled by default so CI/CD would pass here (otherwise they'd fail like here)?

eeaton · 2024-08-05T15:42:08Z

@daniel-cit I think that might have been a flaky error, I recently saw the same error in branch renovate/go-modules-andor-dev-tools for commit 0362342. It worked when I ran the tests a second time.

I suspect that you might be right we need a flag, because I don't understand the chain of dependencies in the CFT ecosystem. But for now I'll rerun the tests and see if it pinpoints anything more specific.

@lpezet in general CSR and Workflows API don't depend on each other, but a remote module referenced by this repo uses Workflows as part of building the container image for Cloud Build.
https://github.com/terraform-google-modules/terraform-google-bootstrap/tree/v8.0.0/modules/tf_cloudbuild_builder#resources-created

eeaton · 2024-08-06T06:21:50Z

I reran the tests and it does indeed look the error related to workflow service accounts was flaky, no need to investigate that further.

Both versions of the CI now fail with the same error from TestBootstrap.

Step #3 - "converge-bootstrap": TestBootstrap 2024-08-05T16:00:12Z command.go:185:     "message": "//serviceusage.googleapis.com/projects/ci-foundation-2e6op7-vmxm/services/sourcerepo.googleapis.com violates a service constraint",

So it looks like the CI project created as part of running docker tests still uses sourcerepo.googleapis.com. I suspect we'll need to unpick that dependency from the upstream CFT framework (not part of this repo).
As a quick fix to this error, I recommend reverting the changes to policy-library/policies/constraints/serviceusage_allow_basic_apis.yaml. Removing CSR from this allowlist will throw errors for the customers who continue to use CSR, but leaving CSR on the allowlist should not impact customers who are unable to enable CSR.

…allow_basic_apis.yaml.

daniel-cit · 2024-08-07T02:58:58Z

/gcbrun

daniel-cit · 2024-08-07T04:33:33Z

4-projects/shared creates a set of pipelines, each one with its own source repository

Step #19 - "converge-projects-shared": TestProjectsShared/bu1 2024-08-07T04:19:14Z retry.go:99: Returning due to fatal error: FatalError{Underlying: error while running command: exit status 1; 
Step #19 - "converge-projects-shared": Error: Error creating Repository: googleapi: Error 403: Cloud Source Repositories API has not been used in project `s7j-c-bu1-infra-pipeline-REDACTED` before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/sourcerepo.googleapis.com/overview?project=s7j-c-bu1-infra-pipeline-REDACTED then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.
Step #19 - "converge-projects-shared": Details:
Step #19 - "converge-projects-shared": [
Step #19 - "converge-projects-shared":   {
Step #19 - "converge-projects-shared":     "@type": "type.googleapis.com/google.rpc.Help",
Step #19 - "converge-projects-shared":     "links": [
Step #19 - "converge-projects-shared":       {
Step #19 - "converge-projects-shared":         "description": "Google developers console API activation",
Step #19 - "converge-projects-shared":         "url": "https://console.developers.google.com/apis/api/sourcerepo.googleapis.com/overview?project=s7j-c-bu1-infra-pipeline-REDACTED"
Step #19 - "converge-projects-shared":       }
Step #19 - "converge-projects-shared":     ]
Step #19 - "converge-projects-shared":   },
Step #19 - "converge-projects-shared":   {
Step #19 - "converge-projects-shared":     "@type": "type.googleapis.com/google.rpc.ErrorInfo",
Step #19 - "converge-projects-shared":     "domain": "googleapis.com",
Step #19 - "converge-projects-shared":     "metadata": {
Step #19 - "converge-projects-shared":       "consumer": "projects/s7j-c-bu1-infra-pipeline-7ego",
Step #19 - "converge-projects-shared":       "service": "sourcerepo.googleapis.com"
Step #19 - "converge-projects-shared":     },
Step #19 - "converge-projects-shared":     "reason": "SERVICE_DISABLED"
Step #19 - "converge-projects-shared":   }
Step #19 - "converge-projects-shared": ]

the step has a check (flag) to create or not the build infrastructure that is based in creation or not of the artifact registry in 0-bootstrap.

We may need to enhance this check to consider a new flag like source repositories ON/OFF

lpezet · 2024-08-07T23:51:55Z

@daniel-cit So things related to like:

terraform-example-foundation/4-projects/business_unit_1/shared/remote.tf

Line 29 in 028f85f

enable_cloudbuild_deploy = local.cloud_builder_artifact_repo != ""

. Is that right?
Does it mean then that, if we set it to false when CSR disabled, the infra-pipelines module won't be applied? and therefore not tested? (I don't fully understand whether the lack of this module will affect tests in projects-shared/projects_shared_test.go).

lpezet · 2024-08-09T00:45:07Z

Just sharing some thoughts here...
Since CSR is being phased out, shouldn't a small rewrite be considered instead?
Phase1: To phase it out smoothly (hopefully), CSR could be a choice, like Github and Gitlab are right now, and CI tests could setup for that option (small change, hopefully), and actual tests would stay the same as today. No defaults, must pick a repo provider.
Phase 2: Update tests setup+actual tests (?) to use another repo (Github/Gitlab). So (new) contributors could run tests using their repo of choice.

I don't want to butt in, just trying to help move the needle...

daniel-cit · 2024-08-09T13:37:07Z

Just sharing some thoughts here... Since CSR is being phased out, shouldn't a small rewrite be considered instead? Phase1: To phase it out smoothly (hopefully), CSR could be a choice, like Github and Gitlab are right now, and CI tests could setup for that option (small change, hopefully), and actual tests would stay the same as today. No defaults, must pick a repo provider. Phase 2: Update tests setup+actual tests (?) to use another repo (Github/Gitlab). So (new) contributors could run tests using their repo of choice.

I don't want to butt in, just trying to help move the needle...

Hi @lpezet.
There is an open PR in the bootstrap module to add support for Cloud Build Repositories (2nd Gen) in the modules used in the foundation. This will help to execute your proposal of allowing the deploy and the tests to switch between CSR (integration build) and GItHub or Gitlab (local testing)

github-actions · 2024-10-09T23:20:44Z

This PR is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 7 days

lpezet added 2 commits August 1, 2024 22:36

Commented out/removed sourcerepo.googleapis.com API.

890022e

Comments added.

bb63855

lpezet requested review from rjerrems, gtsorbo, eeaton, sleighton2022 and a team as code owners August 2, 2024 04:41

lpezet added 2 commits August 6, 2024 20:53

Reverted changes to policy-library/policies/constraints/serviceusage_…

7f68fcc

…allow_basic_apis.yaml.

Synced with master.

903633d

Merge branch 'terraform-google-modules:master' into issue/1309

242e40b

github-actions bot added the Stale label Oct 9, 2024

github-actions bot closed this Oct 17, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: Phasing out CSR dependency #1315

fix: Phasing out CSR dependency #1315

lpezet commented Aug 2, 2024

lpezet commented Aug 2, 2024

eeaton commented Aug 2, 2024 •

edited

Loading

eeaton commented Aug 2, 2024

daniel-cit commented Aug 2, 2024

lpezet commented Aug 3, 2024

eeaton commented Aug 5, 2024

eeaton commented Aug 6, 2024 •

edited

Loading

daniel-cit commented Aug 7, 2024

daniel-cit commented Aug 7, 2024 •

edited

Loading

lpezet commented Aug 7, 2024 •

edited

Loading

lpezet commented Aug 9, 2024

daniel-cit commented Aug 9, 2024

github-actions bot commented Oct 9, 2024

fix: Phasing out CSR dependency #1315

fix: Phasing out CSR dependency #1315

Conversation

lpezet commented Aug 2, 2024

lpezet commented Aug 2, 2024

eeaton commented Aug 2, 2024 • edited Loading

eeaton commented Aug 2, 2024

daniel-cit commented Aug 2, 2024

lpezet commented Aug 3, 2024

eeaton commented Aug 5, 2024

eeaton commented Aug 6, 2024 • edited Loading

daniel-cit commented Aug 7, 2024

daniel-cit commented Aug 7, 2024 • edited Loading

lpezet commented Aug 7, 2024 • edited Loading

lpezet commented Aug 9, 2024

daniel-cit commented Aug 9, 2024

github-actions bot commented Oct 9, 2024

eeaton commented Aug 2, 2024 •

edited

Loading

eeaton commented Aug 6, 2024 •

edited

Loading

daniel-cit commented Aug 7, 2024 •

edited

Loading

lpezet commented Aug 7, 2024 •

edited

Loading