Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Unable to create cloud_run_v2_job resource anymore when using run_execution_token #20602

Closed
emilecaron opened this issue Dec 5, 2024 · 2 comments
Closed

Unable to create cloud_run_v2_job resource anymore when using run_execution_token #20602

emilecaron opened this issue Dec 5, 2024 · 2 comments
Assignees
@NickElliot
Labels
bug forward/review In review; remove label to forward service/run

Comments

@emilecaron
Copy link

emilecaron commented Dec 5, 2024 •
edited
Loading

Community Note

  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
  • Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment.
  • If an issue is assigned to a user, that user is claiming responsibility for the issue.
  • Customers working with a Google Technical Account Manager or Customer Engineer can ask them to reach out internally to expedite investigation and resolution of this issue.

Terraform Version & Provider Version(s)

Terraform v1.5.7
on

  • provider registry.terraform.io/hashicorp/google v6.12.0
  • provider registry.terraform.io/hashicorp/google-beta v6.12.0

Affected Resource(s)

google_cloud_run_v2_job resources

Terraform Configuration

resource "google_cloud_run_v2_job" "migrate" {
  provider     = google-beta
  name         = "migrate-${var.name}-${var.short_sha}"
  location     = var.google_region
  launch_stage = "BETA"
  deletion_protection = false

  
  // trigger and wait for successful execution on deployment
  run_execution_token = "autoexec"

  depends_on = [
    google_secret_manager_secret_version.db_url,
    google_project_service.cloud_run_api,
  ]

  template {
    parallelism = 1
    task_count = 1

    template {
      service_account                  = local.service_account_email
      timeout                          = "21600s"
      max_retries = 0

      volumes {
        name = "cloudsql"
        cloud_sql_instance {
          instances = [google_sql_database_instance.postgres.connection_name]
        }
      }

      containers {
        image   = "<REDACTED>"
        command = ["./database_migrate"]
      }
    }
  }
}

Debug Output

Actual error message:

Error: Error waiting to create Job: `resource` not set in operation response

With log level debug we can get some debug on the last request:

INFO 2024-12-05T14:04:27.799259937Z 2024-12-05T14:04:27.256Z [DEBUG] provider.terraform-provider-google-beta_v6.12.0_x5: 2024/12/05 14:04:27 [DEBUG] Google API Request Details:
INFO 2024-12-05T14:04:27.799260596Z 2024-12-05T14:04:27.256Z [DEBUG] provider.terraform-provider-google-beta_v6.12.0_x5: ---[ REQUEST ]---------------------------------------
INFO 2024-12-05T14:04:27.799261466Z 2024-12-05T14:04:27.256Z [DEBUG] provider.terraform-provider-google-beta_v6.12.0_x5: GET /v2/projects/lpo-dev-363712/locations/europe-west1/operations/729ad1bc-b5ff-428c-99bf-bd4dafdc6b76?alt=json HTTP/1.1
INFO 2024-12-05T14:04:27.799262140Z 2024-12-05T14:04:27.256Z [DEBUG] provider.terraform-provider-google-beta_v6.12.0_x5: Host: run.googleapis.com
INFO 2024-12-05T14:04:27.799262975Z 2024-12-05T14:04:27.256Z [DEBUG] provider.terraform-provider-google-beta_v6.12.0_x5: User-Agent: Terraform/1.5.7 (+https://www.terraform.io) Terraform-Plugin-SDK/2.33.0 terraform-provider-google-beta/6.12.0
INFO 2024-12-05T14:04:27.799263565Z 2024-12-05T14:04:27.256Z [DEBUG] provider.terraform-provider-google-beta_v6.12.0_x5: Content-Type: application/json
INFO 2024-12-05T14:04:27.799265148Z 2024-12-05T14:04:27.256Z [DEBUG] provider.terraform-provider-google-beta_v6.12.0_x5: Accept-Encoding: gzip
INFO 2024-12-05T14:04:27.799265709Z 2024-12-05T14:04:27.256Z [DEBUG] provider.terraform-provider-google-beta_v6.12.0_x5:
INFO 2024-12-05T14:04:27.799266281Z 2024-12-05T14:04:27.256Z [DEBUG] provider.terraform-provider-google-beta_v6.12.0_x5:
INFO 2024-12-05T14:04:27.799266907Z 2024-12-05T14:04:27.256Z [DEBUG] provider.terraform-provider-google-beta_v6.12.0_x5: -----------------------------------------------------
INFO 2024-12-05T14:04:27.799267525Z 2024-12-05T14:04:27.311Z [DEBUG] provider.terraform-provider-google-beta_v6.12.0_x5: 2024/12/05 14:04:27 [DEBUG] Google API Response Details:
INFO 2024-12-05T14:04:27.799268152Z 2024-12-05T14:04:27.311Z [DEBUG] provider.terraform-provider-google-beta_v6.12.0_x5: ---[ RESPONSE ]--------------------------------------
INFO 2024-12-05T14:04:27.799268717Z 2024-12-05T14:04:27.311Z [DEBUG] provider.terraform-provider-google-beta_v6.12.0_x5: HTTP/2.0 200 OK
INFO 2024-12-05T14:04:27.799269283Z 2024-12-05T14:04:27.311Z [DEBUG] provider.terraform-provider-google-beta_v6.12.0_x5: Cache-Control: private
INFO 2024-12-05T14:04:27.799269914Z 2024-12-05T14:04:27.311Z [DEBUG] provider.terraform-provider-google-beta_v6.12.0_x5: Content-Type: application/json; charset=UTF-8
INFO 2024-12-05T14:04:27.799270535Z 2024-12-05T14:04:27.311Z [DEBUG] provider.terraform-provider-google-beta_v6.12.0_x5: Date: Thu, 05 Dec 2024 14:04:27 GMT
INFO 2024-12-05T14:04:27.799271104Z 2024-12-05T14:04:27.311Z [DEBUG] provider.terraform-provider-google-beta_v6.12.0_x5: Server: ESF
INFO 2024-12-05T14:04:27.799271667Z 2024-12-05T14:04:27.311Z [DEBUG] provider.terraform-provider-google-beta_v6.12.0_x5: Vary: Origin
INFO 2024-12-05T14:04:27.799272240Z 2024-12-05T14:04:27.311Z [DEBUG] provider.terraform-provider-google-beta_v6.12.0_x5: Vary: X-Origin
INFO 2024-12-05T14:04:27.799272788Z 2024-12-05T14:04:27.311Z [DEBUG] provider.terraform-provider-google-beta_v6.12.0_x5: Vary: Referer
INFO 2024-12-05T14:04:27.799273419Z 2024-12-05T14:04:27.311Z [DEBUG] provider.terraform-provider-google-beta_v6.12.0_x5: X-Content-Type-Options: nosniff
INFO 2024-12-05T14:04:27.799274003Z 2024-12-05T14:04:27.311Z [DEBUG] provider.terraform-provider-google-beta_v6.12.0_x5: X-Frame-Options: SAMEORIGIN
INFO 2024-12-05T14:04:27.799274563Z 2024-12-05T14:04:27.311Z [DEBUG] provider.terraform-provider-google-beta_v6.12.0_x5: X-Xss-Protection: 0
INFO 2024-12-05T14:04:27.799275085Z 2024-12-05T14:04:27.311Z [DEBUG] provider.terraform-provider-google-beta_v6.12.0_x5:
INFO 2024-12-05T14:04:27.799275607Z 2024-12-05T14:04:27.311Z [DEBUG] provider.terraform-provider-google-beta_v6.12.0_x5: {
INFO 2024-12-05T14:04:27.799276266Z 2024-12-05T14:04:27.311Z [DEBUG] provider.terraform-provider-google-beta_v6.12.0_x5: "name": "projects/lpo-dev-363712/locations/europe-west1/operations/729ad1bc-b5ff-428c-99bf-bd4dafdc6b76",
INFO 2024-12-05T14:04:27.799281627Z 2024-12-05T14:04:27.311Z [DEBUG] provider.terraform-provider-google-beta_v6.12.0_x5: "done": true
INFO 2024-12-05T14:04:27.799282235Z 2024-12-05T14:04:27.311Z [DEBUG] provider.terraform-provider-google-beta_v6.12.0_x5: }
INFO 2024-12-05T14:04:27.799282764Z 2024-12-05T14:04:27.311Z [DEBUG] provider.terraform-provider-google-beta_v6.12.0_x5:
INFO 2024-12-05T14:04:27.799283409Z 2024-12-05T14:04:27.311Z [DEBUG] provider.terraform-provider-google-beta_v6.12.0_x5: -----------------------------------------------------
INFO 2024-12-05T14:04:27.799284080Z 2024-12-05T14:04:27.311Z [DEBUG] provider.terraform-provider-google-beta_v6.12.0_x5: 2024/12/05 14:04:27 [DEBUG] Retry Transport: Stopping retries, last request was successful
INFO 2024-12-05T14:04:27.799284718Z 2024-12-05T14:04:27.311Z [DEBUG] provider.terraform-provider-google-beta_v6.12.0_x5: 2024/12/05 14:04:27 [DEBUG] Retry Transport: Returning after 1 attempts
INFO 2024-12-05T14:04:27.799288909Z 2024-12-05T14:04:27.311Z [DEBUG] provider.terraform-provider-google-beta_v6.12.0_x5: 2024/12/05 14:04:27 [DEBUG] Got done: true while polling for operation projects/lpo-dev-363712/locations/europe-west1/operations/729ad1bc-b5ff-428c-99bf-bd4dafdc6b76's status
INFO 2024-12-05T14:04:27.799291234Z 2024-12-05T14:04:27.323Z [ERROR] provider.terraform-provider-google-beta_v6.12.0_x5: Response contains error diagnostic: tf_rpc=ApplyResourceChange @caller=github.com/hashicorp/[email protected]/tfprotov5/internal/diag/diagnostics.go:58 diagnostic_severity=ERROR diagnostic_summary="Error waiting to create Job: `resource` not set in operation response" tf_proto_version=5.6 tf_provider_addr=registry.terraform.io/hashicorp/google-beta tf_req_id=f2188043-0e2a-3f9e-6aa1-54424b259d9f tf_resource_type=google_cloud_run_v2_job @module=sdk.proto diagnostic_detail= timestamp=2024-12-05T14:04:27.323Z
INFO 2024-12-05T14:04:27.799291635Z 2024-12-05T14:04:27.371Z [DEBUG] states/remote: state read serial is: 1292; serial is: 1292
INFO 2024-12-05T14:04:27.799292329Z 2024-12-05T14:04:27.371Z [DEBUG] states/remote: state read lineage is: 07788846-aceb-ac93-713c-b861e32e5a9d; lineage is: 07788846-aceb-ac93-713c-b861e32e5a9d
INFO 2024-12-05T14:04:27.799292949Z 2024-12-05T14:04:27.545Z [ERROR] vertex "google_cloud_run_v2_job.migrate" error: Error waiting to create Job: `resource` not set in operation response
INFO 2024-12-05T14:04:27.799293501Z 2024-12-05T14:04:27.546Z [DEBUG] states/remote: state read serial is: 1293; serial is: 1293
INFO 2024-12-05T14:04:27.799294118Z 2024-12-05T14:04:27.546Z [DEBUG] states/remote: state read lineage is: 07788846-aceb-ac93-713c-b861e32e5a9d; lineage is: 07788846-aceb-ac93-713c-b861e32e5a9d
INFO 2024-12-05T14:04:27.799294515Z
INFO 2024-12-05T14:04:27.799295326Z Error: Error waiting to create Job: `resource` not set in operation response
INFO 2024-12-05T14:04:27.799295696Z
INFO 2024-12-05T14:04:27.799298245Z with google_cloud_run_v2_job.migrate,
INFO 2024-12-05T14:04:27.799298983Z on migrate.tf line 1, in resource "google_cloud_run_v2_job" "migrate":
INFO 2024-12-05T14:04:27.799299957Z 1: resource "google_cloud_run_v2_job" "migrate" {
INFO 2024-12-05T14:04:27.799300317Z
INFO 2024-12-05T14:04:27.799301010Z 2024-12-05T14:04:27.705Z [DEBUG] provider.terraform-provider-google-beta_v6.12.0_x5: 2024/12/05 14:04:27 [DEBUG] [transport] [server-transport 0xc00117a1a0] Closing: Server.Stop called
INFO 2024-12-05T14:04:27.799301702Z 2024-12-05T14:04:27.706Z [DEBUG] provider.terraform-provider-google_v6.12.0_x5: 2024/12/05 14:04:27 [DEBUG] [transport] [server-transport 0xc00112a000] Closing: Server.Stop called
INFO 2024-12-05T14:04:27.799302521Z 2024-12-05T14:04:27.706Z [DEBUG] provider: plugin process exited: path=.terraform/providers/registry.terraform.io/hashicorp/google/6.12.0/linux_amd64/terraform-provider-google_v6.12.0_x5 pid=128
INFO 2024-12-05T14:04:27.799303246Z 2024-12-05T14:04:27.706Z [DEBUG] provider.stdio: received EOF, stopping recv loop: err="rpc error: code = Canceled desc = context canceled"
INFO 2024-12-05T14:04:27.799304135Z 2024-12-05T14:04:27.706Z [DEBUG] provider.stdio: received EOF, stopping recv loop: err="rpc error: code = Unavailable desc = error reading from server: EOF"
INFO 2024-12-05T14:04:27.799304916Z 2024-12-05T14:04:27.706Z [DEBUG] provider: plugin process exited: path=.terraform/providers/registry.terraform.io/devops-rob/terracurl/1.2.1/linux_amd64/terraform-provider-terracurl_v1.2.1 pid=122
INFO 2024-12-05T14:04:27.799305682Z 2024-12-05T14:04:27.706Z [DEBUG] provider.stdio: received EOF, stopping recv loop: err="rpc error: code = Unavailable desc = error reading from server: EOF"
INFO 2024-12-05T14:04:27.799306433Z 2024-12-05T14:04:27.706Z [DEBUG] provider: plugin exited
INFO 2024-12-05T14:04:27.799307535Z 2024-12-05T14:04:27.706Z [DEBUG] provider: plugin exited
INFO 2024-12-05T14:04:27.799308367Z 2024-12-05T14:04:27.707Z [DEBUG] provider: plugin process exited: path=.terraform/providers/registry.terraform.io/hashicorp/google-beta/6.12.0/linux_amd64/terraform-provider-google-beta_v6.12.0_x5 pid=143
INFO 2024-12-05T14:04:27.799308850Z 2024-12-05T14:04:27.707Z [DEBUG] provider: plugin exited

Expected Behavior

Terraform should not return an error when creating a cloud run job resource

Actual Behavior

The job is actually created but terraform returns the error above.

Steps to reproduce

creating a cloud run job resource with run_execution_token set to a value

Important Factoids

traced the error back to https://github.com/hashicorp/terraform-provider-google/blob/v6.12.0/google/services/cloudrunv2/cloud_run_v2_operation.go#L76
it seems that the api reponse no longer includes the "response" key, thus breaking the checks in terraform and resulting in this error.

References

No response
@emilecaron emilecaron added the bug label Dec 5, 2024
@github-actions github-actions bot added forward/review In review; remove label to forward service/run labels Dec 5, 2024
@emilecaron emilecaron changed the title Unable to create cloud_run_v2_job resource anymore Unable to create cloud_run_v2_job resource anymore when using run_execution_token Dec 5, 2024
@NickElliot NickElliot self-assigned this Dec 5, 2024
@emilecaron
Copy link
Author

i think the issue was resolved !

@MaStFAU
Copy link

MaStFAU commented Dec 11, 2024

May I ask how it was resolved? We are still getting the error.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@NickElliot NickElliot

Labels
bug forward/review In review; remove label to forward service/run
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@emilecaron @MaStFAU @NickElliot