Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

run_cmd in locals block executed twice #1427

Closed
bendavies opened this issue Nov 13, 2020 · 9 comments · May be fixed by #1554
Closed

run_cmd in locals block executed twice #1427

bendavies opened this issue Nov 13, 2020 · 9 comments · May be fixed by #1554
Labels
bug Something isn't working

Comments

@bendavies
Copy link

bendavies commented Nov 13, 2020

Hi there,

It seems a run_cmd inside a locals block will be run twice.

Reproducer:

terragrunt.hcl

locals {
  foo = run_cmd("echo", uuid())
}
terragrunt -v
terragrunt version v0.26.2

terragrunt init
[terragrunt] 2020/11/13 15:33:15 Running command: echo d3d3c70d-4165-6842-f27a-1d846be06553
d3d3c70d-4165-6842-f27a-1d846be06553
[terragrunt] 2020/11/13 15:33:15 run_cmd output: [d3d3c70d-4165-6842-f27a-1d846be06553
]
[terragrunt] [/path] 2020/11/13 15:33:15 Running command: terraform --version
[terragrunt] 2020/11/13 15:33:16 Terraform version: 0.12.29
[terragrunt] 2020/11/13 15:33:16 Reading Terragrunt config file at /path/terragrunt.hcl
[terragrunt] 2020/11/13 15:33:16 Running command: echo e654adfb-84bc-3326-9b89-71198959deee
e654adfb-84bc-3326-9b89-71198959deee
[terragrunt] 2020/11/13 15:33:16 run_cmd output: [e654adfb-84bc-3326-9b89-71198959deee
]
[terragrunt] 2020/11/13 15:33:16 Running command: terraform init
Terraform initialized in an empty directory!

The directory has no Terraform configuration files. You may begin working
with Terraform immediately by creating Terraform configuration files.

notice two different UUIDs in the log, which I believe shows this is not just a double logging issue.

Thanks!

@bendavies bendavies changed the title run_cmd in locals exectuted twice run_cmd in locals block exectuted twice Nov 13, 2020
@ksvladimir
Copy link
Contributor

If definitely happens to me too, I was about to report the same issue.

@brikis98 brikis98 added bug Something isn't working help wanted labels Nov 16, 2020
@brikis98
Copy link
Member

Hm, must be an issue with the parsing logic, as it takes several passes to parse everything. If anyone has time to dig into this, help would be greatly appreciated, as we're quite booked right now.

@amnk
Copy link
Contributor

amnk commented Jan 27, 2021

This happens because of checkVersionConstraints(), which leads to cli_app.go#L405.

Since config.PartialParseConfigFile() always parses locals block, we get two executions.

amnk added a commit to amnk/terragrunt that referenced this issue Jan 27, 2021
@brikis98
Copy link
Member

This happens because of checkVersionConstraints(), which leads to cli_app.go#L405.

Since config.PartialParseConfigFile() always parses locals block, we get two executions.

Oh, bummer. Any thoughts on how to fix this?

amnk added a commit to amnk/terragrunt that referenced this issue Feb 4, 2021
amnk added a commit to amnk/terragrunt that referenced this issue Feb 18, 2021
amnk added a commit to amnk/terragrunt that referenced this issue Feb 18, 2021
Since `locals` can have side effects, we might run into cases when
we don't need to pasre them. One example is `checkVersionConstraints()`,
which is executed early on and relies on parsing Terraform.

Closes: gruntwork-io#1427
@tiwood
Copy link
Contributor

tiwood commented Jun 17, 2021

@yorinasub17, in #1554 you had some great suggestions, do you already have a timeframe in which you plan to implement those? Is there any way one could help?

We're currently discovering a lot of dynamic information with a some run_cmd commands, which are quite expensive time-wise and the duplicate execution is not making it any better :-(

@ChristophShyper
Copy link

Should it be executed even when it's in negative if condition?
For config below it's executed always, and twice.

terraform {
  source = "/my_module"
}

inputs = {
  my_cmd = local.my_cmd
  my_value = read_terragrunt_config(find_in_parent_folders("my_config.hcl")).locals.my_value
}

locals {
  my_cmd = local.my_value == "some_value" ? jsondecode(run_cmd("--terragrunt-quiet", "python3", "./my_script.py")) : {}
}

When running without --terragrunt-quiet I see it's executed twice before terraform init.

@brikis98
Copy link
Member

Fixed in #1763 and released in https://github.com/gruntwork-io/terragrunt/releases/tag/v0.31.6.

@rhoboat rhoboat changed the title run_cmd in locals block exectuted twice run_cmd in locals block executed twice Apr 11, 2022
@Tzrlk
Copy link

Tzrlk commented Jul 15, 2022

I think this may have regressed. I'm running v0.36.7 and the script is executing several times without caching the output:

>terragrunt apply
Please enter the deployment type: test
Deployment type 'test' selected.
test
Please enter the deployment type: test
Deployment type 'test' selected.
test
Please enter the deployment type: test
Deployment type 'test' selected.
test
Please enter the deployment type: test
Deployment type 'test' selected.
test
Please enter the deployment type: test
Deployment type 'test' selected.
test
time=2022-07-15T12:58:14+12:00 level=info msg=Executing hook: AwsAuthCheck
locals {
	config = yamldecode(file("${get_parent_terragrunt_dir()}/config.yml"))
	deploy_type = trimspace(lower(run_cmd("sh", "${get_parent_terragrunt_dir()}/../bin/get-deploy-type.sh")))
	# ...

The bash script just prompts for valid input and only prints to stdout at the very end. Once all invocations of the script succeed, everything works as expected. Happy to know if I've just missed something.

@denis256
Copy link
Member

Hi,
I think it is a side effect of #667 which introduced multiple time evaluations of HCL files to find value of iam_role/get_aws_account_id/get_aws_caller_identity_arn/get_aws_caller_identity_user_id and invalidate cached run_cmd since AWS data may be used in arguments

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

8 participants