Terragrunt and TFLint above 0.50.0 breaking behavior

[velkovb]

Describe the enhancement

TFLint introduced a new way of handling values for undeclared variables in 0.50.0 - terraform-linters/tflint#1941

This directly affects the Terragrunt hook integration as it does the following:

tflint --config ./.terragrunt-cache/ellCj9USNaAqucT2-U5SmL6rf1A/Gu7P37gZ1L3NBihh7GJE9nHhW38/.tflint.hcl --chdir ./.terragrunt-cache/ellCj9USNaAqucT2-U5SmL6rf1A/Gu7P37gZ1L3NBihh7GJE9nHhW38 --var=test=foo

With Terragrunt release of 0.69.0 which bumped TFLint from 0.47.0 to 0.50.3 this started breaking our projects. We do generate a lot of metadata and automatic variables that are not used by every project. terraform-linters/tflint#2080 (comment)

The way that variables are passed to TFLint doesn't match the way they are passed to OpenTofu/ Terraform. For TFLint it seems to be with --var while for the latter it is TF_VAR_* which has no issues. If we pass OpenTofu/Terraform with --var it will error out as well.

Would be great if we can align both approaches and pass variables from input to TFLint as TF_VAR_* as well. If that cannot be done, passing a .tfvars file should work as well. I would guess the magic needs to happen somewhere here -

terragrunt/tflint/tflint.go

Line 154 in 407f65f

func InputsToTflintVar(inputs map[string]interface{}) ([]string, error) {

Not sure if this is a bug or enhancement

Additional context

Things you might want to address include:

• Changes required.
• Implications of the feature.
• Alternatives considered.
• Level of effort.

PoC (Proof of Concept)

Link to a Proof of Concept if you have one:

• PoC

Including a PoC can help others understand the feature better and implement it faster.

RFC Not Needed

• I have evaluated the complexity of this enhancement, and I believe it does not require an RFC.

[yhakbar]

Hey @velkovb ,

Thanks for creating this issue.

Unfortunately, given that there's a very straightforward workaround (calling tflint with the binary installed in PATH however users like), there isn't much incentive to adjusting how the built-in tflint hook works.

That being said, if this issue is important to you, would you consider contributing the code required to have tflint behave how you like? Even if you don't have a lot of Golang experience, we'd be happy to help you ramp up and collaborate with you on developing a solution.

Our focus is on delivering the features and updates that make Terragrunt ready for 1.0, so I'm jealously guarding the attention of the maintainers for that when I can. I hope you understand.

[velkovb]

@yhakbar Thanks for getting back to me.

Initially, I started using the built-in tflint hook as it worked under the .terragrunt-cache folder where the project is expanded. Back then I didn't find it that straightforward to directly run tflint in that folder. Do you mean that as the workaround or to pass the --terragrunt-external-tflint option? The latter still has the same issue.

Giving it a go crossed my mind. Might try it over the holidays :)

[yhakbar]

Ya, I was discussing the --terragrunt-external-tflint option with an older version of tflint installed so that you have the older behavior.

Please make sure to read these docs before you start contributing so that you have some guidance on how to get started.

The steps I generally recommend for new contributors is to start with the following:

1. Build Terragrunt locally, so that you can play with your own version of Terragrunt.
2. Run the relevant tests (you don't need to run the tests for the entire project, just the stuff that you're interested in changing).
3. Write a test that produces the outcomes you're looking for, and get those tests to fail the way you expect.
4. Write code that fixes the tests so that you have the new behavior you want.

I also heavily encourage you to join the Discord if you haven't. This can be a good place to ask questions about best practices, or to just become an active member of our community.

[velkovb]

Thanks, already in discord :).

My journey started because I actually wanted a newer version of TFLint that includes this fix - terraform-linters/tflint#2124 as we had problems with auto generated providers. So I will try to push forward

[velkovb]

@yhakbar I would guess that this logic is already implemented where the inputs are transformed for OpenTofu/ Terraform somewhere in the codebase. I could find it on my first attempt, could I ask for some directions where it is?

[wakeful]

hey,

Have you already read the main function where the tflint CLI is being constructed?

https://github.com/gruntwork-io/terragrunt/blob/main/tflint/tflint.go#L31

[velkovb]

We ended up with just calling TFLint with a custom hook, not the integrated one as we were not using the variables anyway. We have plans to enchance TFLint performance in the future and might revisit this than.

[NicolaSpreafico]

We ended up with just calling TFLint with a custom hook, not the integrated one as we were not using the variables anyway. We have plans to enchance TFLint performance in the future and might revisit this than.

Hello,
I have the similar issue as you, precisely this:

We do generate a lot of metadata and automatic variables that are not used by every project

Right now I installed locally TFLINT 0.49.0 and I added the "--terragrunt-external-tflint" parameter in the TFLINT, but this is forcing me to remain on a fixed and older version of TFLint.

Can I ask you what is your solution about the custom hook?

Is there a way to tell TFLint to simply ignore this specific rule?

"A variable named "xxxx" was assigned, but the root module does not declare a variable of that name."

?

Thank you

[velkovb]

@NicolaSpreafico

terraform {
  before_hook "tflint" {
    commands = ["validate"]
    execute  = ["${get_repo_root()}/scripts/tflint.sh", "${get_repo_root()}/deployment/.tflint.hcl"]
  }
}

#!/bin/bash

tflint --init --config ${1}

[NicolaSpreafico]

@velkovb Thank you.

This is a mere alternative on using "--terragrunt-external-tflint" or does it actually solve the problem I mentioned of variable assigned and not deckared?

Thank you

[velkovb]

This doesn't use the Terragrunt integrated TFLint hook which actually passed the assigned variables so it doesn't have that problem. The issue here is it strips all variables even some that might have been actually used but that works for us for now. As described previously we plan on either trying to add an option to pass the variables as TF_VAR_* instead of with --var or doing something similar with the external hook.

[NicolaSpreafico]

I'm not sure how this can work, you only provide the --config so that you can install the plugins.
But if I'm running the terragrunt command where the .hcl file exists, tflint should work on the directory where the Terraform sources are (based on your structure project), the folder named .terragrunt-cache/xxx/xxx

Right now I found this combination to be working properly:

execute  = [
      "${get_repo_root()}/scripts/tflint.sh", 
      "${get_repo_root()}/project/live/.tflint.hcl", 
      "${get_working_dir()}"
    ]

and this

# https://github.com/terraform-linters/tflint?tab=readme-ov-file#usage
tflint --config "${1}" --chdir "${2}" --color

I removed --init from your example because with that it was only initializing but the analysis was not performed (and no errors)