Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add env var for PluginCacheMayBreakDependencyLockFile #761

Merged
merged 4 commits into from
Jun 17, 2024

Conversation

smonero
Copy link
Contributor

@smonero smonero commented Jun 14, 2024

The situation:

In the attempt to upgrade projects to terraform 1.5.7 (mostly from 1.3.1) we encountered strange behavior where it would not use the cache despite the providers (i.e. aws and github) having already been downloaded to the cache. This resulted in us seeing an error text file busy when different roots would be trying to access the cache (there were also problems with an incorrect checksum, which was probably caused by the file being overwritten / corrupted) It turns out there was an underlying change in terraform 1.4 in regards to how lockfiles interact with the plugin cache.

From this pr hashicorp/terraform@d0a35c6
This change addresses that problem by essentially flipping the decision so that we'll prioritize the lock file behavior over the provider cache behavior. Now a global cache entry is eligible for use if and only if the lock file already contains a checksum that matches the cache entry. This means that the first time a particular configuration sees a new provider it will always be fetched from the configured installation source (typically the origin registry) and record the checksums from that source.

This PR adds the env var which tells terraform to do the old behavior (pre 1.4), which works for us because it will stop our concurrency issues by not overwriting the cache each time.

Note this is a somewhat controversial fix
hashicorp/terraform#32205
https://developer.hashicorp.com/terraform/cli/config/config-file#allowing-the-provider-plugin-cache-to-break-the-dependency-lock-file

Note that other solutions were considered (such as just disabling cache) but those were determined to be suboptimal so not appropriate.

@smonero
Copy link
Contributor Author

smonero commented Jun 16, 2024

tested in staging and seems to work

@asubrocky
Copy link
Contributor

Let's add a detailed commit description explaining what we saw before and how this fixes it

@smonero smonero merged commit 0c16f78 into main Jun 17, 2024
3 checks passed
@smonero smonero deleted the tryEnvVarCacheIssue branch June 17, 2024 18:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants