Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial SQS queue + user policy #377

Merged
merged 2 commits into from
Jan 9, 2024
Merged

Conversation

Mark-Simulacrum
Copy link
Member

This adds a new terragrunt module for the new crates.io account(s) and puts the SQS queue and related IAM policies into the module. However, it does not yet wire up the account or try to deploy this -- I haven't connected those bits yet and getting terragrunt to cooperate with me is usually pretty finicky. Hopefully this helps as a starting point though.

Remaining steps:

  • Confirm whether the IAM user + hardcoded, non-rotating access key is the best way for Heroku crates.io to access this account (at least in the short term)
  • Confirm this all actually works
  • Create the resources

Possibly:

  • Figure out if we want a dead letter queue. This would prevent a message crates.io doesn't know how to read from blocking all other messages in the queue (since it'll constantly fail and crates.io shouldn't call DeleteMessage on it). But, it's more complexity and it's not obvious that any particular message should be any different than others. I think it's not unreasonable to punt on setting this additional infrastructure up for now.
  • Figure out alarming/metrics -- maybe crates.io should self-drive this with the queue metadata API, maybe we want to wire up datadog/grafana to read from this account.

IMO if we can, let's deploy this before we actually finish wiring up s3 -- crates.io can start reading from the queue and such early that way and kick the tires.

cc #372

r? @jdno cc @Turbo87

terragrunt/modules/crates-io-logs/main.tf Outdated Show resolved Hide resolved
@jdno
Copy link
Member

jdno commented Jan 8, 2024

This looks like a good starting point. 👍 Can I help you with Terragrunt somehow?

Co-authored-by: Jan David <[email protected]>
@Mark-Simulacrum
Copy link
Member Author

Hm, I'm not sure there's a good intermediate step, I can probably copy what I did for bors with some adjustments for the deployed-ref business...

@Mark-Simulacrum
Copy link
Member Author

I'm also happy for you to just take this and run with it.

Copy link
Member

@jdno jdno left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let me just run with this for now and make a note to go over our documentation for Terragrunt.

@jdno
Copy link
Member

jdno commented Jan 9, 2024

I'm gonna merge this and continue in a separate branch so that we review new changes more easily.

@jdno jdno merged commit bbab800 into rust-lang:master Jan 9, 2024
3 checks passed
jdno added a commit to jdno/rust-simpleinfra that referenced this pull request Jan 9, 2024
The SQS queue that was configured in rust-lang#377 has been deployed to the new
staging account for crates.io that was created in rust-lang#374. Slight
modifications were necessary to the configuration:

  - The resource and human-readable names of the SQS are now identical.
  - The `sid` for policies matches the naming rules of AWS.
  - The input variable has been changed, since the account number is not
    part of a bucket's ARN and can thus not be extracted from it.

The infrastructure has been deployed with the same version of the
Terraform provider for AWS as the other modules in simpleinfra to ensure
future compatibility.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants