[INF-178] Fetch Brainstore license and Datadog keys from Secrets Manager - #307
[INF-178] Fetch Brainstore license and Datadog keys from Secrets Manager#307Erik Weathers (erikdw) wants to merge 7 commits into
Conversation
Keep license and observability API keys out of EC2 user_data and gateway ECS plain env vars so plans stay readable and credentials are not baked into launch templates or task definitions. Co-authored-by: Cursor <cursoragent@cursor.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d5bdbdaac2
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "Codex (@codex) review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "Codex (@codex) address that feedback".
| resource "aws_iam_role_policy" "task_execution_observability_secrets" { | ||
| count = local.observability_enabled ? 1 : 0 | ||
| resource "aws_iam_role_policy" "task_execution_secrets" { | ||
| count = local.observability_enabled || local.license_key_enabled ? 1 : 0 |
There was a problem hiding this comment.
When create_ai_gateway is enabled with the default internal-observability setting, brainstore_license_key_secret_arn is the ARN of a secret being created in this same apply, so it is unknown during planning; consequently local.license_key_enabled and this count are unknown. Terraform requires count to be known while constructing the plan and will reject the normal one-pass plan, requiring a targeted/multi-phase apply. Pass the already-known creation boolean into this module or otherwise make the count independent of the computed ARN.
AGENTS.md reference: AGENTS.md:L56-L62
Useful? React with 👍 / 👎.
| local.license_key_enabled ? [ | ||
| { | ||
| name = "BRAINSTORE_LICENSE_KEY" | ||
| valueFrom = var.brainstore_license_key_secret_arn |
There was a problem hiding this comment.
When brainstore_license_key is rotated after deployment, this task-definition value remains the same secret ARN, so Terraform does not create a new task-definition revision or roll the ECS service. ECS injects secret values only when tasks start, leaving all running gateway tasks on the previous license indefinitely until an unrelated restart or deployment occurs; include the secret version in the task definition or otherwise trigger a service rollout when the version changes.
AGENTS.md reference: AGENTS.md:L56-L62
Useful? React with 👍 / 👎.
Pass a plan-known enable flag into gateway-ecs so the task-exec secrets policy count is never derived from a computed secret ARN, and pin the license secret version in the task definition so key rotations roll ECS. Co-authored-by: Cursor <cursoragent@cursor.com>
Pin DD_API_KEY / FireLens apikey valueFrom to the secret version so rotating internal_observability_api_key revises gateway and API ECS task definitions and rolls services. Also pass a plan-known enable flag into api-ecs instead of deriving it from the computed secret ARN. Co-authored-by: Cursor <cursoragent@cursor.com>
coalesce(null, "") fails plan when brainstore_license_key is unset; use a null ternary instead. Also pin ECS secrets by version-id only (empty stage) so out-of-band AWSCURRENT moves cannot disagree with the pinned version. Co-authored-by: Cursor <cursoragent@cursor.com>
Cover the null brainstore_license_key path by validating the external-EKS example without a license key in mise validate / validate-tofu (already run by Terraform CI). Co-authored-by: Cursor <cursoragent@cursor.com>
|
Follow-up: plan-mode regression coverage for unset → #308 |
Prevent gateway/API tasks and Brainstore instance refreshes from starting before Secrets Manager GetSecretValue is granted on the execution/instance roles, which can fail secret resolution and trip the ECS circuit breaker. Co-authored-by: Cursor <cursoragent@cursor.com>
Soften comments that overclaimed CI coverage of the null license path; validate only checks wiring, while plan-mode terraform test is deferred. Co-authored-by: Cursor <cursoragent@cursor.com>
1 participant
Summary
BRAINSTORE_LICENSE_KEYvia ECS task secrets.brainstore_license_keyandinternal_observability_api_keyas sensitive.brainstore_license_keyunset (wiring/syntax coverage only —validatedoes not catch eval-time coalesce-on-null).https://linear.app/braintrustdata/issue/INF-178
Follow-up: plan-mode
terraform testfor the null license path → #308Test plan
mise run lint/mise run validate(includes external-EKS example with license key unset — wiring only)secretsfor licenseBRAINSTORE_LICENSE_KEYfrom SM