Skip to content

Commit fd0cb97

Browse files
weastlerCopilot
andcommitted
chore(infrastructure): merge reviewed deployment foundations
Preserve the foundations native command-description suite and release selector, exclude that suite from Vitest, and retain the wizard recovery and operator-input safeguards. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2 parents 9595a12 + b161744 commit fd0cb97

12 files changed

Lines changed: 474 additions & 60 deletions

File tree

‎.github/workflows/hub-infrastructure.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -42,7 +42,7 @@ jobs:
4242
- uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
4343
with:
4444
node-version: 24
45-
- run: node --test deploy-hub/index.test.ts stop-hub/index.test.ts
45+
- run: node --test deploy-hub/index.test.ts stop-hub/index.test.ts ../installer/command/describe/index.test.ts
4646

4747
installer:
4848
name: Installer ${{ matrix.command }}

‎hub/docs/deployment.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ The Hub image runs the frontend, API, and both workers together. It can expose G
66

77
Build from the repository root with `docker build -t replica-hub:check hub`. When an npm proxy is required, add `--build-arg NPM_REGISTRY=https://YOUR_PROXY/`. This resolves locked dependency versions through that registry without changing the source lockfile; the registry URL must not contain credentials. The image runs as a non-root user and contains compiled application code and migration tooling. It writes no application state to its writable layer. Publish and deploy the same image digest for setup and Hub.
88

9-
Host npm configuration is not inherited by the image build. Without `NPM_REGISTRY`, the container needs public npm access. Use `--build-arg NPM_REGISTRY=https://packagefeedproxy.microsoft.io/npm/` where public npm access is not allowed. Explicit scoped registry mappings in the copied `.npmrc` still apply.
9+
Host npm configuration is not inherited by the image build. Without `NPM_REGISTRY`, the container needs public npm access. Where that access is not allowed, supply your approved proxy using the build argument above. Explicit scoped registry mappings in the copied `.npmrc` still apply.
1010

1111
Publishing is an explicit operator step, not a side effect of CI. From the approved checkout, authenticate to an authorized registry, then run `docker buildx build --platform linux/amd64 --tag YOUR_REGISTRY/replica-hub:YOUR_COMMIT --push hub`. Resolve the published digest with `docker buildx imagetools inspect YOUR_REGISTRY/replica-hub:YOUR_COMMIT` and give deployment the `registry/repository@sha256:...` reference, never the mutable tag.
1212

‎infrastructure/README.md‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -218,7 +218,7 @@ Each installation has its own resource group tagged with its application, instal
218218

219219
Two optional modules have separate Terraform states and lifecycles. [`azure/foundation/`](azure/foundation/main.tf) creates an installation-owned ACR and RBAC-enabled Key Vault in a separate resource group, with `AcrPush` and `Key Vault Secrets Officer` grants for the deployment identity. The vault has 90-day soft deletion and purge protection. Pass its registry and vault outputs to the main installation; the main module still accepts existing resources and never owns their teardown.
220220

221-
[`azure/provision/`](azure/provision/main.tf) creates a temporary private-network database-role provisioning job and identity in the installation's Container Apps environment. It accepts the same immutable image and exactly three versioned Key Vault references: `DATABASE_URL` for the administrator, `HUB_MIGRATION_PASSWORD`, and `HUB_RUNTIME_PASSWORD`. The job has one replica, no automatic retries, and a ten-minute timeout. An authorized operator must run it once against a fresh database, verify that exact execution succeeded, and remove the temporary job, identity, and grants before release. Do not rerun it against an installed database or grant its administrator secret to Hub or setup. The existing private-network operator script remains an alternative.
221+
[`azure/provision/`](azure/provision/main.tf) creates a temporary private-network database-role provisioning job and identity in the installation's Container Apps environment. It accepts the same immutable image and exactly three versioned Key Vault references: `DATABASE_URL` for the administrator, `HUB_MIGRATION_PASSWORD`, and `HUB_RUNTIME_PASSWORD`. Each secret URL must belong to the supplied `vault_id`. Omit `registry_id` for a public image; an existing ACR requires its resource ID and grants only the provisioning identity pull access. The job has one replica, no automatic retries, and a ten-minute timeout. An authorized operator must run it once against a fresh database, verify that exact execution succeeded, and remove the temporary job, identity, and grants before release. Do not rerun it against an installed database or grant its administrator secret to Hub or setup. The existing private-network operator script remains an alternative.
222222

223223
Initialize each optional module against its own protected backend state key, review its declared inputs and Terraform plan, and authorize its apply separately. Never share a state key with the main installation. Neither module starts Hub or replaces the serialized release command.
224224

@@ -246,7 +246,7 @@ The deployment identity needs resource creation/update rights for the installati
246246

247247
The main module's optional `grant_deployment_storage_access=true` grants the authenticated Terraform deployment identity `Storage Blob Data Contributor` on the installation account. It defaults to `false`; use it only when this account-level grant matches the deployment policy. It does not grant access to the separate state backend.
248248

249-
Register `Microsoft.App`, `Microsoft.OperationalInsights`, `Microsoft.Network`, `Microsoft.Storage`, `Microsoft.DBforPostgreSQL`, and `Microsoft.ManagedIdentity` beforehand. The provider does not register services across the subscription. Allow time for newly assigned RBAC roles to propagate before setup.
249+
Register `Microsoft.App`, `Microsoft.OperationalInsights`, `Microsoft.Network`, `Microsoft.Storage`, `Microsoft.DBforPostgreSQL`, and `Microsoft.ManagedIdentity` beforehand. Before applying the optional foundation module, also register `Microsoft.ContainerRegistry` and `Microsoft.KeyVault`. The providers do not register services across the subscription. Allow time for newly assigned RBAC roles to propagate before setup.
250250

251251
Copy [`backend.hcl.example`](azure/backend.hcl.example) to `infrastructure/azure/backend.hcl` and [`installation.tfvars.example`](azure/installation.tfvars.example) to `infrastructure/azure/installation.tfvars`. Both real files are ignored. Replace every placeholder, choose a globally unique name, and resolve an existing image to its real `registry/repository@sha256:...` digest. Use either a public image or an existing ACR; private non-ACR registry credentials and ACR's ABAC repository-permissions mode are not covered by this reference.
252252

@@ -298,7 +298,7 @@ Azure can activate the revision during Terraform rollout. Requesting activation
298298

299299
Successful Azure operations that return no data, such as lease release, are checked by exit status rather than parsed as JSON. Responses used for deployment decisions still require valid JSON. Invalid replies name the operation without disclosing tool output.
300300

301-
Plans reject resource deletion and replacement during normal releases. A database major upgrade, region move, rename, or replacement needs a reviewed migration/recovery plan, not an override of that guard. The release script stores sensitive plans under `TF_DATA_DIR` when set, otherwise under the module's `.terraform/`, and deletes them on exit. Failures identify a fixed command label and available exit code without printing arguments or tool output. It retains an acquired lease on failure. A failure after stopping old revisions but before rollout leaves Hub stopped; a failure after rollout can leave the new revision running. The script never reactivates an older process automatically against a possibly migrated schema.
301+
Plans reject resource deletion and replacement during normal releases. A database major upgrade, region move, rename, or replacement needs a reviewed migration/recovery plan, not an override of that guard. The release script stores sensitive plans under `TF_DATA_DIR` when nonempty, otherwise under the module's `.terraform/`, and deletes them on exit. Relative `TF_DATA_DIR` paths resolve against `infrastructure/azure/`, matching Terraform's `-chdir` behavior; protect that directory if it is outside the module. Failures identify a fixed command label and available exit code without printing arguments or tool output. It retains an acquired lease on failure. A failure after stopping old revisions but before rollout leaves Hub stopped; a failure after rollout can leave the new revision running. The script never reactivates an older process automatically against a possibly migrated schema.
302302

303303
### Recover a failed release
304304

@@ -360,7 +360,7 @@ terraform init -backend=false -input=false -lockfile=readonly
360360
terraform fmt -check -recursive
361361
terraform validate
362362
terraform test
363-
node --test deploy-hub/index.test.ts stop-hub/index.test.ts
363+
node --test deploy-hub/index.test.ts stop-hub/index.test.ts ../installer/command/describe/index.test.ts
364364
```
365365

366366
Run the same Terraform initialization, formatting, validation, and test commands from `foundation/` and `provision/` to check the optional modules. CI checks each module using both supported Terraform versions in [the workflow matrix](../.github/workflows/hub-infrastructure.yml).

‎infrastructure/azure/deploy-hub/index.test.ts‎

Lines changed: 106 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,41 @@
11
import assert from "node:assert/strict"
22
import childProcess from "node:child_process"
3-
import { afterEach, describe, it, mock } from "node:test"
3+
import { randomUUID } from "node:crypto"
4+
import { existsSync, mkdirSync, rmSync, statSync, writeFileSync } from "node:fs"
5+
import { basename, dirname, resolve } from "node:path"
6+
import { afterEach, beforeEach, describe, it, mock } from "node:test"
47
import timers from "node:timers/promises"
58
import { inspect } from "node:util"
69

710
import { describeCommand } from "../../installer/command/describe/index.ts"
811
import { deployHub } from "./index.ts"
912

13+
const terraformRoot = resolve(import.meta.dirname, "..")
14+
let fixtureDirectory: string
15+
let previousDataDirectory: string | undefined
16+
let previousMask: number
17+
let defaultDirectoryExisted: boolean
18+
19+
beforeEach(() => {
20+
previousDataDirectory = process.env.TF_DATA_DIR
21+
previousMask = process.umask()
22+
defaultDirectoryExisted = existsSync(resolve(terraformRoot, ".terraform"))
23+
fixtureDirectory = resolve(terraformRoot, `.deploy-hub-test-${randomUUID()}`)
24+
mkdirSync(fixtureDirectory, { mode: 0o700 })
25+
process.env.TF_DATA_DIR = resolve(fixtureDirectory, "terraform")
26+
})
27+
1028
afterEach(() => {
1129
mock.restoreAll()
30+
if (previousDataDirectory === undefined) delete process.env.TF_DATA_DIR
31+
else process.env.TF_DATA_DIR = previousDataDirectory
32+
process.umask(previousMask)
33+
rmSync(fixtureDirectory, { recursive: true, force: true })
34+
const callerDirectory = resolve(basename(fixtureDirectory))
35+
if (callerDirectory !== fixtureDirectory)
36+
rmSync(callerDirectory, { recursive: true, force: true })
37+
if (!defaultDirectoryExisted)
38+
rmSync(resolve(terraformRoot, ".terraform"), { recursive: true, force: true })
1239
})
1340

1441
function installationScenario({
@@ -26,6 +53,7 @@ function installationScenario({
2653
httpStatuses = [200],
2754
commandFailure,
2855
commandOutput,
56+
onPlan,
2957
}: {
3058
appCreated?: boolean
3159
setupStatus?: string
@@ -36,6 +64,7 @@ function installationScenario({
3664
httpStatuses?: number[]
3765
commandFailure?: { args: string[]; error: Error; occurrence?: number }
3866
commandOutput?: { args: string[]; stdout: string }
67+
onPlan?: (path: string) => void
3968
} = {}) {
4069
const installation = {
4170
name: "test-hub",
@@ -77,6 +106,7 @@ function installationScenario({
77106
return appCreated ? "azurerm_container_app.hub[0]" : ""
78107
case "plan":
79108
targeted = args.some((argument: string) => argument.startsWith("-target="))
109+
onPlan?.(args.find((argument) => argument.startsWith("-out="))!.slice(5))
80110
return ""
81111
case "show":
82112
return JSON.stringify({
@@ -118,11 +148,17 @@ function installationScenario({
118148
}
119149
if (args.includes("release") && releaseError) throw releaseError
120150
return ""
121-
case "containerapp revision list":
122-
return JSON.stringify([
151+
case "containerapp revision list": {
152+
const revisions = [
123153
{ name: "old-active", properties: { active: true } },
124154
{ name: "old-terminating", properties: { active: false } },
125-
])
155+
]
156+
return JSON.stringify(
157+
args.includes("--all")
158+
? revisions
159+
: revisions.filter((revision) => revision.properties.active),
160+
)
161+
}
126162
case "containerapp replica list": {
127163
const revision = args[args.indexOf("--revision") + 1]
128164
const reads = (replicaReads.get(revision) ?? 0) + 1
@@ -170,6 +206,72 @@ function installationScenario({
170206
}
171207

172208
describe("deployHub()", () => {
209+
for (const dataDirectory of ["relative", "absolute", "empty", "unset"]) {
210+
it(`keeps sensitive plans in Terraform's data directory when TF_DATA_DIR is ${dataDirectory}`, async () => {
211+
if (dataDirectory === "relative")
212+
process.env.TF_DATA_DIR = `${basename(fixtureDirectory)}/relative`
213+
else if (dataDirectory === "empty") process.env.TF_DATA_DIR = ""
214+
else if (dataDirectory === "unset") delete process.env.TF_DATA_DIR
215+
const expectedDirectory =
216+
dataDirectory === "relative"
217+
? resolve(fixtureDirectory, "relative")
218+
: dataDirectory === "absolute"
219+
? resolve(fixtureDirectory, "terraform")
220+
: resolve(terraformRoot, ".terraform")
221+
const directoryExisted = existsSync(expectedDirectory)
222+
const planFiles: { path: string; directoryMode: number; fileMode: number }[] = []
223+
const { args, events } = installationScenario({
224+
onPlan(path) {
225+
writeFileSync(path, "synthetic-sensitive-plan")
226+
planFiles.push({
227+
path,
228+
directoryMode: statSync(dirname(path)).mode & 0o777,
229+
fileMode: statSync(path).mode & 0o777,
230+
})
231+
},
232+
})
233+
process.umask(0o022)
234+
235+
await deployHub(args)
236+
237+
assert.equal(process.umask(), 0o022)
238+
assert.equal(planFiles.length, 2)
239+
for (const { path, directoryMode, fileMode } of planFiles) {
240+
assert.equal(dirname(path), expectedDirectory)
241+
if (!directoryExisted) assert.equal(directoryMode, 0o700)
242+
assert.equal(fileMode, 0o600)
243+
assert.equal(existsSync(path), false)
244+
}
245+
assert.ok(
246+
events
247+
.filter((event) => event[0] === "terraform")
248+
.every((event) => event[1] === `-chdir=${terraformRoot}`),
249+
)
250+
})
251+
}
252+
253+
it("deletes a sensitive saved plan and restores the umask after plan inspection fails", async () => {
254+
let planPath: string | undefined
255+
const { args } = installationScenario({
256+
onPlan(path) {
257+
planPath = path
258+
writeFileSync(path, "synthetic-sensitive-plan")
259+
},
260+
commandFailure: {
261+
args: [`-chdir=${terraformRoot}`, "show"],
262+
error: new Error("Synthetic plan inspection failure"),
263+
},
264+
})
265+
mock.method(console, "error", () => undefined)
266+
process.umask(0o022)
267+
268+
await assert.rejects(deployHub(args), /Terraform command failed/)
269+
270+
assert.ok(planPath)
271+
assert.equal(existsSync(planPath), false)
272+
assert.equal(process.umask(), 0o022)
273+
})
274+
173275
it("drains all old revisions and completes setup before rollout, readiness, and lease release", async () => {
174276
const { args, events, replicaReads, commandOptions } = installationScenario()
175277
const output: unknown[][] = []

‎infrastructure/azure/deploy-hub/index.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -54,7 +54,7 @@ export async function deployHub(args: string[] = process.argv.slice(2)): Promise
5454
const subscriptionId = subscription
5555
const variablesPath = resolve(variables)
5656
const root = resolve(import.meta.dirname, "..")
57-
const planDirectory = resolve(process.env.TF_DATA_DIR ?? resolve(root, ".terraform"))
57+
const planDirectory = resolve(root, process.env.TF_DATA_DIR || ".terraform")
5858
const planPath = resolve(planDirectory, `deployment-${randomUUID()}.tfplan`)
5959
const previousMask = process.umask(0o077)
6060
let lease: string | undefined

‎infrastructure/azure/provision/main.tf‎

Lines changed: 28 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -20,15 +20,25 @@ variable "name" { type = string }
2020
variable "location" { type = string }
2121
variable "resource_group" { type = string }
2222
variable "environment_id" { type = string }
23-
variable "registry_id" { type = string }
23+
variable "registry_id" {
24+
type = string
25+
default = null
26+
description = "Optional existing ACR resource ID. Omit for a public image."
27+
}
2428
variable "image" {
2529
type = string
2630
validation {
2731
condition = can(regex("^[a-z0-9][a-z0-9.:/-]+@sha256:[a-f0-9]{64}$", var.image))
2832
error_message = "Use an immutable registry/repository@sha256 digest for provisioning."
2933
}
3034
}
31-
variable "vault_id" { type = string }
35+
variable "vault_id" {
36+
type = string
37+
validation {
38+
condition = can(regex("^/subscriptions/[^/]+/resourceGroups/[^/]+/providers/Microsoft.KeyVault/vaults/[^/]+$", var.vault_id))
39+
error_message = "Supply the Key Vault resource ID that owns the provisioning secrets."
40+
}
41+
}
3242
variable "secret_urls" {
3343
type = map(string)
3444
validation {
@@ -40,6 +50,15 @@ variable "secret_urls" {
4050
)
4151
error_message = "Provide exactly three versioned Key Vault references for the administrator URL and two role passwords, never raw values."
4252
}
53+
validation {
54+
condition = (
55+
!can(regex("^/subscriptions/[^/]+/resourceGroups/[^/]+/providers/Microsoft.KeyVault/vaults/[^/]+$", var.vault_id)) ||
56+
alltrue([for secret in values(var.secret_urls) :
57+
lower(try(split("/", secret)[2], "")) == "${lower(try(split("/", var.vault_id)[8], ""))}.vault.azure.net"
58+
])
59+
)
60+
error_message = "Every provisioning secret URL must belong to vault_id."
61+
}
4362
}
4463

4564
resource "azurerm_user_assigned_identity" "provision" {
@@ -56,6 +75,7 @@ resource "azurerm_role_assignment" "secrets" {
5675
}
5776

5877
resource "azurerm_role_assignment" "image" {
78+
count = var.registry_id == null ? 0 : 1
5979
scope = var.registry_id
6080
role_definition_name = "AcrPull"
6181
principal_id = azurerm_user_assigned_identity.provision.principal_id
@@ -78,9 +98,12 @@ resource "azurerm_container_app_job" "provision" {
7898
type = "UserAssigned"
7999
identity_ids = [azurerm_user_assigned_identity.provision.id]
80100
}
81-
registry {
82-
server = split("/", var.image)[0]
83-
identity = azurerm_user_assigned_identity.provision.id
101+
dynamic "registry" {
102+
for_each = var.registry_id == null ? [] : [var.registry_id]
103+
content {
104+
server = split("/", var.image)[0]
105+
identity = azurerm_user_assigned_identity.provision.id
106+
}
84107
}
85108
dynamic "secret" {
86109
for_each = var.secret_urls

0 commit comments

Comments
 (0)