|
1 | 1 | # Google Cloud Run example |
2 | 2 |
|
3 | | -End-to-end deployment + smoke for [`@hyperframes/gcp-cloud-run`](../../packages/gcp-cloud-run) — the Cloud Run + Cloud Workflows adapter for HyperFrames distributed rendering. |
| 3 | +End-to-end deployment and parity testing for |
| 4 | +[`@hyperframes/gcp-cloud-run`](../../packages/gcp-cloud-run), the Cloud Run + |
| 5 | +Cloud Workflows adapter for HyperFrames distributed rendering. |
4 | 6 |
|
5 | 7 | ## Layout |
6 | 8 |
|
| 9 | +```text |
| 10 | +scripts/smoke.sh Owner-isolated real-GCP deploy, render, parity, cleanup |
| 11 | +sample-events/ v1 and v2 handler request examples |
7 | 12 | ``` |
8 | | -scripts/smoke.sh Real-GCP smoke: build → deploy → render → PSNR → destroy |
9 | | -sample-events/ Example request bodies for the Cloud Run handler |
10 | | - (plan.json, render-chunk.json, assemble.json) |
11 | | -``` |
12 | 13 |
|
13 | | -The Terraform module and the Cloud Workflows definition that the smoke deploys live with the package, at `packages/gcp-cloud-run/terraform/` (including `workflow.yaml`). |
| 14 | +The Terraform module and Cloud Workflows definition live in |
| 15 | +`packages/gcp-cloud-run/terraform/`. |
| 16 | + |
| 17 | +## Protocol rollout |
| 18 | + |
| 19 | +The workflow defaults to plan protocol v1 when `PlanProtocol` is absent. V2 is |
| 20 | +accepted only when the caller explicitly sends `PlanProtocol: "v2"`. |
| 21 | + |
| 22 | +V1 and v2 use disjoint plan locators: |
| 23 | + |
| 24 | +- v1: `PlanGcsUri` |
| 25 | +- v2: `PlanV2ManifestGcsUri` and `PlanV2ArtifactGcsPrefix` |
| 26 | + |
| 27 | +The workflow validates that the plan response matches the selected protocol |
| 28 | +before starting chunk fan-out. It never silently falls back from v2 to v1. |
| 29 | +Deploy the v2 workflow only with a Cloud Run image whose handler implements |
| 30 | +the matching v2 request/response contract. An older v1-only handler will keep |
| 31 | +serving default v1 requests, but explicit v2 smoke executions will fail closed. |
14 | 32 |
|
15 | 33 | ## Prerequisites |
16 | 34 |
|
17 | | -- `gcloud` authenticated, with a project that has **billing enabled** |
18 | | -- `terraform` (≥ 1.5), `docker`, `ffmpeg`, `jq` on PATH |
| 35 | +- `gcloud` authenticated to a project with billing enabled |
| 36 | +- `terraform` (>= 1.5), `ffmpeg`, `ffprobe`, `jq`, `tar`, and `sha256sum` |
| 37 | +- the required project APIs already enabled, plus permission to run Cloud |
| 38 | + Build and manage Cloud Run, Workflows, GCS, IAM service accounts, |
| 39 | + Monitoring, and Artifact Registry resources |
19 | 40 |
|
20 | 41 | ## Run the smoke |
21 | 42 |
|
| 43 | +V1 remains the safe default: |
| 44 | + |
22 | 45 | ```bash |
23 | | -# Renders the mp4-h264-sdr fixture through the workflow and PSNR-compares it |
24 | | -# against the in-process baseline, then tears the stack down. |
25 | | -./scripts/smoke.sh --project YOUR_GCP_PROJECT --region us-central1 |
| 46 | +./scripts/smoke.sh \ |
| 47 | + --project YOUR_GCP_PROJECT \ |
| 48 | + --region us-central1 |
| 49 | +``` |
26 | 50 |
|
27 | | -# Keep the stack up to poke at it: |
28 | | -./scripts/smoke.sh --project YOUR_GCP_PROJECT --keep-stack |
| 51 | +Explicitly run v1/v2 end-to-end parity at one or more chunk sizes: |
29 | 52 |
|
30 | | -# Render at several chunk sizes to see the fan-out scaling: |
31 | | -./scripts/smoke.sh --project YOUR_GCP_PROJECT --chunk-sizes 30,15,10 |
| 53 | +```bash |
| 54 | +./scripts/smoke.sh \ |
| 55 | + --project YOUR_GCP_PROJECT \ |
| 56 | + --region us-central1 \ |
| 57 | + --protocols v1,v2 \ |
| 58 | + --chunk-sizes 30,15,10 \ |
| 59 | + --owner plan-v2-parity |
32 | 60 | ``` |
33 | 61 |
|
34 | | -Outputs land in `scripts/gcp-smoke-artifacts/`: `results.json` |
35 | | -(`chunkSize × wallClockMs × psnrAvgDb`), the rendered MP4s, and each |
36 | | -workflow execution's describe output. |
| 62 | +For each chunk size, parity requires exact equality of: |
| 63 | + |
| 64 | +- decoded RGBA video frames |
| 65 | +- decoded 48 kHz stereo PCM audio |
| 66 | +- normalized `ffprobe` stream and duration metadata |
| 67 | + |
| 68 | +The encoded MP4 hash and byte count are recorded but are not the equality |
| 69 | +oracle because mux metadata can differ without changing decoded output. |
| 70 | +Each render is also PSNR-compared with the checked-in in-process fixture |
| 71 | +baseline. |
| 72 | + |
| 73 | +## Isolation and cleanup |
| 74 | + |
| 75 | +Every invocation hashes the owner, project, region, and a fresh invocation |
| 76 | +nonce into a unique resource prefix such as `hf-smoke-a1b2c3d4e5`. Reusing an |
| 77 | +owner label does not reuse old Terraform state or cloud resources. This prefix |
| 78 | +stays within GCP service account naming limits. The smoke: |
| 79 | + |
| 80 | +- never uses the static `hyperframes` prefix |
| 81 | +- copies the Terraform module into an owner-scoped work directory and uses an |
| 82 | + isolated Terraform data directory and state file |
| 83 | +- scopes GCS keys, render outputs, the image package/tag, and the default |
| 84 | + Artifact Registry repository to that owner |
| 85 | +- deletes only an image it built |
| 86 | +- deletes the Artifact Registry repository only when that invocation created it |
| 87 | +- refuses to enable project APIs, because APIs are shared project state |
| 88 | +- stages the bounded Cloud Build source archive in an owner-scoped bucket, |
| 89 | + writes build logs to Cloud Logging, and deletes the staging bucket |
| 90 | + |
| 91 | +Cleanup is on by default. It empties and destroys the owner-scoped bucket and |
| 92 | +stack, deletes owned image/repository/build-staging resources, then verifies |
| 93 | +the Cloud Run service, workflow, buckets, both service accounts, image, and any |
| 94 | +test-created repository are absent. Cleanup fails on API or authentication |
| 95 | +errors rather than interpreting them as successful deletion. GCP retains the |
| 96 | +Cloud Build execution record and Cloud Logging audit entries as project-level |
| 97 | +operational history; the smoke test does not attempt to erase audit records. |
| 98 | + |
| 99 | +`--keep-stack` deliberately retains the stack, image, and repository and |
| 100 | +prints the exact isolated state directory and Terraform cleanup commands. |
| 101 | +Never use it for unattended CI. |
| 102 | + |
| 103 | +Evidence lands under: |
| 104 | + |
| 105 | +```text |
| 106 | +scripts/gcp-smoke-artifacts/<owner-hash>/ |
| 107 | + results.json |
| 108 | + parity.json |
| 109 | + renders/ |
| 110 | + terraform/ |
| 111 | + terraform-data/ |
| 112 | +``` |
| 113 | + |
| 114 | +Use `--image` to test a caller-owned existing image. That image is never |
| 115 | +deleted. `--skip-build` requires `--image`; new invocations never inherit an |
| 116 | +old invocation's state or image implicitly. |
37 | 117 |
|
38 | 118 | ## Test the handler locally |
39 | 119 |
|
40 | | -The sample events exercise the same body shape Cloud Workflows sends. With the |
41 | | -container running locally (`PORT=8080`) and credentials that can reach a GCS |
42 | | -bucket, you can drive a single action: |
| 120 | +The sample events mirror the request bodies sent by Cloud Workflows: |
43 | 121 |
|
44 | 122 | ```bash |
| 123 | +# V1 |
45 | 124 | curl -sX POST localhost:8080/ \ |
46 | 125 | -H 'content-type: application/json' \ |
47 | 126 | --data @sample-events/plan.json | jq . |
| 127 | + |
| 128 | +# Explicit v2 |
| 129 | +curl -sX POST localhost:8080/ \ |
| 130 | + -H 'content-type: application/json' \ |
| 131 | + --data @sample-events/plan-v2.json | jq . |
48 | 132 | ``` |
49 | 133 |
|
50 | | -Replace the `PROJECT` placeholder bucket names and `REPLACE_WITH_PLAN_HASH` |
51 | | -with real values from a prior `plan` response. |
| 134 | +Replace `PROJECT`, locator placeholders, and plan hashes with values returned |
| 135 | +by the preceding plan action. A complete action sequence is |
| 136 | +`plan → renderChunk(s) → assemble`. |
0 commit comments