Skip to content

Commit dfd3881

Browse files
committed
feat(gcp-cloud-run): support plan protocol v2
1 parent ae9fbad commit dfd3881

27 files changed

Lines changed: 2031 additions & 243 deletions

.fallowrc.jsonc

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@
44
"packages/producer/src/**/*.test.ts",
55
"packages/aws-lambda/src/**/*.test.ts",
66
"packages/gcp-cloud-run/src/**/*.test.ts",
7+
"packages/gcp-cloud-run/terraform/*.test.ts",
78
"packages/producer/src/regression-harness.ts",
89
"packages/producer/src/regression-harness-distributed.test.ts",
910
"packages/producer/src/regression-harness-lambda-local.ts",
@@ -403,6 +404,12 @@
403404
// extracting a shared cloud abstraction would couple independent packages.
404405
"packages/aws-lambda/src/handler.ts",
405406
"packages/aws-lambda/src/s3Transport.ts",
407+
// The GCP handler deliberately mirrors the AWS protocol lifecycle while
408+
// retaining provider-specific GCS, HTTP, and Cloud Workflows semantics.
409+
// Its tests also mirror the same wire-contract cases; a cross-provider
410+
// test abstraction would hide the adapter boundary being asserted.
411+
"packages/gcp-cloud-run/src/server.ts",
412+
"packages/gcp-cloud-run/src/server.test.ts",
406413
// sourcePatcher.ts: pre-existing internal clones between the inline-style
407414
// and attribute tag-patchers; only the PatchOperation type gained two
408415
// optional fields here, but the line shift makes fallow re-flag them.

bun.lock

Lines changed: 14 additions & 13 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

examples/gcp-cloud-run/README.md

Lines changed: 108 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -1,51 +1,136 @@
11
# Google Cloud Run example
22

3-
End-to-end deployment + smoke for [`@hyperframes/gcp-cloud-run`](../../packages/gcp-cloud-run) — the Cloud Run + Cloud Workflows adapter for HyperFrames distributed rendering.
3+
End-to-end deployment and parity testing for
4+
[`@hyperframes/gcp-cloud-run`](../../packages/gcp-cloud-run), the Cloud Run +
5+
Cloud Workflows adapter for HyperFrames distributed rendering.
46

57
## Layout
68

9+
```text
10+
scripts/smoke.sh Owner-isolated real-GCP deploy, render, parity, cleanup
11+
sample-events/ v1 and v2 handler request examples
712
```
8-
scripts/smoke.sh Real-GCP smoke: build → deploy → render → PSNR → destroy
9-
sample-events/ Example request bodies for the Cloud Run handler
10-
(plan.json, render-chunk.json, assemble.json)
11-
```
1213

13-
The Terraform module and the Cloud Workflows definition that the smoke deploys live with the package, at `packages/gcp-cloud-run/terraform/` (including `workflow.yaml`).
14+
The Terraform module and Cloud Workflows definition live in
15+
`packages/gcp-cloud-run/terraform/`.
16+
17+
## Protocol rollout
18+
19+
The workflow defaults to plan protocol v1 when `PlanProtocol` is absent. V2 is
20+
accepted only when the caller explicitly sends `PlanProtocol: "v2"`.
21+
22+
V1 and v2 use disjoint plan locators:
23+
24+
- v1: `PlanGcsUri`
25+
- v2: `PlanV2ManifestGcsUri` and `PlanV2ArtifactGcsPrefix`
26+
27+
The workflow validates that the plan response matches the selected protocol
28+
before starting chunk fan-out. It never silently falls back from v2 to v1.
29+
Deploy the v2 workflow only with a Cloud Run image whose handler implements
30+
the matching v2 request/response contract. An older v1-only handler will keep
31+
serving default v1 requests, but explicit v2 smoke executions will fail closed.
1432

1533
## Prerequisites
1634

17-
- `gcloud` authenticated, with a project that has **billing enabled**
18-
- `terraform` (≥ 1.5), `docker`, `ffmpeg`, `jq` on PATH
35+
- `gcloud` authenticated to a project with billing enabled
36+
- `terraform` (>= 1.5), `ffmpeg`, `ffprobe`, `jq`, `tar`, and `sha256sum`
37+
- the required project APIs already enabled, plus permission to run Cloud
38+
Build and manage Cloud Run, Workflows, GCS, IAM service accounts,
39+
Monitoring, and Artifact Registry resources
1940

2041
## Run the smoke
2142

43+
V1 remains the safe default:
44+
2245
```bash
23-
# Renders the mp4-h264-sdr fixture through the workflow and PSNR-compares it
24-
# against the in-process baseline, then tears the stack down.
25-
./scripts/smoke.sh --project YOUR_GCP_PROJECT --region us-central1
46+
./scripts/smoke.sh \
47+
--project YOUR_GCP_PROJECT \
48+
--region us-central1
49+
```
2650

27-
# Keep the stack up to poke at it:
28-
./scripts/smoke.sh --project YOUR_GCP_PROJECT --keep-stack
51+
Explicitly run v1/v2 end-to-end parity at one or more chunk sizes:
2952

30-
# Render at several chunk sizes to see the fan-out scaling:
31-
./scripts/smoke.sh --project YOUR_GCP_PROJECT --chunk-sizes 30,15,10
53+
```bash
54+
./scripts/smoke.sh \
55+
--project YOUR_GCP_PROJECT \
56+
--region us-central1 \
57+
--protocols v1,v2 \
58+
--chunk-sizes 30,15,10 \
59+
--owner plan-v2-parity
3260
```
3361

34-
Outputs land in `scripts/gcp-smoke-artifacts/`: `results.json`
35-
(`chunkSize × wallClockMs × psnrAvgDb`), the rendered MP4s, and each
36-
workflow execution's describe output.
62+
For each chunk size, parity requires exact equality of:
63+
64+
- decoded RGBA video frames
65+
- decoded 48 kHz stereo PCM audio
66+
- normalized `ffprobe` stream and duration metadata
67+
68+
The encoded MP4 hash and byte count are recorded but are not the equality
69+
oracle because mux metadata can differ without changing decoded output.
70+
Each render is also PSNR-compared with the checked-in in-process fixture
71+
baseline.
72+
73+
## Isolation and cleanup
74+
75+
Every invocation hashes the owner, project, region, and a fresh invocation
76+
nonce into a unique resource prefix such as `hf-smoke-a1b2c3d4e5`. Reusing an
77+
owner label does not reuse old Terraform state or cloud resources. This prefix
78+
stays within GCP service account naming limits. The smoke:
79+
80+
- never uses the static `hyperframes` prefix
81+
- copies the Terraform module into an owner-scoped work directory and uses an
82+
isolated Terraform data directory and state file
83+
- scopes GCS keys, render outputs, the image package/tag, and the default
84+
Artifact Registry repository to that owner
85+
- deletes only an image it built
86+
- deletes the Artifact Registry repository only when that invocation created it
87+
- refuses to enable project APIs, because APIs are shared project state
88+
- stages the bounded Cloud Build source archive in an owner-scoped bucket,
89+
writes build logs to Cloud Logging, and deletes the staging bucket
90+
91+
Cleanup is on by default. It empties and destroys the owner-scoped bucket and
92+
stack, deletes owned image/repository/build-staging resources, then verifies
93+
the Cloud Run service, workflow, buckets, both service accounts, image, and any
94+
test-created repository are absent. Cleanup fails on API or authentication
95+
errors rather than interpreting them as successful deletion. GCP retains the
96+
Cloud Build execution record and Cloud Logging audit entries as project-level
97+
operational history; the smoke test does not attempt to erase audit records.
98+
99+
`--keep-stack` deliberately retains the stack, image, and repository and
100+
prints the exact isolated state directory and Terraform cleanup commands.
101+
Never use it for unattended CI.
102+
103+
Evidence lands under:
104+
105+
```text
106+
scripts/gcp-smoke-artifacts/<owner-hash>/
107+
results.json
108+
parity.json
109+
renders/
110+
terraform/
111+
terraform-data/
112+
```
113+
114+
Use `--image` to test a caller-owned existing image. That image is never
115+
deleted. `--skip-build` requires `--image`; new invocations never inherit an
116+
old invocation's state or image implicitly.
37117

38118
## Test the handler locally
39119

40-
The sample events exercise the same body shape Cloud Workflows sends. With the
41-
container running locally (`PORT=8080`) and credentials that can reach a GCS
42-
bucket, you can drive a single action:
120+
The sample events mirror the request bodies sent by Cloud Workflows:
43121

44122
```bash
123+
# V1
45124
curl -sX POST localhost:8080/ \
46125
-H 'content-type: application/json' \
47126
--data @sample-events/plan.json | jq .
127+
128+
# Explicit v2
129+
curl -sX POST localhost:8080/ \
130+
-H 'content-type: application/json' \
131+
--data @sample-events/plan-v2.json | jq .
48132
```
49133

50-
Replace the `PROJECT` placeholder bucket names and `REPLACE_WITH_PLAN_HASH`
51-
with real values from a prior `plan` response.
134+
Replace `PROJECT`, locator placeholders, and plan hashes with values returned
135+
by the preceding plan action. A complete action sequence is
136+
`plan → renderChunk(s) → assemble`.
Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
{
2+
"Action": "assemble",
3+
"PlanProtocol": "v2",
4+
"PlanV2ManifestGcsUri": "gs://hyperframes-render-PROJECT/renders/hf-render-demo/v2/manifest.json",
5+
"PlanV2ArtifactGcsPrefix": "gs://hyperframes-render-PROJECT/renders/hf-render-demo/v2/artifacts/sha256",
6+
"PlanHash": "REPLACE_WITH_PLAN_HASH",
7+
"ChunkGcsUris": [
8+
"gs://hyperframes-render-PROJECT/renders/hf-render-demo/chunks/0000.mp4",
9+
"gs://hyperframes-render-PROJECT/renders/hf-render-demo/chunks/0001.mp4"
10+
],
11+
"AudioGcsUri": null,
12+
"OutputGcsUri": "gs://hyperframes-render-PROJECT/renders/hf-render-demo/output.mp4",
13+
"Format": "mp4"
14+
}

examples/gcp-cloud-run/sample-events/assemble.json

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
{
22
"Action": "assemble",
3+
"PlanProtocol": "v1",
34
"PlanGcsUri": "gs://hyperframes-render-PROJECT/renders/hf-render-demo/plan.tar.gz",
45
"ChunkGcsUris": [
56
"gs://hyperframes-render-PROJECT/renders/hf-render-demo/chunks/0000.mp4",
Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
{
2+
"Action": "plan",
3+
"PlanProtocol": "v2",
4+
"ProjectGcsUri": "gs://hyperframes-render-PROJECT/sites/abc123/project.tar.gz",
5+
"PlanOutputGcsPrefix": "gs://hyperframes-render-PROJECT/renders/hf-render-demo/",
6+
"Config": { "fps": 30, "width": 1920, "height": 1080, "format": "mp4" }
7+
}

examples/gcp-cloud-run/sample-events/plan.json

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
{
22
"Action": "plan",
3+
"PlanProtocol": "v1",
34
"ProjectGcsUri": "gs://hyperframes-render-PROJECT/sites/abc123/project.tar.gz",
45
"PlanOutputGcsPrefix": "gs://hyperframes-render-PROJECT/renders/hf-render-demo/",
56
"Config": { "fps": 30, "width": 1920, "height": 1080, "format": "mp4" }
Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
{
2+
"Action": "renderChunk",
3+
"PlanProtocol": "v2",
4+
"PlanV2ManifestGcsUri": "gs://hyperframes-render-PROJECT/renders/hf-render-demo/v2/manifest.json",
5+
"PlanV2ArtifactGcsPrefix": "gs://hyperframes-render-PROJECT/renders/hf-render-demo/v2/artifacts/sha256",
6+
"PlanHash": "REPLACE_WITH_PLAN_HASH",
7+
"ChunkIndex": 0,
8+
"ChunkOutputGcsPrefix": "gs://hyperframes-render-PROJECT/renders/hf-render-demo/",
9+
"Format": "mp4"
10+
}

examples/gcp-cloud-run/sample-events/render-chunk.json

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
{
22
"Action": "renderChunk",
3+
"PlanProtocol": "v1",
34
"PlanGcsUri": "gs://hyperframes-render-PROJECT/renders/hf-render-demo/plan.tar.gz",
45
"PlanHash": "REPLACE_WITH_PLAN_HASH",
56
"ChunkIndex": 0,

0 commit comments

Comments
 (0)