Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions docs/packages/aws-lambda.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,7 @@ const site = await deploySite({

const handle = await renderToLambda({
siteHandle: site,
planProtocol: "v2",
bucketName: site.bucketName,
stateMachineArn: "arn:aws:states:us-east-1:123456789012:stateMachine:hyperframes-render",
config: {
Expand All @@ -87,6 +88,11 @@ const progress = await getRenderProgress({ executionArn: handle.executionArn });
console.log(progress.status, progress.overallProgress, progress.costs.displayCost);
```

Plan v2 is recommended for new integrations because workers fetch
manifest-selected content-addressed artifacts. For backwards compatibility,
omitting `planProtocol` still selects v1; existing callers do not change
behavior until they opt in.

`renderToLambda()` validates the distributed render config before starting the Step Functions execution, so invalid dimensions, formats, chunk sizes, or payload sizes fail synchronously.

## Using the CDK Construct
Expand Down
6 changes: 6 additions & 0 deletions docs/packages/gcp-cloud-run.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,7 @@ import { getRenderProgress, renderToCloudRun } from "@hyperframes/gcp-cloud-run/

const handle = await renderToCloudRun({
projectDir: "./my-composition",
planProtocol: "v2",
config: { fps: 30, width: 1920, height: 1080, format: "mp4" },
bucketName: "hyperframes-render-my-project",
projectId: "my-project",
Expand All @@ -96,6 +97,11 @@ while (progress.status === "running") {
console.log(progress.status, progress.outputFile, progress.costs.displayCost);
```

Plan v2 is recommended for new integrations because workers fetch
manifest-selected content-addressed artifacts. For backwards compatibility,
omitting `planProtocol` still selects v1; existing callers do not change
behavior until they opt in.

Pass `projectDir` for one-shot uploads, or call `deploySite()` separately and reuse the returned site handle across many renders.

## Related Guides
Expand Down
5 changes: 3 additions & 2 deletions examples/aws-lambda/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ aws stepfunctions start-execution \
"ProjectS3Uri": "s3://${RENDER_BUCKET}/projects/my-project.tar.gz",
"PlanOutputS3Prefix": "s3://${RENDER_BUCKET}/renders/$(date +%s)/",
"OutputS3Uri": "s3://${RENDER_BUCKET}/output.mp4",
"PlanProtocol": "v1",
"PlanProtocol": "v2",
"Config": {
"fps": 30,
"width": 1920,
Expand All @@ -92,7 +92,8 @@ EOF

The Step Functions execution kicks off Plan, fans out RenderChunk via
the Map state, and finally Assemble. Final mp4 lands at `OutputS3Uri`.
`PlanProtocol` may be `"v1"` or `"v2"`; absent defaults to v1. V2 uses
Plan v2 is recommended for new integrations. `PlanProtocol` may be `"v1"` or
`"v2"`; absent still defaults to v1 for backwards compatibility. V2 uses
separate manifest and content-addressed artifact locators throughout the
workflow and never places a v2 object in `PlanS3Uri`.

Expand Down
5 changes: 3 additions & 2 deletions packages/aws-lambda/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,8 +47,9 @@ inside Step Functions' history budget (under 200 bytes per chunk).

### Plan transport selection

`renderToLambda` defaults to the existing monolithic v1 plan transport.
Plan v2 is an explicit whole-render opt-in:
Plan v2 is recommended for new integrations. `renderToLambda` still defaults
an omitted `planProtocol` to the existing monolithic v1 transport for
backwards compatibility, so select v2 explicitly:

```ts
await renderToLambda({
Expand Down
24 changes: 20 additions & 4 deletions packages/aws-lambda/src/events.ts
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,11 @@ interface PlanEventBase {
Config: SerializableDistributedRenderConfig;
}

/** Legacy/default plan transport. Absence is deliberately interpreted as v1. */
/**
* Legacy/default plan transport. Absence is deliberately interpreted as v1.
*
* @deprecated Use {@link PlanV2Event} for new integrations.
*/
export interface PlanV1Event extends PlanEventBase {
PlanProtocol?: "v1";
}
Expand Down Expand Up @@ -85,7 +89,11 @@ interface RenderChunkEventBase {
Format: DistributedFormat;
}

/** Legacy/default chunk event. */
/**
* Legacy/default chunk event.
*
* @deprecated Use {@link RenderChunkV2Event} for new integrations.
*/
export interface RenderChunkV1Event extends RenderChunkEventBase {
PlanProtocol?: "v1";
/** S3 URI of the v1 plan tar produced by a PlanEvent invocation. */
Expand Down Expand Up @@ -127,7 +135,11 @@ interface AssembleEventBase {
Cfr?: boolean;
}

/** Legacy/default assemble event. */
/**
* Legacy/default assemble event.
*
* @deprecated Use {@link AssembleV2Event} for new integrations.
*/
export interface AssembleV1Event extends AssembleEventBase {
PlanProtocol?: "v1";
/** S3 URI of the v1 plan tar produced by a PlanEvent invocation. */
Expand Down Expand Up @@ -163,7 +175,11 @@ interface PlanLambdaResultBase {
DurationMs: number;
}

/** Existing v1 result. Kept unchanged for wire compatibility. */
/**
* Existing v1 result. Kept unchanged for wire compatibility.
*
* @deprecated New integrations should consume {@link PlanV2LambdaResult}.
*/
export interface PlanV1LambdaResult extends PlanLambdaResultBase {
PlanS3Uri: string;
}
Expand Down
4 changes: 2 additions & 2 deletions packages/aws-lambda/src/handler.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ import {
type PlanResult,
type PlanV2ArtifactPublisher,
type PlanV2Manifest,
publishPlanV2FromV1,
publishPlanV2FromExecutionPlan,
} from "@hyperframes/producer/distributed";
import { recomputePlanHashFromPlanDir } from "../../producer/src/services/render/stages/freezePlan.js";
import type { AssembleEvent, LambdaEvent, PlanEvent, RenderChunkEvent } from "./events.js";
Expand Down Expand Up @@ -555,7 +555,7 @@ describe("handler dispatch", () => {
): Promise<PlanV2Manifest> => {
const v1Dir = join(tmpRoot, `v1-${Date.now()}`);
makeMinimalV1PlanDir(v1Dir, true);
const manifest = await publishPlanV2FromV1(v1Dir, publisher);
const manifest = await publishPlanV2FromExecutionPlan(v1Dir, publisher);
expect(options.stagingParentDir).toBe(dirname(projectDir));
expect(existsSync(join(dirname(projectDir), "plan-v2"))).toBe(false);
return manifest;
Expand Down
19 changes: 17 additions & 2 deletions packages/aws-lambda/src/sdk/renderToLambda.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,15 @@ describe("renderToLambda", () => {
expect(handle.projectS3Uri).toMatch(
/^s3:\/\/test-bucket\/sites\/[0-9a-f]{16}\/project\.tar\.gz$/,
);
expect(Object.keys(handle)).toEqual([
"renderId",
"executionArn",
"bucketName",
"stateMachineArn",
"outputS3Uri",
"projectS3Uri",
"startedAt",
]);

expect(sfn.starts).toHaveLength(1);
const start = sfn.starts[0]!;
Expand All @@ -96,7 +105,7 @@ describe("renderToLambda", () => {
it("opts the complete execution into plan protocol v2 explicitly", async () => {
const sfn = new FakeSFN();
const s3 = new FakeS3();
await renderToLambda({
const handle = await renderToLambda({
projectDir,
bucketName: "test-bucket",
stateMachineArn: "arn:aws:states:us-east-1:1234:stateMachine:hf",
Expand All @@ -107,7 +116,13 @@ describe("renderToLambda", () => {
s3: asS3Client(s3),
});

expect(sfn.starts[0]?.input).toMatchObject({ PlanProtocol: "v2" });
expect(sfn.starts[0]?.input).toEqual({
ProjectS3Uri: handle.projectS3Uri,
PlanOutputS3Prefix: "s3://test-bucket/renders/smoke-v2/",
OutputS3Uri: "s3://test-bucket/renders/smoke-v2/output.mp4",
Config: baseConfig,
PlanProtocol: "v2",
});
});

it("derives the file extension from config.format", async () => {
Expand Down
2 changes: 1 addition & 1 deletion packages/aws-lambda/src/sdk/renderToLambda.ts
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ export interface RenderToLambdaOptions {
config: SerializableDistributedRenderConfig;
/**
* Distributed plan transport. Defaults to `"v1"` for backwards
* compatibility; v2 is always an explicit whole-render opt-in.
* compatibility. New integrations should explicitly select `"v2"`.
*/
planProtocol?: LambdaPlanProtocol;
/** S3 bucket from the SAM stack output (`RenderBucketName`). */
Expand Down
21 changes: 19 additions & 2 deletions packages/gcp-cloud-run/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,8 +30,8 @@ GCS bucket ←→ Cloud Run service (plan / renderChunk / assemble)
Cloud Workflows (Plan → parallel RenderChunk → Assemble)
```

- **Plan** downloads the project tarball, runs `plan()`, uploads the planDir
tarball (+ audio) to GCS, and returns the chunk count.
- **Plan** downloads the project tarball and publishes either a legacy v1
planDir tarball or a v2 manifest plus content-addressed artifacts.
- **RenderChunk** runs in a parallel `for` loop in the workflow, fanned out
up to the plan's chunk count. Each invocation renders one chunk and uploads
it.
Expand All @@ -43,6 +43,23 @@ The workflow accumulates each step's small result body and returns
`{ Plan, Chunks, Assemble }` so `getRenderProgress` can read frame totals and
per-step durations on success.

### Plan transport selection

Plan v2 is recommended for new integrations. `renderToCloudRun` still
interprets an omitted `planProtocol` as `"v1"` for backwards compatibility,
so new callers should select v2 explicitly:

```ts
await renderToCloudRun({
// ...project, bucket, workflow, service, and config...
planProtocol: "v2",
});
```

V2 uses separate manifest and content-addressed artifact locators throughout
the workflow. Unknown protocols and integrity failures fail closed; a render
never mixes v1 and v2 artifacts.

## Chrome runtime

Unlike the Lambda adapter — which fights a 250 MB ZIP ceiling and
Expand Down
24 changes: 20 additions & 4 deletions packages/gcp-cloud-run/src/events.ts
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,11 @@ interface PlanEventBase {
Config: SerializableDistributedRenderConfig;
}

/** Legacy/default plan transport. Absence is deliberately interpreted as v1. */
/**
* Legacy/default plan transport. Absence is deliberately interpreted as v1.
*
* @deprecated Use {@link PlanV2Event} for new integrations.
*/
export interface PlanV1Event extends PlanEventBase {
PlanProtocol?: "v1";
}
Expand Down Expand Up @@ -91,7 +95,11 @@ interface RenderChunkEventBase {
Format: DistributedFormat;
}

/** Legacy/default chunk event. */
/**
* Legacy/default chunk event.
*
* @deprecated Use {@link RenderChunkV2Event} for new integrations.
*/
export interface RenderChunkV1Event extends RenderChunkEventBase {
PlanProtocol?: "v1";
/** GCS URI of the v1 plan tar produced by a PlanEvent invocation. */
Expand Down Expand Up @@ -134,7 +142,11 @@ interface AssembleEventBase {
Cfr?: boolean;
}

/** Legacy/default assemble event. */
/**
* Legacy/default assemble event.
*
* @deprecated Use {@link AssembleV2Event} for new integrations.
*/
export interface AssembleV1Event extends AssembleEventBase {
PlanProtocol?: "v1";
/** GCS URI of the v1 plan tar produced by a PlanEvent invocation. */
Expand Down Expand Up @@ -177,7 +189,11 @@ interface PlanResultBodyBase {
DurationMs: number;
}

/** Existing v1 result. Kept unchanged for wire compatibility. */
/**
* Existing v1 result. Kept unchanged for wire compatibility.
*
* @deprecated New integrations should consume {@link PlanV2ResultBody}.
*/
export interface PlanV1ResultBody extends PlanResultBodyBase {
PlanGcsUri: string;
PlanProtocol?: never;
Expand Down
35 changes: 27 additions & 8 deletions packages/gcp-cloud-run/src/sdk/renderToCloudRun.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -66,19 +66,30 @@ describe("renderToCloudRun", () => {
);
expect(handle.outputGcsUri).toBe("gs://b/renders/hf-render-fixed/output.mp4");
expect(handle.projectGcsUri).toBe("gs://b/sites/abc/project.tar.gz");
expect(Object.keys(handle)).toEqual([
"renderId",
"executionName",
"bucketName",
"workflowId",
"outputGcsUri",
"projectGcsUri",
"startedAt",
]);
});

it("builds the workflow argument the YAML expects", async () => {
const fake = new FakeExecutions();
await renderToCloudRun(opts(fake));
const arg = JSON.parse(fake.lastArgument ?? "{}");
expect(arg.RenderId).toBe("hf-render-fixed");
expect(arg.ProjectGcsUri).toBe("gs://b/sites/abc/project.tar.gz");
expect(arg.PlanOutputGcsPrefix).toBe("gs://b/renders/hf-render-fixed/");
expect(arg.OutputGcsUri).toBe("gs://b/renders/hf-render-fixed/output.mp4");
expect(arg.ServiceUrl).toBe("https://render-abc.run.app");
expect(arg.Config.format).toBe("mp4");
expect(arg.PlanProtocol).toBe("v1");
expect(arg).toEqual({
RenderId: "hf-render-fixed",
ProjectGcsUri: "gs://b/sites/abc/project.tar.gz",
PlanOutputGcsPrefix: "gs://b/renders/hf-render-fixed/",
OutputGcsUri: "gs://b/renders/hf-render-fixed/output.mp4",
ServiceUrl: "https://render-abc.run.app",
Config: config,
PlanProtocol: "v1",
});
expect(fake.lastParent).toBe(
"projects/proj/locations/us-central1/workflows/hyperframes-render",
);
Expand All @@ -88,7 +99,15 @@ describe("renderToCloudRun", () => {
const fake = new FakeExecutions();
await renderToCloudRun({ ...opts(fake), planProtocol: "v2" });
const arg = JSON.parse(fake.lastArgument ?? "{}");
expect(arg.PlanProtocol).toBe("v2");
expect(arg).toEqual({
RenderId: "hf-render-fixed",
ProjectGcsUri: "gs://b/sites/abc/project.tar.gz",
PlanOutputGcsPrefix: "gs://b/renders/hf-render-fixed/",
OutputGcsUri: "gs://b/renders/hf-render-fixed/output.mp4",
ServiceUrl: "https://render-abc.run.app",
Config: config,
PlanProtocol: "v2",
});
});

it("derives the output extension from the format", async () => {
Expand Down
2 changes: 1 addition & 1 deletion packages/gcp-cloud-run/src/sdk/renderToCloudRun.ts
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ export interface RenderToCloudRunOptions {
config: SerializableDistributedRenderConfig;
/**
* Distributed plan transport. Defaults to `"v1"` for backwards
* compatibility; v2 is always an explicit whole-render opt-in.
* compatibility. New integrations should explicitly select `"v2"`.
*/
planProtocol?: CloudRunPlanProtocol;
/** GCS bucket from the Terraform output (`render_bucket_name`). */
Expand Down
4 changes: 2 additions & 2 deletions packages/gcp-cloud-run/src/server.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ import {
type PlanResult,
type PlanV2ArtifactPublisher,
type PlanV2Manifest,
publishPlanV2FromV1,
publishPlanV2FromExecutionPlan,
} from "@hyperframes/producer/distributed";
import { recomputePlanHashFromPlanDir } from "../../producer/src/services/render/stages/freezePlan.js";
import { asStorage, FakeGcs } from "./__fixtures__/fakeGcs.js";
Expand Down Expand Up @@ -263,7 +263,7 @@ describe("dispatch", () => {
): Promise<PlanV2Manifest> => {
const v1Dir = join(root, "v1");
makeMinimalV1PlanDir(v1Dir, true);
const manifest = await publishPlanV2FromV1(v1Dir, publisher);
const manifest = await publishPlanV2FromExecutionPlan(v1Dir, publisher);
expect(options.stagingParentDir).toBe(dirname(projectDir));
expect(existsSync(join(dirname(projectDir), "plan-v2"))).toBe(false);
return manifest;
Expand Down
Loading
Loading