Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions cli/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,12 @@ building anything, an existing deployment manifest and no `sandbox.image` overri
that override only seeds the first `qm up` and must be removed afterwards. Every
ordinary `up` also syncs the layer.

AWS Lambda MicroVM sandboxes use QM's fixed runtime image. They accept skills through the
deployment layer, but do not build `sandbox/Dockerfile` or copy tool executables. `check`,
`sandbox build`, and `infra build-image` reject custom tools and Dockerfiles for that
backend. Configure `sandbox.backend: "sprites"` when the deployment needs a custom sandbox
image.

Auto uses its built-in model classifier unless `qm.config.jsonc` declares one
`securityScreen` proxy with a provider label, HTTPS endpoint, and `shadow` or
`enforce` rollout. The proxy token is routed separately through
Expand Down
4 changes: 2 additions & 2 deletions cli/package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion cli/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@yc-software/qm",
"version": "0.1.6",
"version": "0.1.7",
"license": "MIT",
"description": "Control-plane CLI for portable QM deployments on Docker, Fly, and AWS.",
"type": "module",
Expand Down
2 changes: 1 addition & 1 deletion cli/src/backends/registry.ts
Original file line number Diff line number Diff line change
Expand Up @@ -296,7 +296,7 @@ const aws: HostingProvider = {
publishSandbox: async (ctx, opts) => {
if (ctx.config.sandbox?.backend !== "sprites") {
throw new CliError(
`this AWS deployment runs Lambda MicroVM sandboxes (sandbox.backend is not "sprites"); use \`qm sandbox build\` to validate the layer and \`qm infra build-image\` to publish the runtime — or set "sandbox.backend": "sprites" with "sandbox.app" to host sandboxes in an operator-published layer image`,
`this AWS deployment runs Lambda MicroVM sandboxes (sandbox.backend is not "sprites"), which do not support custom sandbox images; set "sandbox.backend": "sprites" with "sandbox.app" to host sandboxes in an operator-published layer image`,
);
}
if (!opts.dryRun) assertAwsSandboxPinRecordable(ctx.config);
Expand Down
1 change: 1 addition & 0 deletions cli/src/cli.ts
Original file line number Diff line number Diff line change
Expand Up @@ -455,6 +455,7 @@ async function dispatch(argv: string[]): Promise<void> {
if (operation === "delete-image") await deleteAwsMicrovmImage(ctx.config);
else await deleteAwsTaskDefinitions(ctx.config);
} else if (operation === "build-image") {
runChecks(ctx.config, ctx.configDir, ctx.sandboxDir, { report: false });
await buildAwsMicrovmImage(ctx.config, ctx.configPath);
} else {
runChecks(ctx.config, ctx.configDir, ctx.sandboxDir, { report: false });
Expand Down
2 changes: 1 addition & 1 deletion cli/src/commands/check.ts
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ export function runChecks(
opts: { report?: boolean } = {},
): ChecksResult {
const report = opts.report ?? true;
const layer = validateSandboxLayer(sandboxDir);
const layer = validateSandboxLayer(sandboxDir, config);
const { plugins, errors: pluginErrors } = discoverPlugins(configDir, config);
const configErrors: Array<{ clause: string; message: string }> = [];
const configError = (message: string, clause = "config.v1"): void => void configErrors.push({ clause, message });
Expand Down
21 changes: 15 additions & 6 deletions cli/src/commands/init.ts
Original file line number Diff line number Diff line change
Expand Up @@ -57,10 +57,11 @@ the scaffolded \`.gitignore\`.
\`{ "id": ..., "advertise": ..., "install": { "binary": ... } }\`, with the
executable next to it when the binary is not already in the base image.
- \`sandbox/Dockerfile\` is optional and only needed for system packages or
runtimes.
runtimes. AWS Lambda MicroVM sandboxes do not build this file or copy tool
executables; use skills only or configure the sprites backend for custom tools.

The scaffold ships a working example, the \`greet\` skill and \`example-tool\`.
Copy its shape, then replace or delete it.
The scaffold ships a working \`greet\` skill. Targets that support custom sandbox
images also get \`example-tool\`. Copy their shape, then replace or delete them.

## The workflow

Expand Down Expand Up @@ -94,6 +95,13 @@ description: Greet a teammate by name. Use whenever asked to say hello to someon
Run \`example-tool <name>\` to greet someone, e.g. \`example-tool Ada\`.
`;

const AWS_GREET_SKILL = `---
name: greet
description: Greet a teammate by name. Use whenever asked to say hello to someone.
---
Reply with a friendly greeting addressed to the requested person.
`;

const EXAMPLE_TOOL_DESCRIPTOR =
JSON.stringify({ id: "example-tool", advertise: "example-tool", install: { binary: "example-tool" } }, null, 2) +
"\n";
Expand All @@ -116,8 +124,9 @@ function writeIfAbsent(dir: string, segments: string[], content: string, mode?:
ok(`wrote ${rel}`);
}

function scaffoldSandbox(dir: string): void {
writeIfAbsent(dir, ["sandbox", "skills", "greet", "SKILL.md"], GREET_SKILL);
function scaffoldSandbox(dir: string, target: Target): void {
writeIfAbsent(dir, ["sandbox", "skills", "greet", "SKILL.md"], target === "aws" ? AWS_GREET_SKILL : GREET_SKILL);
if (target === "aws") return;
writeIfAbsent(dir, ["sandbox", "tools", "example-tool", "tool.json"], EXAMPLE_TOOL_DESCRIPTOR);
writeIfAbsent(dir, ["sandbox", "tools", "example-tool", "example-tool"], EXAMPLE_TOOL_BIN, 0o755);
}
Expand Down Expand Up @@ -295,7 +304,7 @@ export function runInit(opts: {
const manifests = renderSlackManifests(config);
writeIfAbsent(dir, ["slack-app-manifest.yml"], manifests.bot);
if (usesSlackOidc(config)) writeIfAbsent(dir, ["slack-sso-manifest.yml"], manifests.sso);
scaffoldSandbox(dir);
scaffoldSandbox(dir, target);
for (const file of provider.scaffold.files(config)) writeIfAbsent(dir, file.segments, file.content);

note("");
Expand Down
2 changes: 1 addition & 1 deletion cli/src/commands/sandbox.ts
Original file line number Diff line number Diff line change
Expand Up @@ -249,7 +249,7 @@ function assertPublishPlatform(body: string): void {

function prepare(opts: SandboxBuildOpts): PreparedBuild {
const sandboxDir = resolve(opts.sandboxDir);
const layer = validateSandboxLayer(sandboxDir);
const layer = validateSandboxLayer(sandboxDir, opts.config);
if (layer.errors.length) {
throw new CliError(`sandbox check failed:\n${layer.errors.map((error) => ` - ${error}`).join("\n")}`);
}
Expand Down
16 changes: 14 additions & 2 deletions cli/src/sandbox-layer.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import { existsSync, readFileSync, readdirSync, statSync } from "node:fs";
import { join } from "node:path";
import type { QmConfig } from "./config.ts";
import { JUNK_FILE, deploymentLayerBundle } from "./deployment-layer.ts";
import { errMessage } from "./log.ts";

Expand Down Expand Up @@ -787,7 +788,8 @@ const subdirs = (dir: string): string[] => {
const isCidr = (host: string): boolean =>
/^\d{1,3}(\.\d{1,3}){3}\/\d{1,2}$/.test(host) || /^[0-9A-Fa-f:]+\/\d{1,3}$/.test(host);

export function validateSandboxLayer(sandboxDir: string): SandboxValidation {
export function validateSandboxLayer(sandboxDir: string, config?: QmConfig): SandboxValidation {
const awsMicrovm = config?.target === "aws" && config.sandbox?.backend !== "sprites";
const out: SandboxValidation = {
exists: existsSync(sandboxDir),
hasDockerfile: existsSync(join(sandboxDir, "Dockerfile")),
Expand Down Expand Up @@ -835,7 +837,11 @@ export function validateSandboxLayer(sandboxDir: string): SandboxValidation {
const binaryName = descriptor.install?.binary ?? descriptor.id;
const exePath = join(toolsDir, name, binaryName);
const hasExe = isFile(exePath);
if (!hasExe && !out.hasDockerfile) {
if (awsMicrovm) {
out.errors.push(
`tool "${descriptor.id}" cannot be installed in AWS Lambda MicroVM sandboxes because tool executables and sandbox/Dockerfile are not included; remove it or set "sandbox.backend" to "sprites" and publish a sandbox image`,
);
} else if (!hasExe && !out.hasDockerfile) {
out.errors.push(
`tool "${descriptor.id}" (tools/${name}/) can't get its binary on PATH: ship an executable ` +
`"${binaryName}" in the folder, or add a sandbox/Dockerfile that installs it`,
Expand Down Expand Up @@ -899,5 +905,11 @@ export function validateSandboxLayer(sandboxDir: string): SandboxValidation {
}
}

if (awsMicrovm && out.hasDockerfile) {
out.errors.push(
'sandbox/Dockerfile is not built into AWS Lambda MicroVM sandboxes; remove it or set "sandbox.backend" to "sprites" and publish a sandbox image',
);
}

return out;
}
67 changes: 67 additions & 0 deletions cli/test/check.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,24 @@ const CONFIG: QmConfig = {
sandbox: { app: "acme-sandboxes", image: PINNED_SANDBOX_IMAGE },
};

const AWS_CONFIG: QmConfig = {
...CONFIG,
target: "aws",
publicUrl: "https://acme.example.com",
sandbox: undefined,
env: { core: { AWS_DEPLOY_IMAGE: "acme-microvm" } },
aws: {
accountId: "123456789012",
region: "us-west-2",
cluster: "acme-qm",
deployRoleArn: "arn:aws:iam::123456789012:role/deploy",
secretsPrefix: "acme/qm/",
imageLabel: "latest",
networking: { cloudMapNamespace: "acme.internal" },
services: { core: { ecrRepository: "core", ecsService: "core", cpu: 256, memory: 512 } },
},
};

function deployment(setup: (dir: string) => void, config: Partial<QmConfig> = {}): { dir: string; config: QmConfig } {
const dir = mkdtempSync(join(tmpdir(), "qm-check-"));
setup(dir);
Expand Down Expand Up @@ -134,6 +152,55 @@ test("a tool with no executable BUT a sandbox/Dockerfile passes (Dockerfile inst
}
});

test("AWS Lambda MicroVM sandboxes reject every custom tool installation path", () => {
const d = deployment((dir) => {
writeTool(dir, "shipped-tool", { id: "shipped-tool", install: { binary: "shipped-tool" } });
writeTool(dir, "built-tool", { id: "built-tool", install: { binary: "built-tool" } }, false);
writeFileSync(join(dir, "sandbox", "Dockerfile"), "FROM base\nRUN install-built-tool\n");
}, AWS_CONFIG);
try {
assert.throws(
() => check(d),
(error) => {
assert.match(String(error), /sandbox\/Dockerfile is not built into AWS Lambda MicroVM sandboxes/);
assert.match(String(error), /tool "shipped-tool" cannot be installed/);
assert.match(String(error), /tool "built-tool" cannot be installed/);
assert.match(String(error), /"sandbox\.backend" to "sprites"/);
return true;
},
);
} finally {
rmSync(d.dir, { recursive: true, force: true });
}
});

test("AWS sprites sandboxes keep custom Dockerfile and executable tool support", () => {
const d = deployment(
(dir) => {
writeTool(dir, "shipped-tool", { id: "shipped-tool", install: { binary: "shipped-tool" } });
writeFileSync(join(dir, "sandbox", "Dockerfile"), "FROM base\n");
},
{
...AWS_CONFIG,
sandbox: { backend: "sprites", app: "acme-sandboxes", image: PINNED_SANDBOX_IMAGE },
},
);
try {
assert.doesNotThrow(() => check(d));
} finally {
rmSync(d.dir, { recursive: true, force: true });
}
});

test("AWS Lambda MicroVM sandboxes accept skill-only layers", () => {
const d = deployment((dir) => writeSkill(dir, "greet", "name: greet\ndescription: Greet a teammate."), AWS_CONFIG);
try {
assert.doesNotThrow(() => check(d));
} finally {
rmSync(d.dir, { recursive: true, force: true });
}
});

test("duplicate tool ids are flagged", () => {
const d = deployment((dir) => {
writeTool(dir, "folderA", { id: "same", install: { binary: "same" } });
Expand Down
41 changes: 38 additions & 3 deletions cli/test/cli-dispatch.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -412,7 +412,7 @@ test("docker --only is rejected explicitly instead of silently restarting the fu
}
});

test("sandbox publish directs MicroVM AWS deployments (no sandbox.app) to the image-build path", async () => {
test("sandbox publish rejects custom images for MicroVM AWS deployments", async () => {
const dir = mkdtempSync(join(tmpdir(), "qm-dispatch-"));
const configPath = join(dir, CONFIG_FILENAME);
const raw = JSON.stringify({
Expand All @@ -439,14 +439,49 @@ test("sandbox publish directs MicroVM AWS deployments (no sandbox.app) to the im
try {
const result = await run(["sandbox", "publish", "--dry-run"], dir);
assert.equal(result.exitCode, 1, result.out);
assert.match(result.out, /Lambda MicroVM sandboxes/);
assert.match(result.out, /infra build-image/);
assert.match(result.out, /sandbox\/Dockerfile is not built into AWS Lambda MicroVM sandboxes/);
assert.match(result.out, /"sandbox\.backend" to "sprites"/);
assert.equal(readFileSync(configPath, "utf8"), raw);
} finally {
rmSync(dir, { recursive: true, force: true });
}
});

test("infra build-image rejects unsupported AWS sandbox customization before calling AWS", async () => {
const dir = mkdtempSync(join(tmpdir(), "qm-dispatch-"));
writeFileSync(
join(dir, CONFIG_FILENAME),
JSON.stringify({
contract: 1,
orgId: "acme",
publicUrl: "https://acme.example.com",
target: "aws",
services: ["core"],
env: { core: { AWS_DEPLOY_IMAGE: "acme-microvm-app" } },
aws: {
accountId: "123456789012",
region: "us-west-2",
cluster: "c",
deployRoleArn: "arn:aws:iam::123456789012:role/d",
secretsPrefix: "p/",
imageLabel: "release",
networking: { cloudMapNamespace: "n" },
services: { core: { ecrRepository: "repo", ecsService: "s", cpu: 256, memory: 512 } },
},
}),
);
mkdirSync(join(dir, "sandbox"));
writeFileSync(join(dir, "sandbox", "Dockerfile"), "FROM scratch\n");
try {
const result = await run(["infra", "build-image"], dir);
assert.equal(result.exitCode, 1, result.out);
assert.match(result.out, /sandbox\/Dockerfile is not built into AWS Lambda MicroVM sandboxes/);
assert.doesNotMatch(result.out, /missing required env|AWS account mismatch/);
} finally {
rmSync(dir, { recursive: true, force: true });
}
});

test("sandbox publish on an AWS deployment with sandbox.app dry-runs the operator layer image", async () => {
const dir = mkdtempSync(join(tmpdir(), "qm-dispatch-"));
const configPath = join(dir, CONFIG_FILENAME);
Expand Down
5 changes: 5 additions & 0 deletions cli/test/init.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -244,6 +244,11 @@ test("init --target aws scaffolds the full hosted topology, Terraform, and the o
assert.equal(config.sandbox, undefined);
assert.equal(config.aws?.cluster, "acme-qm");
assert.equal(config.aws?.imageLabel, "latest");
assert.equal(existsSync(join(dir, "sandbox", "tools")), false);
const greetSkill = readFileSync(join(dir, "sandbox", "skills", "greet", "SKILL.md"), "utf8");
assert.doesNotMatch(greetSkill, /example-tool/);
assert.deepEqual(validateSandboxLayer(join(dir, "sandbox")).errors, []);
assert.doesNotThrow(() => quiet(() => runChecks(config, dir, join(dir, "sandbox"), { report: false })));
assert.deepEqual(config.aws?.services.core, {
ecrRepository: "acme-qm-core",
ecsService: "acme-qm-core",
Expand Down
10 changes: 10 additions & 0 deletions cli/test/sandbox-build.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -150,3 +150,13 @@ test("a broken layer (tool with no executable and no Dockerfile) fails before bu
rmSync(sb, { recursive: true, force: true });
}
});

test("sandbox build rejects custom tool images for AWS Lambda MicroVM sandboxes", () => {
const sb = sandboxDir((s) => tool(s, "x", { id: "x", install: { binary: "x" } }));
const config: QmConfig = { ...CONFIG, target: "aws", sandbox: undefined };
try {
assert.throws(() => dryRun({ sandboxDir: sb, config }), /cannot be installed in AWS Lambda MicroVM sandboxes/);
} finally {
rmSync(sb, { recursive: true, force: true });
}
});
2 changes: 2 additions & 0 deletions docs/deploy-directory.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,8 @@ sandbox/

The Dockerfile is optional when every declared binary is present in its tool directory. Skill assets delivered through the deployment-layer API are text in v1; binaries belong in the sandbox image.

AWS Lambda MicroVM sandboxes use QM's fixed runtime image, so they do not build the deployment's Dockerfile or copy its tool executables. The CLI rejects both installation paths when the AWS sandbox backend is omitted or set to `"aws"`. Skill-only layers remain supported; deployments that need a custom sandbox image must select `sandbox.backend: "sprites"`.

## Configuration

The root object requires `contract: 1`, `orgId`, `publicUrl`, `target`, and `services` including `core`. Docker and Fly also require `sandbox.app`. On AWS the sandbox substrate is an explicit choice: omitting the `sandbox` block runs named Lambda MicroVM images; declaring one requires `sandbox.backend` — `"sprites"` boots the operator-published layer image in `sandbox.app`, `"aws"` states the MicroVM default in the file. Unknown contract majors fail closed. `target` is `docker`, `fly`, or `aws`.
Expand Down
Loading