Skip to content

docker target has no usable sandbox backend — and SANDBOX_BACKEND=local is nearly there #278

Description

@korjavin

Context. I self-host qm on a single VPS with --target docker (portal/auth/core/web-ui/admin as containers, Traefik in front). Chat works out of the box, but the moment an agent needs a filesystem (clone, shell), sandbox provisioning fails and the agent reports its machine is unreachable.

Why. On the docker target every SANDBOX_BACKEND value is a dead end for a self-hoster:

  • sprites and aws are hosted backends that need credentials/infra a single-box deploy doesn't have.
  • local is dev-only by design: config validation rejects it for this target, and src/sandbox/local-sandbox.ts assumes core runs on a docker host — it shells out to docker (core's image has no client and no socket), publishes sandbox agent ports on 127.0.0.1 and dials 127.0.0.1, which is core's own loopback once core is a container. The default LOCAL_SANDBOX_IMAGE (qm-sandbox-local:latest) also has no published counterpart — sandbox-base on ghcr carries no exec daemon (its CMD is a sleep loop), and npm run sandbox:local:build lives in the repo, not the npm package.

We have this working in production with deploy-side patches, so the gap is narrow. What it took (reference implementation, ansible/roles/qm_stack):

  1. Core's docker run gains /var/run/docker.sock, a static docker client, and --group-add <socket gid>.
  2. local-sandbox.ts publishes and dials the docker0 gateway instead of loopback — a host address every bridge network reaches (and one a cloud firewall keeps private).
  3. The sandbox image is built on the host: pinned sandbox-base + aws/microvm-agent/agent.mjs (which the npm package already ships as a template) — i.e. exactly local/Dockerfile.

What native support could look like:

  • cli/src/config.ts: accept sandbox.backend: "local" for --target docker; derive LOCAL_SANDBOX_IMAGE from sandbox.image.
  • cli/src/backends/docker.ts runArgs(): when the backend is local, mount the socket + client (or bake a client into the core image) and add --group-add.
  • src/sandbox/local-sandbox.ts: a configurable bind/dial host. This piece is tiny and backwards-compatible — reference patch below, running in production on our box:
--- a/src/config.ts
+++ b/src/config.ts
@@ localSandboxEnv()
     ...(env.LOCAL_SANDBOX_IMAGE ? { image: env.LOCAL_SANDBOX_IMAGE } : {}),
     ...(env.LOCAL_SANDBOX_DOCKER_BIN ? { dockerBin: env.LOCAL_SANDBOX_DOCKER_BIN } : {}),
+    ...(env.LOCAL_SANDBOX_BIND_HOST ? { bindHost: env.LOCAL_SANDBOX_BIND_HOST } : {}),
+    ...(env.LOCAL_SANDBOX_DIAL_HOST ? { dialHost: env.LOCAL_SANDBOX_DIAL_HOST } : {}),

--- a/src/sandbox/local-sandbox.ts
+++ b/src/sandbox/local-sandbox.ts
@@ export interface LocalSandboxOptions {
   image?: string;
   dockerBin?: string;
+  /** Host address sandbox agent ports are published on (default 127.0.0.1). */
+  bindHost?: string;
+  /** Address core dials them on (default: bindHost). */
+  dialHost?: string;
@@ export function createLocalSandbox(
   const image = opts.image ?? DEFAULT_LOCAL_SANDBOX_IMAGE;
+  const bindHost = opts.bindHost ?? "127.0.0.1";
+  const dialHost = opts.dialHost ?? bindHost;
@@ runContainer()
-      `127.0.0.1:0:${AGENT_PORT}`,
+      `${bindHost}:0:${AGENT_PORT}`,
@@ daemon()
-    const res = await fetchImpl(`http://127.0.0.1:${port}${path}`, {
+    const res = await fetchImpl(`http://${dialHost}:${port}${path}`, {
  • Publish a sandbox-local image next to sandbox-base (or document the one-layer build).

Per CONTRIBUTING I'm not opening a code PR — the patch above is just reference, validated on our deployment. I realize the socket mount is a security-posture decision (it's root-equivalent on the host), so the design is yours; happy to test a branch on a real single-box deploy.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions