Skip to content

PLAN.md

Latest commit

 

History

History
216 lines (166 loc) · 18.3 KB

PLAN.md

File metadata and controls

216 lines (166 loc) · 18.3 KB
Raw

forq

Fork the queue. Burn down open-source issue & PR backlogs with cloud VMs and coding agents — powered by boxd live-fork.

1. Thesis

Open-source repos and large orgs drown in open issues and PRs. forq turns that backlog into a queue you burn down:

  1. Install the forq GitHub App, land in the dashboard.
  2. See every repo and its open issues + PRs.
  3. Per repo, configure a warm golden environment (your app, spun up, kept in sync with main).
  4. Manually or automatically trigger actions on issues/PRs: spin a real preview environment, triage with a cheap fast model, implement the change with a coding agent, and open a PR — all with live status/logs and a public URL.

The unlimited-compute version (the hackathon headline): select a repo's entire open-issue queue and implement all of them in parallel. Each issue is solved by forq's speculative mode — a forking engine that live-forks the running environment into many candidate fixes at once and lets the test suite collapse the tree onto the green, minimal patch. Point it at a 200-issue repo Friday night, wake up to a wall of green PRs.

2. Why boxd is the unlock

  • Live-fork (~113ms, memory+disk CoW). Warm preview envs for free, and the substrate for Speculative's speculative branching (fork the running machine, not a cold replay).
  • Proven golden→fork→deploy→Claude primitives already exist as the boxd-setup-* skills. forq reuses them rather than rebuilding a fork/PR pipeline.
  • TypeScript SDK (@boxd-sh/sdk) gives forq's backend full VM control: create, fork, exec, suspend, destroy, writeFile/readFile, createProxy, token.create, waitUntilReady.

3. Architecture

                         ┌──────────────────────────────────────────────┐
   GitHub (App webhooks)  │              forq CONTROL PLANE               │
   issues / PRs / push ──▶│                                              │
                          │  ┌────────────┐      ┌─────────────────────┐ │
   Dashboard (browser) ──▶│  │  Web app    │◀───▶ │  Backend / Orchestr. │ │
   queue, logs, config    │  │(React+Vite) │ SSE  │  - GitHub App         │ │
                          │  └────────────┘      │  - action engine      │ │
                          │                       │  - golden manager     │ │
                          │   state store         │  - triage (Nebius)    │ │
                          │   (Postgres)          │  - coding-agent iface │ │
                          └───────────────────────┼──────────┬───────────┘ │
                                                  │ @boxd-sh/sdk            │
                                       ┌──────────▼──────────┐              │
                                       │   boxd COMPUTE PLANE │              │
                                       │  golden (per repo)   │              │
                                       │  ├─ fork: preview    │              │
                                       │  ├─ fork: implement  │              │
                                       │  └─ fork×N: Speculative  │              │
                                       └──────────────────────┘
   Repo plane: user's GitHub repo + versioned `forq.yml` (config, funding, triage, deploy semantics)

Three planes:

  • Control plane — the forq web app + backend (GitHub App, orchestrator, state). This is what we build.
  • Compute plane — boxd goldens and forks, driven through @boxd-sh/sdk.
  • Repo plane — the user's GitHub repos plus a versioned forq.yml that makes everything self-serve.

4. Components

Component Responsibility
Web app (dashboard) Next.js + Tailwind v4 with opencode-derived theme. Repo list → per-repo issues/PRs queue → multi-select + trigger actions → live run logs/status → golden config UI → settings.
Backend / orchestrator TypeScript service. Receives + verifies GitHub App webhooks, runs the action engine, drives boxd via the SDK, runs triage, streams run state to the dashboard (SSE), persists state.
Golden manager Per repo: provisions the golden (install/build/start, chrome-devtools MCP, gh auth, proxy port), keeps it synced to main (reuses golden-sync.sh + deploy.sh, triggered by forq's push webhook).
Action engine The four actions (§5), each a job with a lifecycle state machine.
Coding-agent interface Agent-agnostic abstraction (modeled on opencode's provider union). v1 impl: claude-code (claude -p --output-format stream-json inside the fork).
Fix-strategy interface single-shot (default, = fix-handler.sh) or speculative (Speculative, §6). Same interface; the dashboard picks.
Triage agent Nebius-hosted fast/cheap model (NVIDIA Nemotron-class, OpenAI-compatible) doing agentic classification before any heavy run (§7).
Oracle Tests as objective judge. Synthesizes a failing test for prose-only issues; uses the PR's CI for PRs (§6).
Funding/billing Reads forq.yml; enforces owner-funded vs contributor-BYO-key per contributor; injects keys into forks per-run only (§8).
State store Postgres (tenants, installations, repos, goldens, runs, events, contributor trust).

5. The four actions

All actions operate on a selected issue/PR (or a batch), manual or auto-triggered, with live status/logs in the dashboard and optional GitHub comment status.

  1. Preview environment — fork the golden, sync to a ref, return the public *.boxd.sh URL.
    • Issue → the golden's tracked branch or a branch chosen from a dropdown.
    • PR → the PR's head ref.
    • Reuses preview-handler.sh + lib.sh (fork_with_retry → wedge_probe → sync_fork_to_branch → poll_url).
  2. Preview environment + fix (implement) — fork the golden, run the coding agent to implement, then push a branch + open a PR (issue) or push to the PR's branch (PR).
    • Reuses fix-handler.sh (prompt render → upload → claude -p → push/PR).
    • This is where the fix-strategy plugs in: single-shot by default, Speculative for the speculative mode.
  3. Triage — run the Nebius fast model to classify intent/actionability/scope and return findings + a recommended action. Gates action 2 to control cost.
  4. Basic GitHub actions — close / comment / label issues & PRs via the GitHub API.

Run lifecycle states (surfaced in the dashboard and as the terminal outcome): queued → triaging → provisioning → implementing → testing → {pr_opened | needs_human | aborted | failed}. needs_human is the graceful-degradation state: the agent escalates with its best attempt and a question when the oracle isn't satisfied within budget or the task is ambiguous.

6. Speculative — the speculative fix strategy (the unlimited-compute showpiece)

The default single-shot strategy runs one coding-agent pass and tests it. speculative (Speculative) turns "implement this issue" into a best-first tree search over live-forked machine states, with tests as the fitness function.

  1. Oracle setup. If the issue ships a failing test/repro, use it. Otherwise the agent first writes a failing test that captures the issue (reproduce → red), commits it; that test ships in the final PR as proof. For PRs, the PR's own CI is the oracle (no synthesis).
  2. Genesis = the warm fork. The per-issue preview fork (deps installed, server up, test runner primed) is the search root.
  3. Branch. A Brancher LLM call proposes K diverse candidate fixes (distinct hypotheses, not K rewordings). The orchestrator forks the genesis K ways (~113ms each, CoW, same worker).
  4. Evaluate. Each child applies its candidate, runs the target test then the full suite, reports {target_pass, suite_passed/total, diff}. On fork-resume the in-VM agent catches signal 42 and reads its branch assignment with no orchestrator round-trip.
  5. Search. The Conductor scores children, prunes red subtrees, and expands the most promising frontier nodes (red branches can re-diagnose and re-branch → a real tree, not a one-shot fan-out). Unlimited compute = expand everything; finite = best-first focuses it. This is MCTS with expand = literal VM fork and simulate = run the tests.
  6. Collapse. First fully-green node (target green + no regressions) wins, or keep searching for a smaller-diff green within budget. The winning fork's disk diff is the PR; the winning VM stays live (you can open its preview URL and see the fix running).
  7. Visualize. A live tree: nodes are VMs colored by status (amber exploring / red failing / green passing / gray pruned), edges labeled with the hypothesis, header counter ("1 bug · 312 universes · 14 green · best 3 lines · 41s"). The explode-then-collapse is the demo.

Scale / distributed-systems note. Forks of a golden are worker-local (shared overlay) — deep exploration is cheap and local. To spread a search across the cluster, re-seed: snapshot a promising node to a template and cold-restore it as a fresh root on another worker. Backpressure via per-tenant concurrency caps, boxd quota, aggressive teardown of red/idle forks, and the triage cost gate.

7. Triage (Nebius)

A cheap fast model is the cost governor in front of expensive coding agents — on-theme for "unlimited compute, spent wisely."

  • Model: Nebius AI Studio, OpenAI-compatible endpoint, a fast Nemotron-class model (configurable via forq.yml).
  • Job: agentic classification of an issue/PR — actionable? in scope? well-specified enough to attempt? duplicate/stale? Returns a structured verdict {act: bool, confidence, reason, suggested_action}.
  • Gate: if act (≥ threshold) → hand off to the heavy coding agent; else → comment the reasoning and stop (or route to needs_human).

8. Funding & bring-your-own-key

Inference cost is governed by versioned config in the user's repo, so it's self-serve and auditable.

  • New contributor → BYO key. When an issue/PR is from someone not yet trusted, forq requires the contributor to bring their own inference API key (entered in the dashboard or via a one-time secure link from a forq bot comment). The key is injected into the fork's env for that run only (boxd tmpfs/secret, never persisted to disk or our DB).
  • After first contribution → owner's call. Once a contributor has landed a change, the owner can flip them to owner-funded (a click in the dashboard, written back to forq.yml / the trust list).
  • Owner-funded uses the owner's configured key for all triggered runs.

9. Agent-agnostic & strategy-agnostic design

Modeled on opencode's provider abstraction (tagged union + capabilities + enabled.via discriminator, dynamic loading).

interface CodingAgent {            // claude-code (v1), later: opencode, aider, ...
  run(ctx: { fork: Box; prompt: string; repo: RepoCtx; signal: AbortSignal })
    : AsyncIterable<AgentEvent>;   // streamed to the dashboard
}
interface FixStrategy {            // single-shot (v1) | speculative (Speculative)
  implement(ctx: ImplementCtx): Promise<FixResult>; // { diff, prUrl, state }
}

Start with claude-code + single-shot; add speculative (Speculative) and more agents without touching the action engine.

10. boxd reuse map

forq capability Verdict How
Configure golden per repo Reuse recipe, reimplement driver c.box.create + box.exec (install/clone/start) + box.setProxyPort + box.waitUntilReady; lift the golden recipe (chrome-devtools MCP unit, persisted gh auth) as exec scripts.
Keep golden synced to main Reuse wholesale forq's push webhook → box.exec golden-sync.sh → deploy.sh. No GH Actions, no runner.
Per-issue/PR preview fork + URL Reuse runtime, replace trigger Keep /opt/boxd-platform on the golden, invoke preview-handler.sh via box.exec (v1); reimplement in TS later.
Implement issue → PR Reuse Claude+PR step fix-handler.sh: prompt render → claude -p --output-format stream-json → branch/push/gh pr create. stream-json is what the dashboard tails for live progress.
Check/fix a PR Reuse Same handler against the PR head ref; pushes commits back to the PR branch.

Files to lift: boxd-setup-preview/assets/preview-platform/scripts/{lib.sh,deploy.sh,teardown.sh,fork-sweep.sh}, boxd-setup-fix/assets/fix/scripts/fix-handler.sh + claude-prompt.md, boxd-setup-deploy/assets/scripts/golden-sync.sh, the /etc/boxd-platform.conf contract.

What forq must build: the self-serve dashboard, the GitHub App (vs per-repo webhooks), multi-tenant state, per-tenant boxd token issuance, the triage gate, the funding/BYO-key flow, and Speculative.

11. Stack

  • Web: React + Vite SPA + Tailwind v4; port opencode's semantic CSS-variable token system (warm-gray + #dcde8d→#fab283 accent, JetBrains/IBM Plex Mono, 14px base, 2–10px radii, hairline rgba borders). Talks to the backend over REST + SSE (live logs/status). Statically served; no SSR.
  • Backend: standalone TypeScript service (Node or Bun) — the long-running orchestrator (GitHub App webhooks, boxd SDK calls, background jobs, SSE). Kept separate from the SPA precisely because webhooks + background work + streaming don't fit a serverless/SSR model. @boxd-sh/sdk, @octokit/* (App auth, webhooks, REST/GraphQL). Borrow opencode's github/index.ts loop + OIDC→App-token pattern. Optionally expose an OpenAPI surface and generate the SPA's typed client (opencode uses @hey-api/openapi-ts).
  • Inference: Anthropic (Claude Code in-fork, default) + Nebius (triage). Provider keys per funding rules.
  • State: Postgres. Config: forq.yml validated with zod → generated JSON Schema (opencode-style) for editor support.

12. forq.yml (versioned, in the user's repo)

golden:
  install: "npm ci"
  build: "npm run build"
  start: "npm run dev"          # prefer dev/HMR so forks pick up edits
  port: 3000
  branch: main
  deploy: { up, reload, rebuild, rebuild_paths, recreate, recreate_paths, health_path, warm_path }
test:
  command: "npm test"           # the oracle
triage:
  enabled: true
  model: "nebius/<fast-model>"
  auto_act_threshold: 0.7
actions:
  preview: { auto: false }
  fix:     { auto: false, strategy: single-shot, agent: claude-code }
funding:
  default: owner-funded         # or contributor-byo
  trusted: ["@alice", "@bob"]   # owner-funded for these
  new_contributor: contributor-byo

13. Self-serve install

forq is self-hostable (dogfood: deploy it on boxd/Azin). A tenant: (1) deploys or signs into a forq instance, (2) installs the GitHub App on their repos, (3) provides a boxd API key, (4) adds forq.yml and configures a golden per repo. Then they're burning down the queue.

14. Security / multi-tenancy

  • GitHub identity: forq acts as its App installation token (scoped per repo), not a user PAT. PRs are opened as the forq bot.
  • Inference keys: owner key or contributor BYO key, injected into forks per-run via boxd tmpfs/secret, never persisted.
  • boxd tenancy: per-tenant scoped tokens via c.token.create; self-hosters point at their own cluster.
  • Fork isolation: forks are full memory+disk copies — avoid baking shared secrets into the golden; inject per-run.

15. Milestones

Sequenced so there's always something demoable; "go huge" lands at M6–M7.

  • M0 — SDK loop. Backend creates a golden, forks it, execs a command, returns a preview URL via @boxd-sh/sdk. Proves the path.
  • M1 — App + dashboard shell. GitHub App install + OAuth, repo list + open issues/PRs, opencode-styled UI, manual action 1 (preview) working.
  • M2 — Golden config + deploy sync. Per-repo golden provisioning UI + stack analyzer; reuse golden-sync.sh + deploy.sh via push webhook.
  • M3 — Implement → PR (single-shot). Action 2 via fix-handler.sh reuse; live stream-json logs + lifecycle states in the dashboard.
  • M4 — Triage (Nebius). Action 3 gate before fix; verdict in dashboard + as a comment.
  • M5 — Funding / BYO key. forq.yml schema, new-contributor BYO flow, owner opt-in, basic GitHub action 4.
  • M6 — Speculative. Speculative fix strategy: oracle (+ test synthesis), fork-tree best-first search, live tree visualization. The unlimited-compute showpiece.
  • M7 — Whole-queue burndown + polish. Multi-select queue, parallel runs, the headline demo.

MVP that demos the theme: M0–M3 + M6 (Speculative on one hero issue). M4–M5 make it a real product; M7 is the headline shot.

16. Demo script

  1. Dashboard: a repo with a wall of open issues.
  2. Select ~20 → Burn down. Runs fan out in parallel; rows move triaging → implementing → testing → pr_opened.
  3. Triage visibly skips a couple as out-of-scope (cost saved).
  4. Drill into one hard issue → the Speculative tree explodes into live-forked universes, branches flip red/green, the tree collapses onto a 3-line green fix.
  5. Open the winning fork's preview URL — the fix is running. Click through to the opened PR with the synthesized failing test now green.
  6. Counter: "explored N candidate fixes across N live machines in M seconds; opened K PRs."

17. Decisions (locked 2026-06-13)

  1. Web stack: React + Vite SPA + standalone TS backend service. (Not Next.js — the backend is a long-running orchestrator with webhooks/jobs/SSE that doesn't fit serverless/SSR.)
  2. Golden orchestration: Reuse the on-VM bash platform (lib.sh/deploy.sh/fix-handler.sh) via box.exec for v1; move server-side later.
  3. Multi-tenant credentials: forq GitHub App installation token for GitHub + per-run injected inference key (owner or contributor BYO, tmpfs, never persisted).
  4. boxd tenancy: BYO boxd API key per tenant for v1 (self-serve / self-host friendly).
  5. Speculative centrality: single-shot is the default fix strategy; Speculative is opt-in and the hero demo.