Skip to content

Latest commit

 

History

History
774 lines (585 loc) · 22.2 KB

steps.md

File metadata and controls

774 lines (585 loc) · 22.2 KB

SpawnClaw - Setup & Architecture Guide

Overview

SpawnClaw is a platform that lets users spawn isolated AI agent instances (running OpenClaw) inside Docker containers. Users authenticate via MetaMask wallet, deposit $SPAWNCLAW tokens to get USD-based balance, then create instances connected to Telegram or WhatsApp.

Architecture:

Frontend (Next.js) → Control Plane API → Spawn Agent → Docker Containers (OpenClaw)

Key components:

  • apps/web - Landing page (wallet connect)
  • apps/app - Dashboard (instance management, deposits)
  • apps/api - Control Plane API (auth, billing, orchestration)
  • spawn-agent - Docker container lifecycle manager (runs on VPS)
  • docker-runtime - Dockerfile + scripts for OpenClaw containers

Prerequisites

  • Node.js 22+
  • Docker (or Colima on macOS)
  • PostgreSQL database (local, Supabase, or Neon)
  • OpenRouter API key(s) from https://openrouter.ai
  • MetaMask wallet (for frontend auth)

Step 1: Clone & Install Dependencies

git clone <repo-url> && cd spawnclaw
npm install        # root workspace

Step 2: Build the Docker Runtime Image

This image pre-installs OpenClaw to avoid 4+ minute startup delay per container.

cd docker-runtime
docker build -t spawnclaw-runtime .

Important notes:

  • First build takes ~7 minutes (OpenClaw npm install)
  • Subsequent builds are fast if only entrypoint.sh changes (layer caching)
  • The Dockerfile installs OpenClaw via setup.sh, then copies entrypoint.sh last for optimal caching

Step 3: Configure Environment Variables

3a. Spawn Agent (spawn-agent/.env)

SPAWNCLAW_SECRET=your-shared-secret-between-api-and-agent
PORT=4000
GATEWAY_PORT_START=18790
GATEWAY_PORT_END=18850

3b. Control Plane API (apps/api/.env)

PORT=6702

# Must match spawn-agent secret
SPAWN_AGENT_URL=http://localhost:4000
SPAWN_AGENT_SECRET=your-shared-secret-between-api-and-agent

# Database
DATABASE_URL=postgres://user:password@localhost:5432/spawnclaw

# OpenRouter (passed to every container)
OPENROUTER_API_KEY=sk-or-v1-xxx
# Or multiple keys for rotation:
# OPENROUTER_API_KEYS=sk-or-v1-key1,sk-or-v1-key2,sk-or-v1-key3

OPENROUTER_DEFAULT_MODEL=openrouter/auto

# Instance resources
INSTANCE_MEMORY=1g
INSTANCE_CPUS=1

# $SPAWNCLAW Token (for deposit verification)
TREASURY_BASE=0xYourTreasuryAddress
TOKEN_ADDRESS_BASE=0xYourTokenContract
TOKEN_DECIMALS_BASE=18
INSTANCE_COST_USD=15

# Admin
ADMIN_SECRET=your-admin-secret

3c. Dashboard App (apps/app/.env)

NEXT_PUBLIC_API_URL=http://localhost:6702
NEXT_PUBLIC_LANDING_URL=http://localhost:6700

Step 4: Set Up Database

Create the PostgreSQL database and run migrations:

cd apps/api
npx drizzle-kit push

Database tables:

Table Purpose
users Wallet addresses, USD balance, blocked status
instances Running instances, status, gateway endpoints
instances_channels Telegram/WhatsApp config per instance
deposits Verified $SPAWNCLAW token deposits
payments Payment records for instance creation

Step 5: Start Services

Start in this order:

5a. Spawn Agent (on VPS or locally)

cd spawn-agent
node index.js
# Output: Spawn Agent API running on port 4000

5b. Control Plane API

cd apps/api
node index.js
# Output: Control Plane API running on port 6702

5c. Frontend Dashboard

cd apps/app
npm run dev
# Output: Ready on http://localhost:3000

How It Works

Authentication Flow

  1. User clicks "Connect Wallet" on the landing page (apps/web)
  2. MetaMask prompts for wallet connection
  3. Wallet address (e.g., 0xABC...123) is stored in localStorage as authToken
  4. Every API request includes Authorization: Bearer <wallet-address>
  5. API looks up or creates user by wallet address

Deposit & Balance Flow

  1. User sends $SPAWNCLAW tokens to treasury wallet on Base or Solana chain
  2. User submits transaction hash on the deposit page
  3. API verifies the transaction on-chain:
    • Fetches live token price from DexScreener
    • Verifies token transfer to treasury address
    • Calculates USD value of deposited tokens
  4. USD value is credited to user's balance field
  5. Each instance costs $15 (configurable via INSTANCE_COST_USD)

Instance Creation Flow

User Dashboard → POST /api/instances/create → Control Plane API → Spawn Agent → Docker
  1. User fills form: channel (Telegram/WhatsApp), bot token, model selection
  2. API checks balance >= $15, deducts balance
  3. API builds spawn config with OpenRouter keys from env vars
  4. Compute Pool selects best provider (Docker by default)
  5. Docker Provider sends POST to Spawn Agent /spawn
  6. Spawn Agent allocates a unique port (18790-18850), runs docker run: 
    docker run -d --name <instanceId> \
  --memory=1g --cpus=1 \
  -p <port>:<port> \
  -e OPENROUTER_API_KEY=... \
  -e OPENROUTER_MODEL=... \
  -e GATEWAY_PORT=<port> \
  -e TG_BOT_TOKEN=... \
  spawnclaw-runtime
  7. Container starts → entrypoint.sh runs:
    • Picks random API key from OPENROUTER_API_KEYS (if multiple)
    • Runs openclaw onboard with OpenRouter auth
    • Configures Telegram credentials
    • Starts OpenClaw gateway on assigned port with --bind lan
  8. API saves instance to database, returns gateway endpoint to frontend
  9. Dashboard polls instance status every few seconds

Instance Stop Flow

User Dashboard → POST /api/instances/:id/stop → API → Spawn Agent → docker stop + rm
  1. API verifies user owns the instance
  2. Spawn Agent runs docker stop + docker rm
  3. Port is released back to the pool
  4. Instance status updated to "stopped" in database

OpenRouter Integration

SpawnClaw uses OpenRouter as the only AI provider. This gives access to all major models (Claude, GPT, Llama, Gemini, etc.) through a single API.

Key Rotation

If you provide multiple API keys via OPENROUTER_API_KEYS (comma-separated), each container randomly selects one key at startup. This distributes rate limits across keys.

# In apps/api/.env
OPENROUTER_API_KEYS=sk-or-v1-key1,sk-or-v1-key2,sk-or-v1-key3

Model Selection

Users can choose a model when creating an instance. Default is openrouter/auto (cheapest available). The model is passed through the full chain:

Frontend → API (model param) → Docker Provider → Spawn Agent → Container env → OpenClaw config

Port Allocation

Each OpenClaw instance needs its own gateway port. The Spawn Agent manages this automatically:

  • Range: 18790-18850 (configurable via GATEWAY_PORT_START/GATEWAY_PORT_END)
  • Assignment: Sequential, first available port
  • Mapping: 1:1 host-to-container (-p 18790:18790)
  • Release: Port freed when instance is stopped
  • Capacity: 60 ports per agent, but RAM is usually the bottleneck

Server Capacity

For a VPS with 12GB RAM / 6 CPU cores:

Metric Value
Peak RAM per instance (onboarding) ~800MB
Steady-state RAM per instance ~300MB
Recommended instances 6-8
Max instances (theoretical) ~10
Port range capacity 60

Key constraint: RAM, not ports. Each instance needs ~1GB allocated (--memory=1g), with actual usage around 300MB steady-state but 800MB during OpenClaw onboarding.

Multi-VPS Setup

You can scale beyond a single VPS by adding multiple servers. Each VPS runs its own spawn-agent, and the Control Plane API distributes instances across them.

Frontend → Control Plane API (1 server)
                ↓
           ComputePool (scheduler)
           ↙     ↓      ↘
      VPS-1    VPS-2    VPS-3
  (spawn-agent) (spawn-agent) (spawn-agent)
      ↓          ↓          ↓
   Docker     Docker     Docker

Setup each VPS

  1. Run setup-vps.sh on each VPS (installs Docker, Node.js, etc.)
  2. On each VPS, only start the spawn-agent (not the API or frontend): 
    cd /opt/spawnclaw/spawn-agent
npm install
pm2 start index.js --name spawn-agent
  3. Open firewall port 4000 for the API server's IP only: 
    ufw allow from <API_SERVER_IP> to any port 4000

Configure the API

On the API server, set DOCKER_PROVIDERS in apps/api/.env with all VPS servers:

DOCKER_PROVIDERS='[
  {
    "id": "vps-us-1",
    "name": "US East VPS",
    "url": "http://167.71.0.1:4000",
    "secret": "shared-secret-vps1",
    "region": "us-east",
    "maxInstances": 8,
    "priority": 10
  },
  {
    "id": "vps-eu-1",
    "name": "EU Frankfurt VPS",
    "url": "http://138.68.0.1:4000",
    "secret": "shared-secret-vps2",
    "region": "eu-west",
    "maxInstances": 8,
    "priority": 5
  }
]'

Note: When DOCKER_PROVIDERS is set, the single SPAWN_AGENT_URL/SPAWN_AGENT_SECRET vars are ignored.

Provider fields

Field Required Default Description
id Yes - Unique provider ID (e.g., vps-us-1)
url Yes - Spawn agent URL on this VPS
secret Yes - Shared secret (must match VPS's SPAWNCLAW_SECRET)
name No same as id Human-readable name
region No default Region label for region-based scheduling
maxInstances No 50 Max instances on this VPS
priority No 5 Higher = preferred (used by priority_first strategy)
costPerHour No 0.01 Cost tracking (used by cost_optimized strategy)
enabled No true Set false to disable without removing

Scheduling strategies

Control which VPS gets new instances via SCHEDULER_STRATEGY:

Strategy Behavior
priority_first Use highest-priority VPS first (default)
capacity_balanced Use VPS with most available slots
cost_optimized Use cheapest VPS first
round_robin Rotate across all VPS servers
random Random selection

Failover

If spawn fails on the selected VPS, the ComputePool automatically retries on the next available provider. No manual intervention needed.

Monitoring

# List all providers and their status
curl http://your-api:6702/api/compute/providers

# Check combined capacity across all VPS
curl http://your-api:6702/api/compute/capacity

# Health check per provider
curl http://your-api:6702/api/compute/health

Frontend

The frontend needs zero changes for multi-VPS. The compute pool is transparent - the API picks the best VPS and returns the gateway endpoint. Instances are tracked with their provider_id in the database.

API Endpoints Reference

Public

Method Endpoint Description
GET /health System health check
GET /api/treasury Treasury wallet addresses
GET /api/pricing Token price & instance cost

Authenticated (Bearer: wallet address)

Method Endpoint Description
GET /api/balance User's USD balance & instance count
POST /api/deposits/verify Verify token deposit (chain, txHash)
POST /api/instances/create Create new instance
GET /api/instances List user's instances
GET /api/instances/:id Get instance details
POST /api/instances/:id/stop Stop an instance
POST /api/instances/:id/status Check instance status

Admin (Bearer: ADMIN_SECRET)

Method Endpoint Description
PATCH /api/admin/users/:id/block Block/unblock user
PATCH /api/admin/providers/:id/enable Enable/disable provider
GET /api/compute/providers List compute providers
GET /api/compute/capacity Pool capacity info
GET /api/compute/health Provider health status

Frontend Integration

The dashboard (apps/app) connects to the API using the APIClient class in apps/app/lib/api.ts.

Connecting Wallet

// apps/web/components/ConnectWallet.tsx
const accounts = await window.ethereum.request({ method: "eth_requestAccounts" });
const walletAddress = accounts[0];
apiClient.setToken(walletAddress); // Stored in localStorage as authToken

Creating an Instance

// apps/app/lib/api.ts
const result = await apiClient.createInstance({
  activator: "general",
  channel: "telegram",
  channelConfig: { telegramBotToken: "123456:ABC..." },
  selectedModels: ["openrouter/auto"],
});
// result: { instanceId, status, gatewayEndpoint, ... }

Checking Status

const instances = await apiClient.getInstances();
// Polls every few seconds for live status updates

Verifying Deposit

const result = await apiClient.verifyDeposit("base", "0xTxHash...");
// result: { balanceUsd, instances, creditedUsd, ... }

VPS Setup (Automated)

Run the setup script on a fresh Ubuntu 22.04/24.04 VPS:

sudo bash setup-vps.sh

This installs: Docker, Node.js 22, pnpm, PM2, PostgreSQL, and configures the firewall.

See setup-vps.sh in the repo root for the full script.

Production Deployment - VPS (Backend)

1. Run the setup script

sudo bash setup-vps.sh

2. Clone repo & install

cd /opt/spawnclaw
git clone <your-repo-url> .
pnpm install

3. Build Docker runtime image

cd /opt/spawnclaw/docker-runtime
docker build -t spawnclaw-runtime .
# Takes ~7 min first time (OpenClaw install), fast after that

4. Edit env files

The setup script creates template files. Update them with real values:

# Edit the secrets
nano /opt/spawnclaw/.env.spawn-agent
nano /opt/spawnclaw/.env.api

# Copy to service directories
cp /opt/spawnclaw/.env.spawn-agent /opt/spawnclaw/spawn-agent/.env
cp /opt/spawnclaw/.env.api /opt/spawnclaw/apps/api/.env

CRITICAL: Make sure SPAWNCLAW_SECRET matches in both .env.spawn-agent and .env.api (SPAWN_AGENT_SECRET).

5. Set up database

cd /opt/spawnclaw/apps/api
npx drizzle-kit push

6. Install spawn-agent dependencies

cd /opt/spawnclaw/spawn-agent
npm install

7. Start services with PM2

cd /opt/spawnclaw/spawn-agent && pm2 start index.js --name spawn-agent
cd /opt/spawnclaw/apps/api && pm2 start index.js --name spawnclaw-api
pm2 save
pm2 startup   # auto-start on reboot

8. Verify

curl http://localhost:4000/health
# {"status":"ok","timestamp":"..."}

curl http://localhost:6702/health
# {"status":"ok","timestamp":"...","database":{"count":0},"computePool":{...}}

Production Deployment - Frontend

The frontend has two apps: Landing Page (apps/web) and Dashboard (apps/app).

Option A: Deploy to Vercel (Recommended)

Both are Next.js apps, so Vercel is the easiest option.

Landing Page (apps/web):

cd apps/web

Vercel env variables:

NEXT_PUBLIC_DASHBOARD_URL=https://app.yourdomain.com
NEXT_PUBLIC_DYNAMIC_ENVIRONMENT_ID=your-dynamic-id     # if using Dynamic.xyz for wallet

Dashboard (apps/app):

cd apps/app

Vercel env variables:

NEXT_PUBLIC_API_URL=https://api.yourdomain.com          # Your VPS API URL
NEXT_PUBLIC_LANDING_URL=https://yourdomain.com           # Landing page URL
NEXT_PUBLIC_DYNAMIC_ENVIRONMENT_ID=your-dynamic-id       # if using Dynamic.xyz

Deploy each app separately on Vercel:

# From repo root
cd apps/web && vercel --prod
cd apps/app && vercel --prod

Or connect the repo to Vercel and set the root directory to apps/web or apps/app.

Option B: Self-host on VPS

Build and serve with PM2:

# Landing page
cd /opt/spawnclaw/apps/web
cp .env.example .env.local
# Edit .env.local with production URLs
npm run build
pm2 start npm --name spawnclaw-web -- start   # Serves on port 6700

# Dashboard
cd /opt/spawnclaw/apps/app
cp .env.example .env.local
# Edit .env.local with production URLs
npm run build
pm2 start npm --name spawnclaw-app -- start   # Serves on port 6701

Option C: Reverse proxy with Nginx

If self-hosting, put Nginx in front for SSL:

server {
    listen 443 ssl;
    server_name yourdomain.com;
    # SSL certs (use certbot)

    location / {
        proxy_pass http://127.0.0.1:6700;
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}

server {
    listen 443 ssl;
    server_name app.yourdomain.com;

    location / {
        proxy_pass http://127.0.0.1:6701;
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}

server {
    listen 443 ssl;
    server_name api.yourdomain.com;

    location / {
        proxy_pass http://127.0.0.1:6702;
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}

Then install SSL:

apt install nginx certbot python3-certbot-nginx -y
certbot --nginx -d yourdomain.com -d app.yourdomain.com -d api.yourdomain.com

Environment Variables Reference

Spawn Agent (spawn-agent/.env)

Variable Required Default Description
SPAWNCLAW_SECRET Yes - Shared secret with API (must match SPAWN_AGENT_SECRET)
PORT No 4000 HTTP port for spawn agent API
GATEWAY_PORT_START No 18790 First port in gateway range
GATEWAY_PORT_END No 18850 Last port in gateway range

Control Plane API (apps/api/.env)

Variable Required Default Description
Core
PORT No 6702 HTTP port for API server
NODE_ENV No development development or production
SPAWN_AGENT_URL No http://localhost:4000 URL to spawn agent
SPAWN_AGENT_SECRET Yes - Shared secret (must match SPAWNCLAW_SECRET)
DATABASE_URL Yes - PostgreSQL connection string
ADMIN_SECRET No - Secret for admin endpoints
OpenRouter
OPENROUTER_API_KEY Yes* - Single OpenRouter API key
OPENROUTER_API_KEYS Yes* - Comma-separated keys for rotation (overrides single key)
OPENROUTER_DEFAULT_MODEL No openrouter/auto Default AI model
Instance Resources
INSTANCE_MEMORY No 1g RAM per container (min 1g for OpenClaw)
INSTANCE_CPUS No 1 CPU cores per container
Token & Billing
INSTANCE_COST_USD No 15 USD cost per instance
TREASURY_BASE No - Base chain treasury wallet(s), comma-separated
TREASURY_SOLANA No - Solana treasury wallet(s), comma-separated
TOKEN_ADDRESS_BASE No - $SPAWNCLAW contract on Base
TOKEN_ADDRESS_SOLANA No - $SPAWNCLAW mint on Solana
TOKEN_DECIMALS_BASE No 18 Token decimals (Base)
TOKEN_DECIMALS_SOLANA No 6 Token decimals (Solana)
BASE_RPC_URL No https://mainnet.base.org Base chain RPC
SOLANA_RPC_URL No https://api.mainnet-beta.solana.com Solana RPC
Docker Provider
DOCKER_PROVIDER_ENABLED No true Enable Docker provider
DOCKER_PROVIDER_PRIORITY No 5 Scheduling priority
DOCKER_PROVIDER_MAX_INSTANCES No 50 Max instances this provider
DOCKER_PROVIDER_REGION No default Region label
DOCKER_PROVIDER_COST No 0.01 Cost per hour (internal tracking)
SCHEDULER_STRATEGY No priority_first priority_first, capacity_balanced, cost_optimized, round_robin, random

*One of OPENROUTER_API_KEY or OPENROUTER_API_KEYS is required.

Landing Page (apps/web/.env.local)

Variable Required Default Description
NEXT_PUBLIC_DASHBOARD_URL No http://localhost:6701 Dashboard URL (redirect after wallet connect)
NEXT_PUBLIC_DYNAMIC_ENVIRONMENT_ID No - Dynamic.xyz environment ID (if using Dynamic wallet)

Dashboard (apps/app/.env.local)

Variable Required Default Description
NEXT_PUBLIC_API_URL Yes http://localhost:6702 Control Plane API URL
NEXT_PUBLIC_LANDING_URL No http://localhost:6700 Landing page URL
NEXT_PUBLIC_DYNAMIC_ENVIRONMENT_ID No - Dynamic.xyz environment ID

Default Ports

Service Port Notes
Landing Page (apps/web) 6700 Next.js
Dashboard (apps/app) 6701 Next.js
Control Plane API (apps/api) 6702 Express
Spawn Agent (spawn-agent) 4000 Express (internal only)
Gateway range 18790-18850 One per OpenClaw instance

Test End-to-End

# 1. Health check
curl http://your-vps:6702/health

# 2. Create instance via curl (use any wallet address for auth)
curl -X POST http://your-vps:6702/api/instances/create \
  -H "Authorization: Bearer 0xYourWalletAddress" \
  -H "Content-Type: application/json" \
  -d '{
    "activator": "general",
    "channel": "telegram",
    "channelConfig": { "telegramBotToken": "YOUR_BOT_TOKEN" },
    "model": "openrouter/auto"
  }'

# 3. List instances
curl http://your-vps:6702/api/instances \
  -H "Authorization: Bearer 0xYourWalletAddress"

# 4. Check container is running
docker ps

# 5. Check container logs
docker logs <instanceId>

Troubleshooting

Issue Fix
Container OOM during onboard Increase INSTANCE_MEMORY to 2g
--model flag not found Don't pass model to openclaw onboard, only to gateway config
Gateway bind error Use --bind lan (not 0.0.0.0)
OpenClaw not found in container Rebuild image: docker build -t spawnclaw-runtime .
Port already in use Check docker ps for orphaned containers, stop them
4+ min container startup Rebuild Docker image (OpenClaw should be pre-installed)
set -e exits on onboard Already fixed with `
API returns "User not found" User must deposit first, or use ensureUserByWallet endpoint
"Insufficient balance" on create Deposit $SPAWNCLAW tokens worth at least $15 USD
Spawn Agent unreachable Check firewall (port 4000), verify SPAWN_AGENT_URL matches
Database connection error Verify DATABASE_URL, check PostgreSQL is running
Frontend can't reach API Check NEXT_PUBLIC_API_URL, add CORS if needed