Skip to content

runpod.md

Latest commit

 

History

History
224 lines (182 loc) · 8.75 KB

runpod.md

File metadata and controls

224 lines (182 loc) · 8.75 KB
Raw

RunPod Provider

Read when:

  • choosing provider: runpod;
  • pointing Crabbox at a RunPod pod;
  • changing internal/providers/runpod.

RunPod is a GPU/CPU cloud built around pods, GPU types, CPU flavors, templates, and machines. Crabbox treats a pod as a standard SSH-lease box: it provisions one through the RunPod REST API, waits for public SSH, and from then on uses the usual sync/run/ssh/status/stop paths. RunPod is direct-only — it never goes through the coordinator.

How leasing works

A RunPod pod can expose a public IP and a NAT-mapped public port for each exposed private port. Crabbox deploys pods with ports: ["22/tcp"] and supportPublicIp: true, then polls until RunPod reports the public mapping for port 22. RunPod returns it as publicIp plus portMappings["22"]; older responses surface the same data through runtime.ports[]:

{
  "ip": "203.0.113.7",
  "privatePort": 22,
  "publicPort": 41010,
  "isIpPublic": true,
  "type": "tcp"
}

Once the mapping is live, Crabbox hands the caller a normal SSHTarget pointed at root@<publicIp>:<publicPort> and uses its standard SSH transport. RunPod's basic SSH proxy is not used, because rsync needs the SCP/SFTP support the proxy lacks.

SSH-endpoint discovery has a ten-minute budget covering both API requests and jittered retry waits. Caller cancellation stops discovery and retains its error cause; a completed provider response is not replaced by a simultaneous cancellation. An independent API-client timeout remains a request failure.

SSH auth is public-key only. Crabbox reads the public key matching its configured SSH key and supplies it to the pod as PUBLIC_KEY. The default image installs that key for SSH; a custom image must honor PUBLIC_KEY and start SSH. Provide an existing key with ssh.key in config or CRABBOX_SSH_KEY; the default is ~/.ssh/id_ed25519. A matching public-key file must exist alongside it.

Lifecycle

  • Acquire — deploys a pod named crabbox-<slug>-<leaseSuffix>, then waits for the public TCP mapping for port 22.
  • Resolve — read-only status can look up a pod by lease id, pod id, or pod name. Reusing an existing pod requires an exact local claim bound to its RunPod id and name; use --reclaim once to adopt an unclaimed or legacy pod explicitly.
  • List — enumerates the caller's pods via GET /v1/pods and filters to the crabbox- prefix unless --all is set.
  • Release — requires the exact resource-bound local claim, fetches the pod again to match its id and name, then calls DELETE /v1/pods/{id}. Raw ids, names, and legacy claims alone cannot terminate a pod.
  • Doctor — runs read-only identity and pod-list checks; it never creates a pod, so it is safe to run on every CI invocation.

The local claim owns heartbeat activity, idle timeout, creation time, TTL, and keep policy. Read-only status and list preserve recorded lifecycle values without renewing the claim or replacing them with current configuration. Unclaimed pods have no inferred lifecycle history; explicit adoption initializes that policy. Native stopped or failed state still takes precedence over stored activity. After a native restart, a saved runtime stop/failure no longer masks the running pod. Logical deletion and expiry holds remain intact.

An ordinary heartbeat preserves the recorded idle timeout. An explicit --idle-timeout commits the new timeout together with activity, without extending the recorded creation-based TTL. Missing or stale claim snapshots, cancellation, and changed pod identity are errors rather than successful updates. Earlier releases returned heartbeat changes without persisting them; repeat an intended timeout change after upgrading if it was not recorded in the claim.

Release uses the exact claim observed during acquisition or resolution. If that claim changes before deletion, release refuses the stale target without deleting the pod; resolve it again before retrying. It never substitutes a newer claim for the observed one.

Capabilities

Capability Supported
OS targets Linux only
SSH Yes (public IP + NAT-mapped port 22)
Crabbox sync (rsync over SSH) Yes
Provider-managed sync No
Desktop / browser / code No
Actions hydration Yes (Linux SSH lease)
Coordinator (broker) No — direct only
Tailscale No (rejected; public SSH only)

A non-Linux --target, --tailscale, --class, and --type are all rejected for this provider. Use --runpod-instance-id instead of --class/--type and --runpod-image to set the pod image.

Commands

crabbox warmup --provider runpod
crabbox run --provider runpod -- pnpm test
crabbox ssh --provider runpod
crabbox status --provider runpod
crabbox stop --provider runpod $LEASE_ID
crabbox list --provider runpod

Auth

export RUNPOD_API_KEY=...   # required; from https://www.runpod.io/console/user/settings

CRABBOX_RUNPOD_API_KEY is also accepted and takes precedence over RUNPOD_API_KEY. The key is read from the environment only — there is no CLI flag for it, so it cannot be passed on the command line.

Confirm the same credential RunPod expects:

curl https://rest.runpod.io/v1/pods \
  -H "Authorization: Bearer $RUNPOD_API_KEY"

Crabbox sends the identical Authorization: Bearer $RUNPOD_API_KEY header to the REST pod endpoints. Cross-origin redirects are rejected before credentials or pod-create bodies can be replayed to another destination.

Config

provider: runpod
target: linux
runpod:
  apiUrl: https://rest.runpod.io/v1
  cloudType: SECURE       # SECURE | COMMUNITY
  instanceId: NVIDIA L4,NVIDIA RTX 4000 Ada Generation,NVIDIA RTX A4000,NVIDIA GeForce RTX 3090,NVIDIA GeForce RTX 4090,NVIDIA RTX A5000,NVIDIA RTX A4500
  image: runpod/pytorch:2.8.0-py3.11-cuda12.8.1-cudnn-devel-ubuntu22.04
  templateId: ""          # optional
  diskGB: 20
  user: root
  workRoot: /tmp/crabbox

The defaults pick secure-cloud GPU pods on the RunPod PyTorch image with a 20 GB container disk, because that path reliably exposes the public TCP SSH mapping Crabbox needs for rsync. instanceId accepts a comma-separated GPU priority list — RunPod tries each in order. Override the GPU type, CPU flavor, or image as needed, but verify that the shape you pick exposes public TCP SSH before relying on it.

Provider flags:

--runpod-url
--runpod-cloud-type
--runpod-instance-id
--runpod-image
--runpod-template-id
--runpod-disk-gb
--runpod-user
--runpod-work-root

Environment overrides:

CRABBOX_RUNPOD_API_KEY      (or RUNPOD_API_KEY)
CRABBOX_RUNPOD_API_URL      (or RUNPOD_API_URL)
CRABBOX_RUNPOD_CLOUD_TYPE   (or RUNPOD_CLOUD_TYPE)
CRABBOX_RUNPOD_INSTANCE_ID  (or RUNPOD_INSTANCE_ID)
CRABBOX_RUNPOD_IMAGE        (or RUNPOD_IMAGE)
CRABBOX_RUNPOD_TEMPLATE_ID  (or RUNPOD_TEMPLATE_ID)
CRABBOX_RUNPOD_DISK_GB
CRABBOX_RUNPOD_USER
CRABBOX_RUNPOD_WORK_ROOT

Input and default behavior

The nine settings use shared typed bindings; API keys remain environment-only. Nonempty strings override prior values without trimming, while empty YAML or environment strings preserve them. Visited string flags can explicitly clear a value before the existing selected-provider defaults run.

For diskGB, omitted/null/zero YAML preserves the prior value; a nonzero integer, including a negative value, assigns. The environment parser preserves the prior value for malformed input but accepts parsed zero and negative integers. Runtime defaults then replace a nonpositive disk size with 20 GB. File application and runtime defaulting are separate steps.

The raw provider user and workRoot defaults remain empty. The values in the example above are effective settings: Runpod can inherit a custom generic SSH user or work root, otherwise using its existing root and /tmp/crabbox fallbacks. Endpoint provenance and credential-destination checks remain unchanged.

Cost discipline

Pods are terminated immediately on release. If --keep is set, the pod stays up and keeps accruing cost until you terminate it. Run crabbox list --provider runpod to spot leaked pods. A Crabbox-claimed pod can be cleaned up with crabbox stop --provider runpod <id>. Adopt an unclaimed pod first through a supported reuse command such as crabbox run --provider runpod --id <pod-id> --reclaim -- true, or terminate it directly in RunPod.

Gotchas

  • A funded RunPod account is required. crabbox doctor --provider runpod succeeds on a zero-balance account because it only reads the pod list — the balance shortfall only surfaces when an Acquire runs.
  • The pod's public SSH port is allocated at runtime and changes between pods; never hard-code --ssh-port.
  • RunPod's basic SSH proxy is not a Crabbox transport, because rsync needs SCP/SFTP support.

Related docs: