📌 Proposal: Sandbox Session Heartbeat Monitoring + Snapshot Recovery

[rayrayraykk]

Background / Motivation

Currently, our sandbox session management suffers from several issues:

• Containers remain running even when idle, consuming CPU/memory resources indefinitely.
  This can eventually exhaust system capacity ("blast" the resource limits).
• When a container is destroyed, only the filesystem is persisted;
  runtime configurations such as MCP settings, tool list caches, environment variables are lost.
• Users returning to their session must reinitialize state manually, hurting UX.
• No automatic idle resource cleanup exists in the current design.

Our goals are:

1. Automatically monitor container activity and release idle containers.
2. Save a full runtime snapshot before releasing a container (filesystem + runtime config).
3. Restore snapshot when a new container is created, ensuring seamless continuity.
4. Allow transparent use — end-users should not notice the container restart.
5. Enable extensibility for future runtime data recovery.

---

🎯 Feature Objectives

1. Heartbeat Mechanism

   • Automatically record a container’s last active time (last_active_timestamp).

   • Update heartbeat whenever the user interacts with the sandbox session.

   • Store mapping in Redis or in-memory:

     session_ctx_id -> last_active_timestamp
     

2. Idle Timeout Handling

   • A background job periodically scans heartbeat records.
   • If now - last_active > timeout (default: 5 minutes, configurable) → trigger container destruction.

3. Snapshot Saving

   • Before destroying, save a runtime snapshot:
     • Filesystem (already supported via upload)
     • MCP configuration (via client API)
     • Tools list cache
     • Runtime environment variables / state
     • Secret tokens (optional)
   • Store snapshot as JSON in object storage alongside filesystem data.

4. Snapshot Restoration

   • When starting a new container:
     • Download filesystem & snapshot from storage.
     • Apply snapshot configuration back to the container via API calls:
       • MCP servers → add_mcp_servers(...)
       • Tools → re-register or restore cached state
       • Runtime config → set_runtime_config(...)
       • Secret tokens → inject into environment
   • Ensure the new container's runtime is identical to the previous one.

5. Extensibility

   • Design snapshot format with versioning for future additions:

     {
       "version": "v1",
       "mcp_servers": [...],
       "runtime_config": {...},
       "secret_token": "..."
     }

   • Add more runtime state in later iterations without breaking compatibility.

---

🏗 Module Design

1. Heartbeat Recording Module

• Hook into every user interaction (tool execution, MCP calls, etc.).
• In container connection method (_establish_connection), call update_heartbeat(session_ctx_id).
• Storage options:
  • Redis (recommended for multi-instance deployments)
  • In-memory (single process deployments)

Data structure:

Key: session_ctx_id
Val: last_active_timestamp (float)

---

2. Heartbeat Scanning Module

• Runs periodically in a dedicated thread, scheduler, or cron job.
• For each session in session_mapping:
  • If last active exceeds timeout → restart container(s) in that session.

Example scan interval:
HEARTBEAT_SCAN_INTERVAL = 60 seconds
HEARTBEAT_TIMEOUT = 300 seconds

---

3. Snapshot Saving Module

• In release() process:
  1. Download runtime config via sandbox API (MCP, tools, environment).
  2. Save to snapshot.json in mount directory.
  3. Upload both filesystem and snapshot to object storage (storage_path).

Snapshot file example:

{
  "version": "v1",
  "mcp_servers": [{ "name": "example", "url": "...", "auth": {...} }],
  "runtime_config": { "ENV_VAR_1": "value" },
  "secret_token": "abcdef123456"
}

---

4. Snapshot Restoration Module

• After creating a new container:
  1. Download filesystem and snapshot.json to the new mount directory.
  2. Parse snapshot and call API methods to restore:
     • add_mcp_servers(snapshot.mcp_servers, overwrite=true)
     • Restore tool cache (optional)
     • set_runtime_config(snapshot.runtime_config)
     • Inject secret_token if applicable.

---

🔄 End-to-End Flow

User action (tool call / MCP operation)
       ↓
_establish_connection
       ↓
update_heartbeat(session_ctx_id)

Background heartbeat scan
       ↓
Check inactivity
       ↓
Timeout reached → Restart container:
   ① Save snapshot
   ② Destroy container
   ③ Create new container (mount existing FS)
   ④ Restore snapshot configs
       ↓
Continue serving user without noticeable change

---

📦 Data Storage Layout

Redis Keys:

heartbeat_mapping:<session_ctx_id> -> timestamp
session_mapping:<session_ctx_id> -> [container_names]

Object Storage Paths:

/storage_path/<session_id>/filesystem/...
/storage_path/<session_id>/snapshot.json

---

⚙️ Configurable Parameters

• HEARTBEAT_TIMEOUT → idle timeout duration (seconds)
• HEARTBEAT_SCAN_INTERVAL → period between scans (seconds)
• SNAPSHOT_VERSION → current snapshot schema version
• REDIS_ENABLED → store heartbeat data in Redis or memory

---

📈 Benefits

• Automatically cleans idle containers, reducing resource usage.
• Recovers complete runtime state, making restarts transparent to users.
• Low-intrusion integration — minimal changes to existing API flow.
• Snapshot format supports future expansion.

⚠️ Considerations

• Snapshot integrity: corrupted/missing snapshots should still fall back to filesystem-only recovery.
• Container image upgrades may require converting runtime snapshot configs for compatibility.
• Snapshot save/restore speed depends on object storage performance.

---

🗳 Review Points

1. Default timeout duration and scan interval — suitable values?
2. Should heartbeat timeout be adjustable per user/session?
3. Perform a health check after restoring snapshot to verify container integrity?
4. Define mandatory vs optional snapshot sections (MCP, tools).

[rayrayraykk]

Related to #380