|
| 1 | +# AGENTS.md — term-executor |
| 2 | + |
| 3 | +## Project Purpose |
| 4 | + |
| 5 | +**term-executor** is a remote evaluation executor for the [term-challenge](https://github.com/PlatformNetwork/term-challenge) platform. It runs as a containerized Rust service on [Basilica](https://basilica.ai) that receives agent code submissions, executes them against a cloned task repository, runs validation test scripts, and reports pass/fail results. It is the core compute backend that evaluates AI agent coding challenges. |
| 6 | + |
| 7 | +## Architecture Overview |
| 8 | + |
| 9 | +This is a **single-crate Rust binary** (`term-executor`) built with Axum. There are no sub-crates or workspaces. |
| 10 | + |
| 11 | +### Data Flow |
| 12 | + |
| 13 | +``` |
| 14 | +Platform Server → POST /evaluate → term-executor |
| 15 | + 1. Download task archive (.tar.gz / .zip) from task_url |
| 16 | + 2. Parse workspace.yaml, prompt.md, tests/ |
| 17 | + 3. git clone the target repository at base_commit |
| 18 | + 4. Run install commands (pip install, etc.) |
| 19 | + 5. Write & execute agent code in the repo |
| 20 | + 6. Write test source files into the repo |
| 21 | + 7. Run test scripts (bash), collect exit codes |
| 22 | + 8. Return results via GET /evaluate/{id} |
| 23 | +``` |
| 24 | + |
| 25 | +### Module Map |
| 26 | + |
| 27 | +| File | Responsibility | |
| 28 | +|---|---| |
| 29 | +| `src/main.rs` | Entry point — bootstraps config, session manager, executor, Axum server, reaper tasks | |
| 30 | +| `src/config.rs` | `Config` struct loaded from environment variables with defaults | |
| 31 | +| `src/handlers.rs` | Axum route handlers: `/health`, `/status`, `/metrics`, `/evaluate`, `/evaluate/{id}`, `/evaluations` | |
| 32 | +| `src/auth.rs` | Bearer token authentication middleware and `check_token()` helper | |
| 33 | +| `src/executor.rs` | Core evaluation engine — spawns async tasks that clone repos, run agents, run tests | |
| 34 | +| `src/session.rs` | `SessionManager` with `DashMap`, `Session`, `EvalResult`, `EvalStatus`, `EvalStep` types | |
| 35 | +| `src/task.rs` | Task archive download/extraction (zip/tar.gz), `workspace.yaml` parsing, test file loading | |
| 36 | +| `src/metrics.rs` | Atomic counter-based Prometheus metrics (total, passed, failed, active, duration) | |
| 37 | +| `src/cleanup.rs` | Work directory removal, stale session reaping, process group killing | |
| 38 | + |
| 39 | +### Key Shared State (via `Arc`) |
| 40 | + |
| 41 | +- `AppState` (in `handlers.rs`) holds `Config`, `SessionManager`, `Metrics`, `Executor`, `Semaphore` |
| 42 | +- `SessionManager` uses `DashMap<String, Arc<Session>>` for lock-free concurrent access |
| 43 | +- `Semaphore` controls max concurrent evaluations (default: 4) |
| 44 | + |
| 45 | +## Tech Stack |
| 46 | + |
| 47 | +- **Language**: Rust (edition 2021, nightly toolchain for fmt/clippy) |
| 48 | +- **Async Runtime**: Tokio (full features + process) |
| 49 | +- **Web Framework**: Axum 0.7 with Tower middleware |
| 50 | +- **HTTP Client**: reqwest 0.12 (for downloading task archives) |
| 51 | +- **Serialization**: serde + serde_json + serde_yaml |
| 52 | +- **Concurrency**: `DashMap` 6, `parking_lot` 0.12, `tokio::sync::Semaphore` |
| 53 | +- **Archive Handling**: `flate2` + `tar` (tar.gz), `zip` 2 (zip) |
| 54 | +- **Error Handling**: `anyhow` 1 + `thiserror` 2 |
| 55 | +- **Logging**: `tracing` + `tracing-subscriber` with env-filter |
| 56 | +- **Build Tooling**: `mold` linker via `.cargo/config.toml`, `clang` as linker driver |
| 57 | +- **Container**: Multi-stage Dockerfile — `rust:1.93-slim-bookworm` builder → `debian:bookworm-slim` runtime |
| 58 | +- **CI**: GitHub Actions on Blacksmith runners (4/32 vCPU), nightly Rust |
| 59 | + |
| 60 | +## CRITICAL RULES |
| 61 | + |
| 62 | +1. **Always use `cargo +nightly fmt --all` before committing.** The CI enforces `--check` and will reject unformatted code. The project uses the nightly formatter exclusively. |
| 63 | + |
| 64 | +2. **All clippy warnings are errors.** Run `cargo +nightly clippy --all-targets -- -D warnings` locally. CI runs the same command and will fail on any warning. |
| 65 | + |
| 66 | +3. **Never expose secrets in logs or responses.** The `AUTH_TOKEN` environment variable is sensitive. Auth failures log only the `x-forwarded-for` header, never the token value. Follow this pattern for any new secrets. |
| 67 | + |
| 68 | +4. **All process execution MUST have timeouts.** Every call to `run_cmd`/`run_shell` in `src/executor.rs` takes a `Duration` timeout. Never spawn a child process without a timeout — agent code is untrusted and may hang forever. |
| 69 | + |
| 70 | +5. **Output MUST be truncated.** The `truncate_output()` function in `src/executor.rs` caps output at `MAX_OUTPUT` (1MB). Any new command output capture must use this function to prevent memory exhaustion from malicious agent output. |
| 71 | + |
| 72 | +6. **Shared state must use `Arc` + lock-free structures.** `SessionManager` uses `DashMap` (not `Mutex<HashMap>`). Metrics use `AtomicU64`. New shared state should follow these patterns — never use `std::sync::Mutex` for hot-path data. |
| 73 | + |
| 74 | +7. **Semaphore must gate evaluation capacity.** The `Semaphore` in `AppState` limits concurrent evaluations to `MAX_CONCURRENT_EVALS`. Any new evaluation path must acquire a permit before spawning work. |
| 75 | + |
| 76 | +8. **Session cleanup is mandatory.** Every evaluation must clean up its work directory in `src/executor.rs` (the `Cleanup` step). The stale session reaper in `src/cleanup.rs` is a safety net, not a primary mechanism. |
| 77 | + |
| 78 | +9. **Error handling: use `anyhow::Result` for internal logic, `(StatusCode, String)` for HTTP responses.** Handler functions in `src/handlers.rs` return `Result<impl IntoResponse, (StatusCode, String)>`. Internal executor/task functions return `anyhow::Result<T>`. |
| 79 | + |
| 80 | +10. **All new fields on serialized structs must use `#[serde(default)]` or `Option<T>`.** The `EvalRequest`, `EvalResult`, and `WorkspaceConfig` structs are deserialized from external input. Missing fields must not break deserialization. |
| 81 | + |
| 82 | +## DO / DO NOT |
| 83 | + |
| 84 | +### DO |
| 85 | +- Write unit tests for all new public functions (see existing `#[cfg(test)]` modules in every file) |
| 86 | +- Use `tracing::info!`/`warn!`/`error!` for logging (not `println!`) |
| 87 | +- Add new routes in `src/handlers.rs` via the `router()` function |
| 88 | +- Use `tokio::fs` for async file I/O in the executor pipeline |
| 89 | +- Keep the Dockerfile minimal — runtime image has no compilers or language runtimes |
| 90 | +- Use conventional commits (`feat:`, `fix:`, `perf:`, `chore:`, etc.) |
| 91 | + |
| 92 | +### DO NOT |
| 93 | +- Do NOT add `unsafe` code — there is none in this project and it should stay that way |
| 94 | +- Do NOT add synchronous blocking I/O in async functions — use `tokio::task::spawn_blocking` for CPU-heavy work (see `extract_archive` in `src/task.rs`) |
| 95 | +- Do NOT store large data (agent output, test output) in memory without truncation |
| 96 | +- Do NOT add new dependencies without justification — the binary must stay small for container deployment |
| 97 | +- Do NOT use `unwrap()` in production code paths — use `?` or `context()` from anyhow. `unwrap()` is only acceptable in tests and infallible cases (like parsing a known-good string) |
| 98 | +- Do NOT modify `.cargo/config.toml` — it configures the mold linker for fast builds |
| 99 | + |
| 100 | +## Build & Test Commands |
| 101 | + |
| 102 | +```bash |
| 103 | +# Build (debug) |
| 104 | +cargo build |
| 105 | + |
| 106 | +# Build (release, matches CI) |
| 107 | +cargo +nightly build --release -j $(nproc) |
| 108 | + |
| 109 | +# Run tests |
| 110 | +cargo test |
| 111 | + |
| 112 | +# Run tests (release, matches CI) |
| 113 | +cargo +nightly test --release -j $(nproc) -- --test-threads=$(nproc) |
| 114 | + |
| 115 | +# Format (required before commit) |
| 116 | +cargo +nightly fmt --all |
| 117 | + |
| 118 | +# Format check (what CI runs) |
| 119 | +cargo +nightly fmt --all -- --check |
| 120 | + |
| 121 | +# Lint (required before commit) |
| 122 | +cargo +nightly clippy --all-targets -- -D warnings |
| 123 | + |
| 124 | +# Run locally |
| 125 | +AUTH_TOKEN=test PORT=8080 cargo run |
| 126 | + |
| 127 | +# Docker build |
| 128 | +docker build -t term-executor . |
| 129 | +``` |
| 130 | + |
| 131 | +## Git Hooks |
| 132 | + |
| 133 | +The `.githooks/` directory contains automated quality gates: |
| 134 | + |
| 135 | +### pre-commit |
| 136 | +- Runs `cargo +nightly fmt --all -- --check` to enforce formatting |
| 137 | +- Runs `cargo +nightly clippy --all-targets -- -D warnings` to enforce lint |
| 138 | +- Skip with `SKIP_GIT_HOOKS=1 git commit ...` |
| 139 | + |
| 140 | +### pre-push |
| 141 | +- Runs format check, clippy, full test suite, and release build |
| 142 | +- This is the full quality gate matching CI |
| 143 | +- Skip with `SKIP_GIT_HOOKS=1 git push ...` |
| 144 | + |
| 145 | +Both hooks are activated via `git config core.hooksPath .githooks`. |
| 146 | + |
| 147 | +## Environment Variables |
| 148 | + |
| 149 | +| Variable | Default | Description | |
| 150 | +|---|---|---| |
| 151 | +| `PORT` | `8080` | HTTP listen port | |
| 152 | +| `AUTH_TOKEN` | *(none)* | Bearer token for `/evaluate`. If unset, auth is disabled | |
| 153 | +| `SESSION_TTL_SECS` | `1800` | Max session lifetime before reaping | |
| 154 | +| `MAX_CONCURRENT_EVALS` | `4` | Maximum parallel evaluations | |
| 155 | +| `CLONE_TIMEOUT_SECS` | `120` | Git clone timeout | |
| 156 | +| `AGENT_TIMEOUT_SECS` | `600` | Agent execution timeout | |
| 157 | +| `TEST_TIMEOUT_SECS` | `300` | Test suite timeout | |
| 158 | +| `MAX_AGENT_CODE_BYTES` | `5242880` | Max agent code payload (5MB) | |
| 159 | +| `MAX_OUTPUT_BYTES` | `1048576` | Max captured output per command (1MB) | |
| 160 | +| `WORKSPACE_BASE` | `/tmp/sessions` | Base directory for session workspaces | |
0 commit comments