diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..74ac8d0 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,41 @@ +# AI Agent Instructions + +This repository uses progressive disclosure documentation. Docs live under +`docs/ai/` in three levels. + +## How to Load + +1. Read [docs/ai/L0_repo_card.md](docs/ai/L0_repo_card.md) to identify the repo. +2. Load ALL 8 files in `docs/ai/L1/`. They are small — load all upfront. +3. Follow L2 deep-dive links only when L1 isn't detailed enough. + +## Git Conventions + +### Commit messages — conventional commits + +- **Format:** `type: description` or `type(scope): description` +- **Types:** `feat:` (new feature), `fix:` (bug fix), `chore:` (maintenance, version bumps), `test:` (test additions/changes), `docs:` (documentation) +- **Scoped variant:** `feat(scope):`, `fix(scope):` — e.g. `feat(auth): add token refresh` +- **Lowercase after prefix** — `feat: add feature`, not `feat: Add feature` +- **Present tense** — "add feature", not "added feature" + +### Branch names + +- **Format:** `type/short-description` — lowercase, hyphen-separated +- **Types match commit types:** `feat/`, `fix/`, `chore/`, `test/`, `docs/` +- **Examples:** `feat/token-refresh`, `fix/null-pointer`, `docs/progressive-disclosure` + +### General rules + +- **No AI tool names** — never mention claude, cursor, copilot, cody, aider, gemini, codex, chatgpt, or gpt-3/4 +- **No Co-Authored-By trailers** — omit AI attribution lines +- **No --no-verify** — let git hooks run normally +- **No git config changes** — do not modify user.name or user.email + +## Doc Commands + +| Command | When to use | +| ------------- | -------------------------------------------- | +| generate docs | no `docs/ai/` directory exists yet | +| update docs | code changed since last `last_reviewed` date | +| test docs | verify docs give agents the right context | diff --git a/CLAUDE.md b/CLAUDE.md index 519d569..8939b6b 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1 +1,3 @@ +Read @AGENTS.md for AI agent instructions, git conventions, and progressive disclosure docs. + Read [AGENT.md](AGENT.md) for project context, configuration guides, and implementation details. Read [README.md](README.md) for architecture overview and getting started. diff --git a/docs/ai/L0_repo_card.md b/docs/ai/L0_repo_card.md new file mode 100644 index 0000000..0e3e1e7 --- /dev/null +++ b/docs/ai/L0_repo_card.md @@ -0,0 +1,27 @@ +# agent-samples — Repo Card + +> Multi-app sample stack demonstrating Agora Conversational AI with Python backend, React voice/video clients, and extensible recipes. + +## Identity + +| Field | Value | +| ------------- | -------------------------------------------- | +| Repo | `AgoraIO-Conversational-AI/agent-samples` | +| Type | `distributed-system` | +| Language | Python (backend), TypeScript/React (clients) | +| Deploy Target | PM2 + nginx (EC2), AWS Lambda (backend) | +| Owner | Agora Conversational AI | +| Last Reviewed | 2026-04-16 | + +## L1 — Summaries + +| File | Purpose | Audience | +| ---------------------------------------- | ---------------------------------------------------- | -------------- | +| [01_setup](L1/01_setup.md) | Environment setup, quick commands, env vars | Use & Maintain | +| [02_architecture](L1/02_architecture.md) | System design, component diagram, data flow | Maintain | +| [03_code_map](L1/03_code_map.md) | Directory tree, module map, "where does X live?" | Maintain | +| [04_conventions](L1/04_conventions.md) | Naming, patterns, error handling, testing | Maintain | +| [05_workflows](L1/05_workflows.md) | Step-by-step: add feature, deploy, configure | Use | +| [06_interfaces](L1/06_interfaces.md) | API contracts, message schemas, SDK integration | Use & Maintain | +| [07_gotchas](L1/07_gotchas.md) | Critical gotchas, tribal knowledge, incident lessons | Maintain | +| [08_security](L1/08_security.md) | Security model, trust boundaries, auth, secrets | Maintain | diff --git a/docs/ai/L1/01_setup.md b/docs/ai/L1/01_setup.md new file mode 100644 index 0000000..e2cc0f6 --- /dev/null +++ b/docs/ai/L1/01_setup.md @@ -0,0 +1,94 @@ +# 01 Setup + +> Environment setup, dependencies, ports, and quick commands for the multi-app sample stack. + +## Prerequisites + +| Requirement | Version | Notes | +| ----------- | ------- | -------------------------------------------- | +| Node.js | ≥20 | Required for React clients; use `nvm use 20` | +| Python | 3.9+ | Required for backend | +| npm | ≥9 | Package manager for React clients | +| pip | latest | Python dependency management | + +- React clients use **npm** (not pnpm/yarn); always use `--legacy-peer-deps` +- `package-lock.json` is gitignored in both React clients +- Pre-commit hook runs Prettier on markdown — run `npx prettier --write` before committing `.md` files +- Commit-msg hook blocks "claude" (case-insensitive) — omit AI tool names + +## Ports + +| App | Port | Profile | +| ------------------------- | ---- | ------- | +| simple-backend | 8082 | — | +| react-voice-client | 8083 | VOICE | +| react-video-client-avatar | 8084 | VIDEO | + +## Quick Start + +### Backend + +```bash +cd simple-backend +python3 -m venv venv && source venv/bin/activate +pip install -r requirements-local.txt +cp .env.example .env # edit with your Agora credentials +python3 -u local_server.py +``` + +### Voice Client + +```bash +cd react-voice-client +npm install --legacy-peer-deps +npm run dev +``` + +### Video Avatar Client + +```bash +cd react-video-client-avatar +npm install --legacy-peer-deps +npm run dev +``` + +## Required Environment Variables (Backend) + +| Variable | Purpose | Required? | +| ----------------- | -------------------------- | --------- | +| `APP_ID` | Agora App ID | Yes | +| `APP_CERTIFICATE` | Agora App Certificate | Yes | +| `CUSTOMER_ID` | Agora REST API customer ID | Yes | +| `CUSTOMER_SECRET` | Agora REST API secret | Yes | + +Profile-specific variables use the pattern `_` (e.g., `VOICE_TTS_VENDOR=rime`). + +## Quick Commands + +| Command | Where | What It Does | +| ---------------------------- | -------------- | -------------------------- | +| `python3 -u local_server.py` | simple-backend | Start backend (unbuffered) | +| `npm run dev` | react clients | Start Next.js dev server | +| `npm run build` | react clients | Production build | +| `pytest` | simple-backend | Run backend tests | +| `npx prettier --write .` | any directory | Format markdown files | + +## PM2 Production Mode + +```bash +pm2 start ecosystem.config.js +``` + +Starts all apps (backend + both clients) with correct ports and environment variables. + +## Common Setup Issues + +- **Python output buffering** — always use `python3 -u` or `PYTHONUNBUFFERED=1`; without this, logs silently buffer +- **npm vs pnpm** — this repo uses npm, not pnpm; `pnpm install` will not work +- **Node version** — Next.js 16 requires Node ≥20.9.0; use `nvm use 20` +- **Missing --legacy-peer-deps** — React 19 + some packages need `--legacy-peer-deps` +- **MLLM_LOCATION not MLLM_REGION** — Agora expects `MLLM_LOCATION` (e.g., `us-central1`) + +## Related Deep Dives + +- [Profile Configuration](L2/profile_configuration.md) — Profile-based env var system, MLLM setup, vendor config diff --git a/docs/ai/L1/02_architecture.md b/docs/ai/L1/02_architecture.md new file mode 100644 index 0000000..1da4310 --- /dev/null +++ b/docs/ai/L1/02_architecture.md @@ -0,0 +1,98 @@ +# 02 Architecture + +> System design overview: multi-app stack with Python backend, React clients, and Agora Conversational AI integration. + +## High-Level Architecture + +``` +┌──────────────────────────────────────────────────────────────┐ +│ User's Browser │ +│ │ +│ ┌─────────────────────┐ ┌──────────────────────────┐ │ +│ │ react-voice-client │ │ react-video-client-avatar │ │ +│ │ (Next.js, port 8083) │ │ (Next.js, port 8084) │ │ +│ │ │ │ │ │ +│ │ - Audio capture │ │ - Audio + video capture │ │ +│ │ - Voice visualization│ │ - Avatar rendering │ │ +│ │ - Transcript display │ │ - Biomarker panels │ │ +│ └──────────┬───────────┘ └──────────┬────────────────┘ │ +│ │ │ │ +│ └────────────┬───────────────┘ │ +│ │ HTTP │ +└──────────────────────────┼────────────────────────────────────┘ + │ + ┌────────────▼────────────┐ + │ simple-backend │ + │ (Flask, port 8082) │ + │ │ + │ - Token generation │ + │ - Agent lifecycle │ + │ - Profile config │ + │ - Optional auth │ + └────────────┬────────────┘ + │ HTTP + ┌────────────▼────────────┐ + │ Agora ConvoAI API │ + │ │ + │ - Agent creation │ + │ - STT + LLM + TTS │ + │ - RTC audio transport │ + │ - RTM messaging │ + └─────────────────────────┘ +``` + +## App Responsibilities + +| App | Language | Role | +| --------------------------- | ---------- | ------------------------------------- | +| `simple-backend` | Python | Token generation, agent orchestration | +| `react-voice-client` | TypeScript | Audio-only conversational UI | +| `react-video-client-avatar` | TypeScript | Video avatar conversational UI | + +## Request Lifecycle + +``` +1. Client → GET /start-agent?channel=X&profile=VOICE +2. Backend → initialize_constants("voice") # load profile env vars +3. Backend → build_token_with_rtm() # v007 token generation +4. Backend → create_agent_payload() # assemble Agora API payload +5. Backend → send_agent_to_channel() # POST to Agora ConvoAI API +6. Backend → return {token, uid, channel, appid, agent_response} +7. Client → RTC join(channel, uid, token) # audio/video connection +8. Client → RTM subscribe(channel) # transcript messages +9. Client → AgoraVoiceAI.init() # toolkit integration +10. [Conversation flows via RTC audio + RTM messages] +11. Client → GET /hangup-agent?agent_id=X +12. Backend → POST to Agora leave API +``` + +## SDK Layer + +React clients depend on three npm packages: + +| Package | Purpose | +| ---------------------------- | ----------------------------- | +| `agora-rtc-sdk-ng` | RTC audio/video transport | +| `agora-rtm` | Real-time messaging | +| `agora-agent-client-toolkit` | Conversational AI integration | +| `@agora/agent-ui-kit` | Pre-built UI components | + +## Key Design Decisions + +- **Profile-based configuration** — backend loads env vars dynamically per request using `_` pattern; supports unlimited profiles without code changes +- **Stateless backend** — no database, no session storage (except optional auth); all state lives in Agora's infrastructure +- **Async custom LLM registration** — `/register-agent` called in background thread; agent starts even if custom LLM is down +- **Memory-only auth tokens** — JWT tokens held in React refs, not localStorage; clears on page refresh for security +- **Vendor abstraction** — TTS, ASR, avatar, and MLLM vendors configured via env vars; backend builds vendor-specific payloads + +## Optional Extensions + +- **Custom LLM** — proxy LLM calls through your own server for augmentation +- **MCP servers** — tool calling via Model Context Protocol +- **Biomarkers** — Shen (video vitals, client-side) and Thymia (voice biomarkers, server-side) +- **Auth** — Google OAuth + SMS 2FA + encrypted session memory + +## Related Deep Dives + +- [Profile Configuration](L2/profile_configuration.md) — Profile system, vendor configs, MLLM setup +- [Agent Lifecycle](L2/agent_lifecycle.md) — Payload building, API calls, custom LLM registration diff --git a/docs/ai/L1/03_code_map.md b/docs/ai/L1/03_code_map.md new file mode 100644 index 0000000..5733a9e --- /dev/null +++ b/docs/ai/L1/03_code_map.md @@ -0,0 +1,91 @@ +# 03 Code Map + +> Directory structure, module responsibilities, and where to find things in the multi-app repo. + +## Top-Level Layout + +``` +agent-samples/ +├── simple-backend/ # Python Flask backend (port 8082) +│ ├── local_server.py # Flask routes and server entry +│ ├── lambda_handler.py # AWS Lambda wrapper +│ ├── core/ # Business logic modules +│ │ ├── config.py # Profile-based env var loading +│ │ ├── agent.py # Payload building + Agora API calls +│ │ ├── tokens.py # v007 token generation (RTC+RTM) +│ │ ├── auth.py # Optional OAuth + SMS 2FA +│ │ ├── consultant_dashboard.py # Optional dashboard integration +│ │ ├── phone_numbers.py # Phone validation +│ │ └── utils.py # Shared utilities +│ ├── templates/ # HTML templates for auth pages +│ ├── tests/ # Pytest test suite +│ └── start.sh # PM2 wrapper script +│ +├── react-voice-client/ # React voice UI (port 8083) +│ ├── app/ # Next.js app directory +│ │ ├── page.tsx # Main page +│ │ ├── layout.tsx # Root layout +│ │ └── globals.css # Global styles +│ ├── components/ +│ │ ├── VoiceClient.tsx # Main voice component +│ │ └── ThemeToggle.tsx # Dark/light mode +│ ├── hooks/ +│ │ ├── useAgoraVoiceClient.ts # RTC+RTM wrapper +│ │ ├── useAudioVisualization.ts # Volume bars animation +│ │ ├── use-audio-devices.ts # Mic selection +│ │ ├── use-is-mobile.ts # Responsive detection +│ │ └── useAutoScroll.ts # Transcript auto-scroll +│ └── lib/theme/ # Theme utilities +│ +├── react-video-client-avatar/ # React video avatar UI (port 8084) +│ ├── (same structure as voice client) +│ ├── components/ +│ │ └── VideoAvatarClient.tsx # Main video avatar component +│ ├── hooks/ +│ │ ├── useAgoraVideoClient.ts # RTC+RTM+video wrapper +│ │ └── useShenai.ts # Video biomarkers hook +│ └── public/shenai-sdk/ # Shen.AI WASM SDK (vendored) +│ +├── recipes/ # Configuration recipes +│ ├── therapist.md # Wellness/biomarker demo +│ └── thymia.md # Voice biomarker integration +│ +├── design/ # Design rationale documents +│ ├── AI_SAMPLES_DESIGN.md +│ └── AI_SAMPLES_UIKIT_TOOLKIT_DEV.md +│ +├── simple-voice-client-no-backend/ # Vanilla JS client (no backend) +├── simple-voice-client-with-backend/ # Vanilla JS client (with backend) +├── ecosystem.config.js # PM2 production config +└── session-timeline.sh # Operational debugging script +``` + +## Core Files by Task + +| Task | Start Here | +| ---------------------------- | ------------------------------------------------------------ | +| Change agent creation logic | `simple-backend/core/agent.py` | +| Add a new profile variable | `simple-backend/core/config.py` | +| Modify token generation | `simple-backend/core/tokens.py` | +| Add a backend route | `simple-backend/local_server.py` | +| Change voice UI | `react-voice-client/components/VoiceClient.tsx` | +| Change video avatar UI | `react-video-client-avatar/components/VideoAvatarClient.tsx` | +| Modify RTC/RTM connection | `*/hooks/useAgoraVoiceClient.ts` or `useAgoraVideoClient.ts` | +| Add a new recipe | `recipes/` | +| Change production deployment | `ecosystem.config.js` | +| Update backend tests | `simple-backend/tests/` | + +## Backend Module Responsibilities + +| Module | Responsibility | +| ------------------- | --------------------------------------------------- | +| `config.py` | Profile-based env var loading, defaults, validation | +| `agent.py` | Build Agora API payloads, send HTTP requests | +| `tokens.py` | v007 token generation with RTC+RTM services | +| `auth.py` | Google OAuth, SMS 2FA, encrypted session memory | +| `local_server.py` | Flask routes, CORS, request handling | +| `lambda_handler.py` | AWS Lambda entry point (wraps Flask app) | + +## Related Deep Dives + +- [Agent Lifecycle](L2/agent_lifecycle.md) — Payload building details, vendor-specific configs diff --git a/docs/ai/L1/04_conventions.md b/docs/ai/L1/04_conventions.md new file mode 100644 index 0000000..277d888 --- /dev/null +++ b/docs/ai/L1/04_conventions.md @@ -0,0 +1,85 @@ +# 04 Conventions + +> Coding patterns, naming rules, error handling, and testing standards used across the sample stack. + +## Language-Specific Conventions + +### Python (Backend) + +- **Module organization** — business logic in `core/`, Flask routes in `local_server.py` +- **Configuration** — centralized in `core/config.py`, lazy-loaded via `initialize_constants(profile)` +- **Docstrings** — present for public functions and classes +- **Error handling** — `ValueError` for config validation, `jsonify()` for HTTP responses +- **Secrets** — always redacted in debug output (regex on sensitive key names) +- **Unbuffered output** — always `python3 -u` or `PYTHONUNBUFFERED=1` + +### TypeScript/React (Clients) + +- **File naming** — PascalCase for components (`VoiceClient.tsx`), camelCase for hooks (`useAgoraVoiceClient.ts`) +- **Hooks** — `use*` prefix, return types explicitly typed +- **State management** — React hooks only (no Redux, no external state library) +- **useState/useRef** — always typed (e.g., `useState(null)`) +- **useEffect** — dependencies always specified (no infinite loops) + +## API Naming + +| Context | Convention | Example | +| ---------------- | ----------------- | ------------------------------- | +| Query parameters | lowercase + `_` | `channel`, `pipeline_id` | +| JSON payload | camelCase | `enable_rtm`, `system_messages` | +| Env variables | UPPER_SNAKE_CASE | `VOICE_TTS_VENDOR` | +| Profile prefix | `_` | `VIDEO_AVATAR_VENDOR` | + +## Profile System + +- Profile names are case-insensitive (VOICE, voice, Voice all work) +- Pattern: `_` — single underscore between profile and variable name +- No fallback: if `VOICE_APP_ID` is empty, it does **not** fall back to `APP_ID` +- Profiles loaded lazily at request time, not at startup + +## Git Hooks + +- **Pre-commit** — runs Prettier on markdown files; always run `npx prettier --write` before committing `.md` +- **Commit-msg** — blocks messages containing "claude" (case-insensitive) +- **No Co-Authored-By** — omit AI attribution lines + +## Commit Messages + +- Lowercase start, present tense +- No AI tool names (claude, cursor, copilot, etc.) +- Format: `type: description` — e.g., `feat: add mcp server config` + +## Package Management + +- **React clients** — npm only, always `--legacy-peer-deps` +- `package-lock.json` is gitignored in both React clients +- **Backend** — pip with `requirements-local.txt` for local dev + +## Testing + +- **Backend** — pytest framework with fixtures in `simple-backend/tests/` +- **Clients** — manual testing + UI kit storybook; no unit test files in client directories +- Coverage thresholds not enforced (sample code, not production library) + +## Secret Handling in Debug Output + +- Backend redacts secrets in `/start-agent?debug=1` responses +- Regex pattern: `(key|token|api_key|secret|certificate|password|authorization|credentials)` +- Strings >8 chars matching: `XXXX***XXXX` +- Curl debug dumps saved to `/tmp/agora_curl_*_YYYYMMDD_HHMMSS.sh` + +## CORS Pattern + +- If request includes `Authorization` header: response uses specific `Origin` + `Credentials: true` +- Otherwise: `Access-Control-Allow-Origin: *` +- Preflight (`OPTIONS`) handled for all routes + +## URL Parameter Conventions + +- React clients read URL params for automation: `?profile=voice&autoconnect=true&returnurl=...` +- Profile param overrides the default profile for backend calls +- `autoconnect=true` starts the agent immediately on page load (used in OAuth return flow) + +## Related Deep Dives + +- [Profile Configuration](L2/profile_configuration.md) — Full profile system details diff --git a/docs/ai/L1/05_workflows.md b/docs/ai/L1/05_workflows.md new file mode 100644 index 0000000..8899cc0 --- /dev/null +++ b/docs/ai/L1/05_workflows.md @@ -0,0 +1,88 @@ +# 05 Workflows + +> Step-by-step guides for common development tasks across the sample stack. + +## Add a New Profile + +1. Choose a profile name (e.g., `THERAPY`) +2. Add `THERAPY_` prefixed variables to `.env` (copy from existing profile section) +3. Minimum required: `THERAPY_APP_ID`, `THERAPY_APP_CERTIFICATE`, `THERAPY_CUSTOMER_ID`, `THERAPY_CUSTOMER_SECRET` +4. Call backend with `?profile=therapy` — config loads automatically +5. No code changes needed; profile system is fully data-driven + +## Add a New TTS Vendor + +1. Edit `simple-backend/core/agent.py` — find `build_tts_config()` +2. Add a new `elif vendor == "your_vendor":` block +3. Build the vendor-specific config dict following the pattern of existing vendors +4. Add env variables: `_TTS_VENDOR=your_vendor`, `_TTS_API_KEY=...` +5. Add defaults to `simple-backend/core/config.py` if needed + +## Add a New Backend Route + +1. Edit `simple-backend/local_server.py` +2. Add route function with `@app.route('/your-route')` +3. Add CORS headers: set `Access-Control-Allow-Origin` and `Access-Control-Allow-Headers` +4. If the route needs profile config, call `initialize_constants(profile)` first +5. Add a test in `simple-backend/tests/` + +## Add a New React Hook + +1. Create `hooks/use{Name}.ts` in the relevant client directory +2. Follow the pattern of existing hooks (typed state, explicit deps) +3. Import and use in the main component (`VoiceClient.tsx` or `VideoAvatarClient.tsx`) +4. If the hook needs cleanup, return a cleanup function from `useEffect` + +## Add a New UI Component + +1. Check if `@agora/agent-ui-kit` already provides the component +2. If not, create in the client's `components/` directory +3. Import Tailwind classes for styling (no external CSS files) +4. For responsive behavior, use `use-is-mobile.ts` hook + +## Deploy with PM2 + +1. Edit `ecosystem.config.js` to add or modify app config +2. Set environment variables in the `env` block +3. For Next.js apps, ensure `NEXT_PUBLIC_BASE_PATH` is set in both build and runtime env +4. Run `pm2 start ecosystem.config.js` +5. Verify with `pm2 status` and `pm2 logs` + +## Deploy Backend to AWS Lambda + +1. Backend supports Lambda via `lambda_handler.py` +2. Package: `simple-backend/core/` + `lambda_handler.py` + `requirements.txt` +3. Set env vars in Lambda configuration (no `.env` file needed) +4. Entry point: `lambda_handler.handler` + +## Run Backend Tests + +```bash +cd simple-backend +source venv/bin/activate +pytest # all tests +pytest tests/test_agent.py # specific test file +pytest -v # verbose output +``` + +## Add a New Recipe + +1. Create `recipes/your_recipe.md` +2. Document: purpose, architecture, required env vars, setup steps +3. Include a profile template section showing all needed `_*` variables +4. Run `npx prettier --write recipes/your_recipe.md` before committing + +## Debug Agent Creation Failures + +1. Start backend with debug: `GET /start-agent?channel=test&profile=voice&debug=1` +2. Check response `debug` field for redacted payload +3. Check `/tmp/agora_curl_*.sh` for full request/response +4. Common issues: + - `"location": null` → `MLLM_LOCATION` not set + - 400 error → invalid avatar ID or missing vendor config + - `-11033 user offline` → agent creation returned 400 + +## Related Deep Dives + +- [Profile Configuration](L2/profile_configuration.md) — Vendor configs, MLLM setup +- [Agent Lifecycle](L2/agent_lifecycle.md) — Payload building, API flow diff --git a/docs/ai/L1/06_interfaces.md b/docs/ai/L1/06_interfaces.md new file mode 100644 index 0000000..cf6ebac --- /dev/null +++ b/docs/ai/L1/06_interfaces.md @@ -0,0 +1,132 @@ +# 06 Interfaces + +> Backend HTTP endpoints, Agora API calls, SDK integration points, and message formats. + +## Backend HTTP Endpoints + +### GET /start-agent + +Creates an Agora Conversational AI agent and returns connection details. + +| Parameter | Type | Required | Description | +| ------------- | ------ | -------- | ------------------------------- | +| `channel` | string | Yes | Channel name to join | +| `profile` | string | No | Config profile (default: voice) | +| `prompt` | string | No | Override system prompt | +| `greeting` | string | No | Override greeting message | +| `connect` | string | No | Connection mode override | +| `pipeline_id` | string | No | Pipeline ID for pipeline mode | +| `debug` | string | No | Include redacted debug payload | + +**Response:** + +```json +{ + "token": "", + "uid": "", + "channel": "", + "appid": "", + "agent": { "agent_id": "...", "create_ts": ... }, + "agent_response": { ... } +} +``` + +### GET /hangup-agent + +| Parameter | Type | Required | Description | +| ---------- | ------ | -------- | ---------------- | +| `agent_id` | string | Yes | Agent ID to stop | +| `profile` | string | No | Config profile | + +### POST /speak + +Push text directly to an agent's TTS pipeline via the Agora Speak API, bypassing the LLM entirely — the agent speaks the exact text provided. + +```json +{ + "agent_id": "string", + "text": "string", + "profile": "string (optional, defaults to 'video')", + "priority": "APPEND | INTERRUPT (optional, defaults to APPEND)" +} +``` + +**Priority semantics:** + +| Priority | Behavior | Use when | +| ----------- | ------------------------------------------------------- | ---------------------------------------------- | +| `APPEND` | Queues text after any currently playing speech | Adding speech without disrupting current audio | +| `INTERRUPT` | Cuts off current speech and speaks new text immediately | Urgent or time-sensitive announcements | + +Invalid priority values default to `APPEND`. + +### GET /health + +Returns `{ "status": "ok", "service": "agora-convoai-backend" }`. + +## Agora ConvoAI API Calls (from Backend) + +| Method | Endpoint | Purpose | +| ------ | --------------------------------------------------------- | ------------ | +| POST | `/api/conversational-ai-agent/v2/projects/{APP_ID}/join` | Create agent | +| POST | `/api/conversational-ai-agent/v2/projects/{APP_ID}/leave` | Stop agent | +| POST | `/api/conversational-ai-agent/v2/projects/{APP_ID}/speak` | Push TTS | + +Authorization: v007 token (preferred) or Basic auth header. + +## SDK Integration (React Clients) + +```typescript +import AgoraRTC from "agora-rtc-sdk-ng"; +import AgoraRTM from "agora-rtm"; +import { AgoraVoiceAI } from "agora-agent-client-toolkit"; +import { AgentVisualizer, Conversation, ... } from "@agora/agent-ui-kit"; +``` + +### RTC Connection + +```typescript +const client = AgoraRTC.createClient({ mode: "rtc", codec: "vp9" }); +await client.join(appId, channel, token, uid); +const audioTrack = await AgoraRTC.createMicrophoneAudioTrack({ + AEC: true, + ANS: true, + AGC: true, +}); +await client.publish(audioTrack); +``` + +### RTM Connection + +```typescript +const rtmClient = new AgoraRTM.RTM(appId, rtmUid, { token }); +await rtmClient.login(); +await rtmClient.subscribe(channel); +``` + +## Custom LLM Endpoints (Optional) + +| Endpoint | Method | Purpose | +| ------------------- | ------ | ------------------------------- | +| `/chat/completions` | POST | OpenAI-compatible LLM proxy | +| `/register-agent` | POST | Start audio subscriber + Thymia | +| `/unregister-agent` | POST | Stop audio subscriber + Thymia | + +## MCP Server Configuration + +```json +[ + { + "name": "memory", + "url": "http://localhost:8090/mcp", + "transport": "streamable_http" + } +] +``` + +Set as `MCP_SERVERS` env var (JSON array). + +## Related Deep Dives + +- [Agent Lifecycle](L2/agent_lifecycle.md) — Full payload structure, vendor-specific configs +- [Profile Configuration](L2/profile_configuration.md) — Environment variable patterns diff --git a/docs/ai/L1/07_gotchas.md b/docs/ai/L1/07_gotchas.md new file mode 100644 index 0000000..54a5c40 --- /dev/null +++ b/docs/ai/L1/07_gotchas.md @@ -0,0 +1,126 @@ +# 07 Gotchas + +> Critical gotchas, tribal knowledge, and non-obvious behaviors across the sample stack. + +## Backend Gotchas + +### Python Output Buffering (Critical) + +- Without `python3 -u` or `PYTHONUNBUFFERED=1`, stdout buffers and logs never appear +- Affects all process managers: local dev, PM2, systemd, Lambda +- Agent IDs, API response codes, and error messages silently disappear + +### MLLM Variable Naming (Critical) + +- Correct: `VOICE_MLLM_LOCATION=us-central1` +- WRONG: `VOICE_MLLM_REGION=us-central1` — Agora API expects LOCATION, not REGION +- WRONG: `VOICE_MLLM_MLLM_VENDOR=vertexai` — double MLLM prefix + +### Profile Prefix Pattern + +- Pattern: `_` — single underscore between profile and variable +- No fallback: if `VOICE_APP_ID` is empty, does NOT fall back to `APP_ID` +- Case-insensitive: VOICE, voice, Voice all normalize to lowercase internally + +### Avatar Sample Rates + +- Akool avatars ONLY support 16kHz audio — set `TTS_SAMPLE_RATE=16000` +- Others (HeyGen, Anam) support 24kHz (ElevenLabs default) or 16kHz +- Mismatch causes audio playback distortion + +### Custom LLM Registration Is Non-Blocking + +- `/register-agent` called in background thread (`daemon=True`) +- If custom-llm server is down, agent still starts — request silently fails +- Response sent to client before registration attempt completes + +### Consultant Dashboard Lookup Failure Modes + +- `REQUIRE_CONSULTANT_DASHBOARD_CLIENT=false` (default): fail-open — if dashboard lookup fails, agent starts without dashboard context +- `REQUIRE_CONSULTANT_DASHBOARD_CLIENT=true`: fail-hard — returns 403 if dashboard status is not `resolved` +- Status codes: `disabled` (not configured), `missing_identity` (no stored identity), `not_found` (404 or `found=false`), `lookup_failed` (HTTP error or network failure) + +### Pipeline Mode Transcript + +- `parameters.transcript` is connection-level (NOT in pipeline config) +- Without it, agent transcript messages are NOT delivered via RTM +- User transcript may still appear (from ASR) but agent responses are missing + +## Frontend Gotchas + +### RTM Console Error Noise + +- Agora RTM SDK logs "joinPresenceColl error: Presence service not connected" +- App does not use presence; this is SDK noise +- Suppressed in `useAgoraVideoClient.ts` via console.error override + +### Audio Track Cleanup Order + +- Must call `rtcClient.unsubscribe(user, "audio")` BEFORE updating React state +- Otherwise: track still playing while state updated → audio glitches + +### Auth Token Persistence + +- Tokens held in memory (`authTokenRef`), NOT in localStorage/sessionStorage +- On page refresh, user must re-authenticate +- Design decision: security over convenience (prevents cross-user access) + +### Audio Visualization Threshold + +- If threshold ≥1.0, bars never light up +- If threshold = 0, bars always active (no volume detection) +- Default 0.15 = require 15% volume before visualization shows + +### Mic Selection Persistence + +- Selected mic stored in localStorage as `selectedMicId` +- If device removed/unplugged, subsequent uses fail silently +- No error recovery — user must manually select a new mic + +### Build-Time Environment Variables + +- `NEXT_PUBLIC_*` env vars evaluated at **build time**, not runtime +- `NEXT_PUBLIC_BASE_PATH` must be set at both build AND `next start` time +- If only set at build, basePath evaluates to `""` at runtime → 404 on all pages + +## Production Deployment Gotchas + +### Nginx Prefix Stripping + +- `location /simple-backend/` with `proxy_pass http://localhost:8082/;` (trailing slash) +- Strips `/simple-backend/` prefix before forwarding to Flask +- Flask routes are `/start-agent`, NOT `/simple-backend/start-agent` + +### PM2 Python Interpreter Bug + +- PM2 ignores `interpreter: "bash"` for Python scripts +- Wraps in JS ProcessContainerFork.js → Python fails to parse JS +- Workaround: use bash wrapper script (`start.sh`) instead + +### SharedArrayBuffer for Shen Biomarkers + +- Shen.AI WASM SDK requires SharedArrayBuffer +- Needs COEP, COOP, CORP headers (configured in `next.config.ts`) +- Missing headers: Shen crashes silently (browser disables SharedArrayBuffer) +- SDK must load from root `/shenai-sdk/index.mjs` — using the Next.js basePath proxy breaks Emscripten pthread workers +- In production, nginx must serve `/shenai-sdk/` as a direct alias with correct MIME types and COOP/COEP headers +- Debug: check browser console for `[Shen]` prefixed logs (`"Loading SDK module..."`, `"SDK ready"`) + +## Debugging + +### Curl Debug Dumps + +- Backend saves timestamped curl scripts to `/tmp/agora_curl_*_YYYYMMDD_HHMMSS.sh` +- View most recent: `ls -lt /tmp/agora_curl_*.sh | head -1` +- Contains full request/response payload (unredacted) + +### Agent Creation 400 Errors + +- Symptom: agent fails, RTM error `-11033: user offline` +- Root cause: Agora API returned 400 +- Debug: check `"location": null` in MLLM params → `MLLM_LOCATION` not set + +## Related Deep Dives + +- [Profile Configuration](L2/profile_configuration.md) — Variable naming gotchas in detail +- [Agent Lifecycle](L2/agent_lifecycle.md) — API error patterns diff --git a/docs/ai/L1/08_security.md b/docs/ai/L1/08_security.md new file mode 100644 index 0000000..83a888d --- /dev/null +++ b/docs/ai/L1/08_security.md @@ -0,0 +1,88 @@ +# 08 Security + +> Security model, trust boundaries, credential handling, and auth options across the sample stack. + +## Trust Boundaries + +``` +┌────────────────────────────────────┐ +│ User's Browser (untrusted) │ +│ - No credentials stored │ +│ - Auth tokens in memory only │ +│ - Receives pre-generated tokens │ +└──────────────┬─────────────────────┘ + │ HTTP (CORS-protected) +┌──────────────▼─────────────────────┐ +│ simple-backend (trusted) │ +│ - Holds all Agora credentials │ +│ - Generates v007 tokens │ +│ - Redacts secrets in debug output │ +│ - Optional OAuth + SMS 2FA │ +└──────────────┬─────────────────────┘ + │ HTTP (authenticated) +┌──────────────▼─────────────────────┐ +│ Agora ConvoAI API (external) │ +│ - Agent lifecycle management │ +│ - Audio/video transport │ +│ - Messaging infrastructure │ +└────────────────────────────────────┘ +``` + +## Credential Storage + +| Credential | Stored Where | Access | +| ------------------ | ------------------- | -------------------------- | +| APP_ID | Backend `.env` | Server-side only | +| APP_CERTIFICATE | Backend `.env` | Server-side only | +| CUSTOMER_ID/SECRET | Backend `.env` | Server-side only | +| TTS/ASR API keys | Backend `.env` | Server-side only | +| RTC token | Client memory (ref) | Per-session, not persisted | +| RTM token | Client memory (ref) | Per-session, not persisted | +| Auth JWT | Client memory (ref) | Clears on page refresh | + +## Token Generation + +- **v007 tokens** — generated server-side with `APP_ID` + `APP_CERTIFICATE` +- Tokens include RTC + RTM service grants +- Tokens are short-lived; clients receive them per-session +- Fallback: Basic auth when `APP_CERTIFICATE` not available (not recommended for production) + +## Secret Redaction + +- Backend redacts secrets in `/start-agent?debug=1` responses +- Regex pattern: `(key|token|api_key|secret|certificate|password|authorization|credentials)` +- Strings >8 chars matching pattern: first 4 + `***` + last 4 +- Curl debug dumps at `/tmp/agora_curl_*.sh` contain unredacted payloads — local access only + +## Optional Authentication Layer + +- **Google OAuth** — login via Google, callback to `/auth/login` +- **SMS 2FA** — Twilio-based phone verification +- **Session encryption** — encrypted session memory on disk +- **JWT tokens** — held in client memory only (not cookies, not localStorage) +- Blueprint-based Flask integration — disabled by default + +## CORS Policy + +- Requests with `Authorization` header: specific `Origin` + `Access-Control-Allow-Credentials: true` +- Requests without auth: `Access-Control-Allow-Origin: *` +- Preflight (`OPTIONS`) handled for all routes + +## Client-Side Security + +- No credentials stored in browser storage (localStorage, sessionStorage, cookies) +- Auth tokens held in React refs — cleared on page refresh +- Sensitive fields redacted in UI debug displays (6+ char strings masked) +- No user input sent directly to Agora API — always proxied through backend + +## Production Recommendations + +- Always use v007 tokens (not Basic auth) in production +- Set `DEBUG=false` to disable curl dump files +- Rotate `APP_CERTIFICATE` periodically +- Use HTTPS termination at nginx/load balancer level +- For Shen biomarkers: COEP/COOP/CORP headers required (configured in `next.config.ts`) + +## Related Deep Dives + +- None diff --git a/docs/ai/L1/L2/_index.md b/docs/ai/L1/L2/_index.md new file mode 100644 index 0000000..2d6d7a4 --- /dev/null +++ b/docs/ai/L1/L2/_index.md @@ -0,0 +1,6 @@ +# Deep Dives Index + +| Document | Summary | Load When | +| ---------------------------------------------------- | -------------------------------------------------- | --------------------------------------------------------------------- | +| [profile_configuration.md](profile_configuration.md) | Profile-based env vars, vendor configs, MLLM setup | Adding profiles, changing vendors, debugging config issues | +| [agent_lifecycle.md](agent_lifecycle.md) | Payload building, Agora API calls, custom LLM flow | Modifying agent creation, adding vendor support, debugging API errors | diff --git a/docs/ai/L1/L2/agent_lifecycle.md b/docs/ai/L1/L2/agent_lifecycle.md new file mode 100644 index 0000000..680dfda --- /dev/null +++ b/docs/ai/L1/L2/agent_lifecycle.md @@ -0,0 +1,136 @@ +# Agent Lifecycle + +> **When to Read This:** Load this document when modifying agent creation logic, adding vendor support, debugging API errors, or understanding the custom LLM registration flow. + +## Overview + +The agent lifecycle spans creation, conversation, and teardown. The backend orchestrates agent creation through the Agora ConvoAI API, optionally registers with a custom LLM server, and handles cleanup on disconnect. + +## Creation Flow + +``` +GET /start-agent?channel=X&profile=voice + │ + ▼ +initialize_constants("voice") + │ load _* env vars + ▼ +build_token_with_rtm(channel, uid) + │ v007 token with RTC + RTM services + ▼ +create_agent_payload() + ├── build_tts_config() # vendor-specific TTS + ├── build_asr_config() # vendor-specific ASR + ├── build_mllm_config() # optional multimodal LLM + ├── build_avatar_config() # optional avatar vendor + ├── build_mcp_servers() # optional MCP tools + │ Assembles full JSON payload for Agora API + ▼ +send_agent_to_channel(payload) + │ POST /api/conversational-ai-agent/v2/projects/{APP_ID}/join + │ Auth: v007 token or Basic auth + ▼ +Return to client: {token, uid, channel, appid, agent_response} + │ + ▼ (async, non-blocking) +register_agent_with_custom_llm() + │ POST /register-agent to custom LLM server + │ Background thread (daemon=True) + │ Silently fails if custom LLM is down +``` + +## Payload Structure (Simplified) + +```json +{ + "name": "_agent", + "properties": { + "channel": "", + "token": "", + "agent_rtc_uid": "", + "remote_rtc_uids": [""], + "enable_rtm": true, + "advanced_features": { + "enable_aivad": true + }, + "llm": { ... }, + "tts": { ... }, + "asr": { ... }, + "mllm": { ... }, // optional + "avatar": { ... }, // optional + "parameters": { + "transcript": { "enable": true } + } + } +} +``` + +## Teardown Flow + +``` +GET /hangup-agent?agent_id=X + │ + ▼ +POST /api/conversational-ai-agent/v2/projects/{APP_ID}/leave + │ body: {"agent_id": X} + ▼ +Return to client: {agent_response} + │ + ▼ (async, non-blocking) +unregister_agent_with_custom_llm() + │ POST /unregister-agent to custom LLM server +``` + +## Vendor-Specific Config Patterns + +### TTS Vendors + +| Vendor | Required Keys | Notes | +| ------------ | ----------------------------- | ----------------------- | +| `rime` | `TTS_API_KEY`, `TTS_VOICE_ID` | Default sample rate 24k | +| `elevenlabs` | `TTS_API_KEY`, `TTS_VOICE_ID` | Default sample rate 24k | +| `openai` | `TTS_API_KEY`, `TTS_VOICE_ID` | — | +| `cartesia` | `TTS_API_KEY`, `TTS_VOICE_ID` | — | + +### Avatar Vendors + +| Vendor | Required Keys | Notes | +| -------- | ------------- | --------------------------------- | +| `heygen` | `AVATAR_ID` | ID format: `Name_Position_public` | +| `anam` | `AVATAR_ID` | — | +| `akool` | `AVATAR_ID` | MUST use 16kHz sample rate | + +### MLLM Vendors + +| Vendor | Required Keys | Notes | +| ----------------- | --------------------------------------------- | ------------------- | +| `vertexai` | `MLLM_API_KEY`, `MLLM_MODEL`, `MLLM_LOCATION` | Location NOT region | +| `openai_realtime` | `MLLM_API_KEY`, `MLLM_MODEL` | Built-in TTS | + +## Pipeline Mode Feature Composition + +Pipeline mode (`PIPELINE_ID` set) delegates TTS/ASR/AIVAD to the pipeline. Not all optional features compose with it: + +| Feature | With Pipeline? | Notes | +| ----------- | -------------- | -------------------------------------------------------------------------- | +| Custom LLM | Yes | `LLM_VENDOR=custom` + `LLM_URL` builds a full `properties.llm` block | +| MCP Servers | No | `build_mcp_servers()` only called in non-pipeline `create_agent_payload()` | +| Avatar | Yes | Avatar config is connection-level, independent of pipeline | +| MLLM | Yes | MLLM config applied via pipeline overrides | +| Transcript | Yes | `parameters.transcript` is connection-level (NOT in pipeline config) | + +If you set `MCP_SERVERS` with a pipeline profile, the MCP config is silently ignored — no error, no servers attached. + +## Common API Errors + +| HTTP Code | Cause | Fix | +| --------- | ------------------------------- | ---------------------------------- | +| 400 | Invalid avatar ID | Check avatar ID format | +| 400 | Missing MLLM location | Set `

_MLLM_LOCATION` | +| 401 | Invalid credentials | Check APP*CERTIFICATE, CUSTOMER*\* | +| 409 | Agent already exists in channel | Call hangup first | + +## See Also + +- [Back to Architecture](../02_architecture.md) +- [Back to Interfaces](../06_interfaces.md) diff --git a/docs/ai/L1/L2/profile_configuration.md b/docs/ai/L1/L2/profile_configuration.md new file mode 100644 index 0000000..76d9189 --- /dev/null +++ b/docs/ai/L1/L2/profile_configuration.md @@ -0,0 +1,91 @@ +# Profile Configuration + +> **When to Read This:** Load this document when adding new profiles, changing vendor configurations, setting up MLLM (multimodal LLM), or debugging configuration issues. + +## Overview + +The backend uses a profile-based configuration system that loads environment variables dynamically per request. Profiles allow multiple configurations (voice, video, therapy, etc.) to coexist in a single `.env` file. + +## How Profiles Work + +```python +# Request: GET /start-agent?profile=voice +initialize_constants("voice") +# Loads: VOICE_TTS_VENDOR, VOICE_ASR_VENDOR, VOICE_APP_ID, etc. +``` + +1. Profile name normalized to lowercase +2. All env vars matching `_*` pattern are loaded +3. Variables stored in module-level globals in `core/config.py` +4. No fallback — if `VOICE_APP_ID` is empty, it stays empty (does NOT use `APP_ID`) + +## Variable Categories + +### Core (Required) + +``` +_APP_ID=... +_APP_CERTIFICATE=... +_CUSTOMER_ID=... +_CUSTOMER_SECRET=... +``` + +### TTS (Text-to-Speech) + +| Variable | Values | +| --------------------- | ------------------------------------------ | +| `

_TTS_VENDOR` | `rime`, `elevenlabs`, `openai`, `cartesia` | +| `

_TTS_API_KEY` | Vendor API key | +| `

_TTS_VOICE_ID` | Vendor-specific voice identifier | +| `

_TTS_SAMPLE_RATE` | `16000` or `24000` (Akool requires 16000) | + +### ASR (Speech-to-Text) + +| Variable | Values | +| ------------------ | ------------------ | +| `

_ASR_VENDOR` | `ares`, `deepgram` | +| `

_ASR_LANGUAGE` | Language code | + +### MLLM (Multimodal LLM) + +| Variable | Values | +| ------------------- | ------------------------------------------- | +| `

_MLLM_VENDOR` | `vertexai`, `openai_realtime` | +| `

_MLLM_MODEL` | Model name | +| `

_MLLM_API_KEY` | Vendor API key | +| `

_MLLM_LOCATION` | Region (e.g., `us-central1`) — NOT `REGION` | + +### Avatar + +| Variable | Values | +| ------------------- | ------------------------- | +| `

_AVATAR_VENDOR` | `heygen`, `anam`, `akool` | +| `

_AVATAR_ID` | Vendor-specific avatar ID | + +### Custom LLM + +| Variable | Values | +| ----------------- | ------------------------ | +| `

_LLM_URL` | URL of custom LLM server | +| `

_LLM_MODEL` | Model name | +| `

_LLM_API_KEY` | API key for custom LLM | + +### MCP Servers + +``` +_MCP_SERVERS=[{"name":"memory","url":"http://localhost:8090/mcp","transport":"streamable_http"}] +``` + +## Common Configuration Mistakes + +| Mistake | Correct Form | +| ---------------------------------- | ---------------------------------- | +| `VOICE_MLLM_REGION=us-central1` | `VOICE_MLLM_LOCATION=us-central1` | +| `VOICE_MLLM_MLLM_VENDOR=vertexai` | `VOICE_MLLM_VENDOR=vertexai` | +| Using `APP_ID` as fallback | Must set `VOICE_APP_ID` explicitly | +| `TTS_SAMPLE_RATE=24000` with Akool | `TTS_SAMPLE_RATE=16000` | + +## See Also + +- [Back to Setup](../01_setup.md) +- [Back to Gotchas](../07_gotchas.md) diff --git a/docs/ai/test-results.md b/docs/ai/test-results.md new file mode 100644 index 0000000..427fb27 --- /dev/null +++ b/docs/ai/test-results.md @@ -0,0 +1,95 @@ +# PD Documentation Test Results + +Tested: 2026-04-16 +Agent: Claude Opus 4.6 +Repo: agent-samples + +## Summary + +- Total questions: 14 +- Passed: 14 (after doc fixes) +- L1 gaps fixed: 3 (06_interfaces.md, 07_gotchas.md ×2) +- L2 gaps fixed: 1 (agent_lifecycle.md) +- Cross-ref issues: 0 +- Structural checks: 11/11 passed + +## Structural Checks + +All checks passed: + +- L0 exists and is under 50 lines +- All 8 L1 files exist (01_setup through 08_security) +- Each L1 file is 80-200 lines +- Each L1 file starts with a purpose statement +- Each L1 file ends with `## Related Deep Dives` +- Total L1 lines under 1,600 +- L2 `_index.md` exists and lists all L2 files +- Each L2 file starts with `> **When to Read This:**` +- All relative links resolve to existing files +- AGENTS.md exists with How to Load, Git Conventions, Doc Commands +- CLAUDE.md references @AGENTS.md + +## Results + +### Setup & Build + +| # | Question | Answer Correct? | Files Read | Level Loaded | Result | +| --- | ---------------------------------------------------------------- | --------------- | ------------------------- | ------------ | ------ | +| 1 | How do I install dependencies and start the backend server? | Yes | L0, 01_setup | L0+L1 | Pass | +| 2 | What environment variables are required and where do I set them? | Yes | L0, 01_setup, 08_security | L0+L1 | Pass | + +### Test & Run + +| # | Question | Answer Correct? | Files Read | Level Loaded | Result | +| --- | ----------------------------------------------------------- | --------------- | ------------ | ------------ | ------ | +| 3 | How do I start both voice and video avatar clients locally? | Yes | L0, 01_setup | L0+L1 | Pass | +| 4 | What ports do the different services run on? | Yes | L0, 01_setup | L0+L1 | Pass | + +### Conventions + +| # | Question | Answer Correct? | Files Read | Level Loaded | Result | +| --- | ---------------------------------------------------------------- | --------------- | --------------------------------- | ------------ | ------ | +| 5 | What naming conventions does this project use for API endpoints? | Yes | L0, 04_conventions, 06_interfaces | L0+L1 | Pass | +| 6 | How are errors handled across the backend and clients? | Yes | L0, 04_conventions, 07_gotchas | L0+L1 | Pass | + +### Development + +| # | Question | Answer Correct? | Files Read | Level Loaded | Result | +| --- | ----------------------------------------- | --------------- | ------------------------------------------ | ------------ | ------ | +| 7 | How would I add a new TTS vendor profile? | Yes | L0, 05_workflows, L2/profile_configuration | L0+L1+L2 | Pass | +| 8 | How would I add a new backend API route? | Yes | L0, 05_workflows, 03_code_map | L0+L1 | Pass | + +### Deep Dive + +| # | Question | Answer Correct? | Files Read | Level Loaded | Result | +| --- | ------------------------------------------------------------------- | --------------- | ------------------------------------------ | ------------ | ------ | +| 9 | How does the profile configuration variable system work internally? | Yes | L0, 05_workflows, L2/profile_configuration | L0+L1+L2 | Pass | +| 10 | What happens during the agent creation and teardown lifecycle? | Yes | L0, 02_architecture, L2/agent_lifecycle | L0+L1+L2 | Pass | + +### Round 2 — Targeted Coverage (Higher-Risk Contracts) + +| # | Question (short) | Answer Correct? | Files Read | Level Loaded | Result | +| --- | ----------------------------------------------------- | --------------- | ------------------------------------------------------------------------------- | ------------ | ------ | +| 11 | POST /speak APPEND vs INTERRUPT semantics | Yes (after fix) | L1/06_interfaces, L2/agent_lifecycle | L0+L1 | Pass | +| 12 | Debug Shen not loading / SharedArrayBuffer / basePath | Yes (after fix) | L1/07_gotchas, L1/03_code_map, L1/08_security | L0+L1 | Pass | +| 13 | Custom LLM reg + consultant-dashboard failure | Yes (after fix) | L1/07_gotchas, L1/02_architecture, L2/agent_lifecycle | L0+L1+L2 | Pass | +| 14 | Pipeline mode + custom LLM / MCP composition | Yes (after fix) | L1/05_workflows, L1/06_interfaces, L2/profile_configuration, L2/agent_lifecycle | L0+L1+L2 | Pass | + +Round 2 found 4 gaps, all fixed: + +- Q11: Added APPEND vs INTERRUPT behavioral table to `06_interfaces.md` +- Q12: Added Shen basePath constraint, nginx requirement, and debug logs to `07_gotchas.md` +- Q13: Added consultant-dashboard failure modes (fail-open vs fail-hard) to `07_gotchas.md` +- Q14: Added pipeline feature composition matrix to `agent_lifecycle.md` + +## Summary (Updated) + +- Total questions: 14 +- Passed: 14 (after fixes) +- L1 gaps fixed: 3 (06_interfaces.md, 07_gotchas.md ×2) +- L2 gaps fixed: 1 (agent_lifecycle.md) +- Cross-ref issues: 0 + +## Recommended Fixes + +All applied — see Round 2 notes above.