AgoraIO-Conversational-AI · BenWeekes · Apr 16, 2026 · Apr 16, 2026 · Apr 16, 2026 · Apr 17, 2026
diff --git a/AGENTS.md b/AGENTS.md
@@ -0,0 +1,41 @@
+# AI Agent Instructions
+
+This repository uses progressive disclosure documentation. Docs live under
+`docs/ai/` in three levels.
+
+## How to Load
+
+1. Read [docs/ai/L0_repo_card.md](docs/ai/L0_repo_card.md) to identify the repo.
+2. Load ALL 8 files in `docs/ai/L1/`. They are small — load all upfront.
+3. Follow L2 deep-dive links only when L1 isn't detailed enough.
+
+## Git Conventions
+
+### Commit messages — conventional commits
+
+- **Format:** `type: description` or `type(scope): description`
+- **Types:** `feat:` (new feature), `fix:` (bug fix), `chore:` (maintenance, version bumps), `test:` (test additions/changes), `docs:` (documentation)
+- **Scoped variant:** `feat(scope):`, `fix(scope):` — e.g. `feat(auth): add token refresh`
+- **Lowercase after prefix** — `feat: add feature`, not `feat: Add feature`
+- **Present tense** — "add feature", not "added feature"
+
+### Branch names
+
+- **Format:** `type/short-description` — lowercase, hyphen-separated
+- **Types match commit types:** `feat/`, `fix/`, `chore/`, `test/`, `docs/`
+- **Examples:** `feat/token-refresh`, `fix/null-pointer`, `docs/progressive-disclosure`
+
+### General rules
+
+- **No AI tool names** — never mention claude, cursor, copilot, cody, aider, gemini, codex, chatgpt, or gpt-3/4
+- **No Co-Authored-By trailers** — omit AI attribution lines
+- **No --no-verify** — let git hooks run normally
+- **No git config changes** — do not modify user.name or user.email
+
+## Doc Commands
+
+| Command       | When to use                                   |
+| ------------- | --------------------------------------------- |
+| generate docs | no `docs/ai/` directory exists yet            |
+| update docs   | code changed since last `last_reviewed` date  |
+| test docs     | verify docs give agents the right context     |
diff --git a/CLAUDE.md b/CLAUDE.md
@@ -1,5 +1,7 @@
 # CLAUDE.md
 
+Read @AGENTS.md for AI agent instructions, git conventions, and progressive disclosure docs.
+
 Guidelines for AI-assisted development in this repository.
 
 ## Package manager

diff --git a/docs/ai/L0_repo_card.md b/docs/ai/L0_repo_card.md
@@ -0,0 +1,27 @@
+# agora-agent-client-toolkit — Repo Card
+
+> Framework-agnostic TypeScript SDK for integrating Agora Conversational AI into web applications, with optional React hooks.
+
+## Identity
+
+| Field         | Value                                              |
+| ------------- | -------------------------------------------------- |
+| Repo          | `AgoraIO-Conversational-AI/agent-client-toolkit-ts` |
+| Type          | `sdk-library`                                      |
+| Language      | TypeScript                                         |
+| Deploy Target | npm registry (CJS + ESM)                           |
+| Owner         | Agora Conversational AI                            |
+| Last Reviewed | 2026-04-16                                         |
+
+## L1 — Summaries
+
+| File                                     | Purpose                                              | Audience       |
+| ---------------------------------------- | ---------------------------------------------------- | -------------- |
+| [01_setup](L1/01_setup.md)               | Environment setup, quick commands, env vars          | Use & Maintain |
+| [02_architecture](L1/02_architecture.md) | System design, component diagram, data flow          | Maintain       |
+| [03_code_map](L1/03_code_map.md)         | Directory tree, module map, "where does X live?"     | Maintain       |
+| [04_conventions](L1/04_conventions.md)   | Naming, patterns, error handling, testing            | Maintain       |
+| [05_workflows](L1/05_workflows.md)       | Step-by-step: add feature, release, test             | Use            |
+| [06_interfaces](L1/06_interfaces.md)     | API contracts, event schemas, message formats        | Use & Maintain |
+| [07_gotchas](L1/07_gotchas.md)           | Critical gotchas, tribal knowledge, incident lessons | Maintain       |
+| [08_security](L1/08_security.md)         | Security model, trust boundaries, auth, secrets      | Maintain       |
diff --git a/docs/ai/L1/01_setup.md b/docs/ai/L1/01_setup.md
@@ -0,0 +1,91 @@
+# 01 Setup
+
+> Environment setup, dependencies, build commands, and local development for the toolkit monorepo.
+
+## Prerequisites
+
+| Requirement | Version   | Notes                            |
+| ----------- | --------- | -------------------------------- |
+| Node.js     | ≥20       | CI tests on 20, 22, 24          |
+| pnpm        | ≥9        | Only supported package manager   |
+
+- npm and yarn lockfiles are rejected; pnpm only
+- No `.env` files required — this is a library, not a service
+
+## Install and Build
+
+```bash
+pnpm install
+pnpm build          # builds both core and react packages
+```
+
+## Quick Commands
+
+| Command              | What It Does                                    |
+| -------------------- | ----------------------------------------------- |
+| `pnpm install`       | Install all workspace dependencies              |
+| `pnpm build`         | Build core + react packages (tsup)              |
+| `pnpm test`          | Run all tests (vitest)                          |
+| `pnpm lint`          | Run ESLint across all packages                  |
+| `pnpm format`        | Run Prettier across all packages                |
+| `pnpm format:check`  | Check Prettier formatting without writing       |
+| `pnpm typecheck`     | Type-check all packages                         |
+
+## Package Manager
+
+- **pnpm only** — workspace uses `pnpm-workspace.yaml`
+- Workspaces: `packages/*` and `apps/*`
+- The `packages/conversational-ai` package references source in root `src/` (no source duplication)
+
+## Build System
+
+- **tsup** — bundles both packages to CJS + ESM + type declarations
+- Output: `dist/index.js`, `dist/index.mjs`, `dist/index.d.ts`, `dist/index.d.mts`
+- Tree-shakeable (`sideEffects: false`)
+
+## Running Demo Apps
+
+```bash
+cd apps/demo && pnpm dev        # Vanilla TS demo (Vite)
+cd apps/playground && pnpm dev  # React playground
+```
+
+Both apps require valid Agora App ID and token — see their respective README files.
+
+## CI Pipeline
+
+- GitHub Actions: `.github/workflows/ci.yml`
+- Matrix: Node 20, 22, 24 on Ubuntu
+- Steps: lint → format check → build → typecheck → test → coverage (Node 20 only)
+- Publish: triggered by tag push; publishes both packages to npm with provenance
+
+## Package Filter Names
+
+| Filter Name                          | Resolves To              |
+| ------------------------------------ | ------------------------ |
+| `agora-agent-client-toolkit`         | Core SDK package         |
+| `agora-agent-client-toolkit-react`   | React hooks package      |
+| `agora-conversational-ai-demo`       | Vanilla TS demo app      |
+
+Use with `pnpm --filter <name> <script>` to target a specific package.
+
+## Peer Dependencies (for consuming apps)
+
+| Package              | Version   | Required? |
+| -------------------- | --------- | --------- |
+| `agora-rtc-sdk-ng`   | ≥4.23.4   | Yes       |
+| `agora-rtm`          | ≥2.0.0    | Optional  |
+| `react`              | ≥18       | React only |
+| `agora-rtc-react`    | ≥2.0      | React only |
+
+## Common Setup Issues
+
+- **Wrong package manager** — `npm install` or `yarn install` will fail; use `pnpm install`
+- **Node version too low** — Node 18 may work but is not tested; use ≥20
+- **Missing peer deps in apps** — demo apps need `agora-rtc-sdk-ng` installed separately
+- **Workspace protocol errors** — if you see `workspace:*` resolution errors, run `pnpm install` from repo root
+- **Build order matters** — core package must build before react package; `pnpm build` handles this automatically
+
+## Related Deep Dives
+
+- [Build Pipeline](L2/build_pipeline.md) — Multi-target builds, workspace structure, publish workflow
diff --git a/docs/ai/L1/02_architecture.md b/docs/ai/L1/02_architecture.md
@@ -0,0 +1,106 @@
+# 02 Architecture
+
+> System design overview: singleton lifecycle, event-driven rendering, and structural typing for Agora RTC/RTM integration.
+
+## High-Level Architecture
+
+```
+┌─────────────────────────────────────────────────────────────────────┐
+│                         User Application                            │
+│  (Vanilla JS / React / any framework)                               │
+└──────────┬────────────────────────────────────┬─────────────────────┘
+           │                                    │
+    ┌──────▼──────────┐              ┌──────────▼──────────┐
+    │  Core SDK        │              │  React Package       │
+    │  agora-agent-    │◄─────────────│  agora-agent-        │
+    │  client-toolkit  │  wraps       │  client-toolkit-     │
+    │                  │              │  react                │
+    └──────┬──────────┘              └─────────────────────┘
+           │
+    ┌──────▼──────────────────────────────────────────────┐
+    │                  AgoraVoiceAI (singleton)            │
+    │                                                      │
+    │  ┌─────────────┐  ┌──────────────┐  ┌────────────┐ │
+    │  │ Chunked      │  │ SubRender    │  │ Event      │ │
+    │  │ Message      │  │ Controller   │  │ Helper     │ │
+    │  │ Assembler    │  │ (TEXT/WORD/  │  │ (typed     │ │
+    │  │              │  │  CHUNK/AUTO) │  │  emitter)  │ │
+    │  └──────┬───────┘  └──────┬──────┘  └─────┬──────┘ │
+    └─────────┼─────────────────┼────────────────┼────────┘
+              │                 │                │
+    ┌─────────▼─────────────────▼────────────────▼────────┐
+    │          Agora Peer SDKs (structural contracts)      │
+    │  ┌──────────────┐          ┌──────────────┐         │
+    │  │ RTCEngine    │          │ RTMEngine    │         │
+    │  │ (required)   │          │ (optional)   │         │
+    │  └──────────────┘          └──────────────┘         │
+    └─────────────────────────────────────────────────────┘
+```
+
+## Package Structure
+
+| Package                            | Purpose                           | Dependencies          |
+| ---------------------------------- | --------------------------------- | --------------------- |
+| `agora-agent-client-toolkit`       | Core SDK — framework-agnostic     | Zero runtime deps     |
+| `agora-agent-client-toolkit-react` | React hooks wrapping core SDK     | Peer: react, core SDK |
+
+- Core package is usable standalone (Vanilla JS, Vue, Angular, etc.)
+- React package adds hooks and context; depends on core via workspace protocol
+
+## Singleton Lifecycle
+
+```
+init(config) ──► getInstance() ──► subscribeMessage(channel)
+                                          │
+                                   binds RTC listeners
+                                   binds RTM listeners (if present)
+                                   starts SubRenderController
+                                          │
+                                   ◄── events flow ──►
+                                          │
+                              unsubscribe() ──► destroy()
+```
+
+- `init()` is async; validates engine shape in dev mode
+- Calling `init()` while already initializing waits for first, then re-inits
+- `destroy()` is idempotent; resets singleton to null
+
+## Message Processing Pipeline
+
+```
+RTC stream-message ──► ChunkedMessageAssembler.assemble()
+                              │ (reassembles multi-part messages)
+                              ▼
+                       CovSubRenderController.handleMessage()
+                              │ (TEXT/WORD/CHUNK/AUTO mode)
+                              ▼
+                       SubRenderQueue (manages history)
+                              │
+                              ▼
+                       emit TRANSCRIPT_UPDATED
+```
+
+## Agent State Flow
+
+```
+RTM presence event ──► AgoraVoiceAI._handleRtmPresence()
+                              │
+                              ▼
+                       CovSubRenderController.handleAgentStatus()
+                              │
+                              ▼
+                       emit AGENT_STATE_CHANGED
+```
+
+## Key Design Decisions
+
+- **Structural typing** — RTC/RTM engines matched by method shape, not by importing peer SDK types; eliminates version coupling
+- **Optional RTM** — RTC-only mode works for receive-only use cases; RTM adds send/interrupt/state
+- **Pre-bound handlers** — event handlers stored as instance fields for reliable unbinding
+- **Callback wrapping** — each listener wrapped in try/catch so one error does not crash others
+- **Optional dependencies** — `@agora-js/report` and `jszip` dynamically imported with fallback
+
+## Related Deep Dives
+
+- [Rendering Controller](L2/rendering_controller.md) — TEXT/WORD/CHUNK modes, PTS timing, queue system
+- [Event System](L2/event_system.md) — Event types, listener management, error routing
diff --git a/docs/ai/L1/03_code_map.md b/docs/ai/L1/03_code_map.md
@@ -0,0 +1,83 @@
+# 03 Code Map
+
+> Directory structure, module responsibilities, and where to find things in the monorepo.
+
+## Top-Level Layout
+
+```
+repo-root/
+├── src/                          # Core SDK source (canonical)
+│   ├── core/                     # Singleton, config, types, events
+│   ├── rendering/                # Transcript rendering controller
+│   ├── messaging/                # Chunked message assembly
+│   ├── utils/                    # Debug logging, metrics reporters
+│   └── index.ts                  # Public export barrel
+├── packages/
+│   ├── conversational-ai/        # Core package (build config only)
+│   │   ├── __tests__/            # Core tests
+│   │   ├── __typetests__/        # Type-level tests
+│   │   ├── tsup.config.ts
+│   │   ├── vitest.config.ts
+│   │   └── package.json
+│   └── react/                    # React hooks package
+│       ├── src/                  # React source
+│       ├── __tests__/            # React tests
+│       ├── tsup.config.ts
+│       ├── vitest.config.ts
+│       └── package.json
+├── apps/
+│   ├── demo/                     # Vanilla TS demo (Vite)
+│   └── playground/               # React playground
+├── .github/workflows/ci.yml     # CI pipeline
+├── pnpm-workspace.yaml          # Workspace config
+└── package.json                  # Root workspace scripts
+```
+
+## Core SDK Modules (`src/`)
+
+| File                              | Responsibility                                          | Lines | Risk   |
+| --------------------------------- | ------------------------------------------------------- | ----- | ------ |
+| `core/conversational-ai.ts`      | `AgoraVoiceAI` singleton — lifecycle, RTC/RTM binding   | ~850  | High   |
+| `core/types.ts`                   | All type definitions, enums, error classes               | ~350  | Medium |
+| `core/events.ts`                  | `EventHelper` base class, event type definitions         | ~150  | Medium |
+| `core/config.ts`                  | Configuration types and defaults                         | ~90   | Low    |
+| `rendering/sub-render.ts`         | `CovSubRenderController` — transcript rendering          | ~740  | High   |
+| `rendering/sub-render-queue.ts`   | `SubRenderQueue` — chat history and queue management     | ~570  | High   |
+| `rendering/sub-render-pts.ts`     | `SubRenderPTS` — PTS-based word timing                   | ~100  | High   |
+| `messaging/chunked.ts`            | `ChunkedMessageAssembler` — multi-part message assembly  | ~200  | Medium |
+| `utils/debug.ts`                  | Debug logging utilities                                  | ~50   | Low    |
+| `utils/metrics.ts`                | Metrics reporters (Console + optional Agora)             | ~80   | Low    |
+| `index.ts`                        | Public export barrel                                     | ~60   | Low    |
+
+## React Package (`packages/react/src/`)
+
+| File                           | Responsibility                                      |
+| ------------------------------ | --------------------------------------------------- |
+| `use-conversational-ai.ts`     | Flagship hook — init, subscribe, event handling      |
+| `context.ts`                   | `ConversationalAIProvider` + standalone hooks         |
+| `use-transcript.ts`            | Standalone transcript hook (reads from context)      |
+| `use-agent-state.ts`           | Standalone agent state hook                          |
+| `use-agent-metrics.ts`         | Standalone metrics hook                              |
+| `index.ts`                     | Public export barrel                                 |
+
+## Key Observation
+
+- **Source lives in root `src/`** — the `packages/conversational-ai/` directory has build config and tests but no source files
+- This means the core package's `tsup.config.ts` points to `../../src/index.ts` as entry
+
+## Core Files by Task
+
+| Task                              | Start Here                                |
+| --------------------------------- | ----------------------------------------- |
+| Add a new event type              | `src/core/events.ts` + `src/core/types.ts` |
+| Modify transcript rendering       | `src/rendering/sub-render.ts`             |
+| Change message parsing            | `src/messaging/chunked.ts`                |
+| Add a React hook                  | `packages/react/src/`                     |
+| Update public API                 | `src/index.ts` + package `index.ts`       |
+| Add a new config option           | `src/core/config.ts` + `src/core/types.ts` |
+| Fix singleton lifecycle           | `src/core/conversational-ai.ts`           |
+| Update CI pipeline                | `.github/workflows/ci.yml`                |
+
+## Related Deep Dives
+
+- [Build Pipeline](L2/build_pipeline.md) — How source maps to packages, dual CJS/ESM output