feat: add multi-agent support (Opencode, VS Code Terminal)

README.md

@@ -1,42 +1,50 @@
 # Pixel Agents
 
-A VS Code extension that turns your AI coding agents into animated pixel art characters in a virtual office.
+A VS Code extension that turns your coding agents and terminals into animated pixel art characters in a virtual office.
 
-Each Claude Code terminal you open spawns a character that walks around, sits at desks, and visually reflects what the agent is doing — typing when writing code, reading when searching files, waiting when it needs your attention.
-
-This is the source code for the free [Pixel Agents extension for VS Code](https://marketplace.visualstudio.com/items?itemName=pablodelucca.pixel-agents) — you can install it directly from the marketplace with the full furniture catalog included.
+Every terminal you open — whether it's Claude Code, Opencode, or a plain VS Code shell — can spawn a character that walks around, sits at desks, and visually shows what it's doing in real time.
 
+This is a fork of the [Pixel Agents extension](https://marketplace.visualstudio.com/items?itemName=pablodelucca.pixel-agents) by [pablodelucca](https://github.com/pablodelucca/pixel-agents), with added support for multiple agent types beyond Claude Code.
 
 ![Pixel Agents screenshot](webview-ui/public/Screenshot.jpg)
 
 ## Features
 
-- **One agent, one character** — every Claude Code terminal gets its own animated character
-- **Live activity tracking** — characters animate based on what the agent is actually doing (writing, reading, running commands)
+- **Multi-agent support** — connect Claude Code, Opencode, or any VS Code terminal as an animated character
+- **Live activity tracking** — characters animate based on real activity (writing, reading, running commands)
+- **VS Code Terminal tracking** — any shell command (`dir`, `npm install`, `ping`, etc.) makes the character animate via VS Code's shell integration API
 - **Office layout editor** — design your office with floors, walls, and furniture using a built-in editor
 - **Speech bubbles** — visual indicators when an agent is waiting for input or needs permission
 - **Sound notifications** — optional chime when an agent finishes its turn
 - **Sub-agent visualization** — Task tool sub-agents spawn as separate characters linked to their parent
 - **Persistent layouts** — your office design is saved and shared across VS Code windows
-- **Diverse characters** — 6 diverse characters. These are based on the amazing work of [JIK-A-4, Metro City](https://jik-a-4.itch.io/metrocity-free-topdown-character-pack).
+- **Diverse characters** — 6 diverse characters based on the amazing work of [JIK-A-4, Metro City](https://jik-a-4.itch.io/metrocity-free-topdown-character-pack)
 
 <p align="center">
   <img src="webview-ui/public/characters.png" alt="Pixel Agents characters" width="320" height="72" style="image-rendering: pixelated;">
 </p>
 
+## Supported Agent Types
+
+| Agent Type | How It Works | Activity Detection |
+|---|---|---|
+| **Claude Code** | Launches `claude` CLI in a terminal | Watches JSONL transcript files — shows specific tool status (Reading, Writing, Running, etc.) |
+| **Opencode** | Launches `opencode` CLI in a terminal | Watches JSONL transcript files — similar to Claude Code |
+| **VS Code Terminal** | Creates a new shell or adopts an existing terminal | Uses VS Code shell integration — detects command start/end, shows the command being run |
+
 ## Requirements
 
-- VS Code 1.109.0 or later
-- [Claude Code CLI](https://docs.anthropic.com/en/docs/claude-code) installed and configured
+- VS Code 1.107.0 or later
+- **For Claude Code agents:** [Claude Code CLI](https://docs.anthropic.com/en/docs/claude-code) installed
+- **For Opencode agents:** [Opencode](https://github.com/nichochar/opencode) installed
+- **For VS Code Terminal agents:** No extra requirements — works with any terminal
 
 ## Getting Started
 
-If you just want to use Pixel Agents, the easiest way is to download the [VS Code extension](https://marketplace.visualstudio.com/items?itemName=pablodelucca.pixel-agents). If you want to play with the code, develop, or contribute, then:
-
 ### Install from source
 
 ```bash
-git clone https://github.com/pablodelucca/pixel-agents.git
+git clone https://github.com/Drepheus/pixel-agents.git
 cd pixel-agents
 npm install
 cd webview-ui && npm install && cd ..
@@ -48,11 +56,24 @@ Then press **F5** in VS Code to launch the Extension Development Host.
 ### Usage
 
 1. Open the **Pixel Agents** panel (it appears in the bottom panel area alongside your terminal)
-2. Click **+ Agent** to spawn a new Claude Code terminal and its character
-3. Start coding with Claude — watch the character react in real time
-4. Click a character to select it, then click a seat to reassign it
+2. Click **+ Agent** and choose an agent type:
+   - **Claude Code** — spawns a Claude Code CLI terminal
+   - **Opencode** — spawns an Opencode CLI terminal
+   - **VS Code Terminal** — spawns a new shell (or use **Adopt Terminal** to connect an existing one)
+3. Run commands or start coding — watch the character react in real time
+4. Click a character to select it, then click a chair to assign it a seat
 5. Click **Layout** to open the office editor and customize your space
 
+### VS Code Terminal Agents
+
+VS Code Terminal agents work with any shell. They detect commands using VS Code's built-in shell integration:
+
+- When you run a command (e.g., `dir`, `npm install`, `git status`), the character walks to its desk and starts typing
+- When the command finishes, the character goes back to idle wandering
+- The status overlay shows the actual command being run (e.g., "Running: npm install")
+
+**Note:** Shell integration must be active in the terminal (it's enabled by default in modern VS Code).
+
 ## Layout Editor
 
 The built-in editor lets you design your office:
@@ -67,21 +88,20 @@ The grid is expandable up to 64×64 tiles. Click the ghost border outside the cu
 
 ### Office Assets
 
-The office tileset used in this project and available via the extension is **[Office Interior Tileset (16x16)](https://donarg.itch.io/officetileset)** by **Donarg**, available on itch.io for **$2 USD**.
+The office tileset used in this project is **[Office Interior Tileset (16x16)](https://donarg.itch.io/officetileset)** by **Donarg**, available on itch.io for **$2 USD**.
 
-This is the only part of the project that is not freely available. The tileset is not included in this repository due to its license. To use Pixel Agents locally with the full set of office furniture and decorations, purchase the tileset and run the asset import pipeline:
+The tileset is not included in this repository due to its license. To use the full furniture catalog, purchase the tileset and run:
 
 ```bash
 npm run import-tileset
 ```
 
-Fair warning: the import pipeline is not exactly straightforward — the out-of-the-box tileset assets aren't the easiest to work with, and while I've done my best to make the process as smooth as possible, it may require some manual tweaking. If you have experience creating pixel art office assets and would like to contribute freely usable tilesets for the community, that would be hugely appreciated.
-
-The extension will still work without the tileset — you'll get the default characters and basic layout, but the full furniture catalog requires the imported assets.
+The extension works without the tileset — you get default characters and basic layout, but the full furniture catalog requires the imported assets.
 
 ## How It Works
 
-Pixel Agents watches Claude Code's JSONL transcript files to track what each agent is doing. When an agent uses a tool (like writing a file or running a command), the extension detects it and updates the character's animation accordingly. No modifications to Claude Code are needed — it's purely observational.
+- **Claude Code / Opencode:** Watches JSONL transcript files to track tool usage. When an agent reads a file, writes code, or runs a command, the character animates accordingly.
+- **VS Code Terminal:** Hooks into `onDidStartTerminalShellExecution` and `onDidEndTerminalShellExecution` to detect when commands are running.
 
 The webview runs a lightweight game loop with canvas rendering, BFS pathfinding, and a character state machine (idle → walk → type/read). Everything is pixel-perfect at integer zoom levels.
 
@@ -92,41 +112,14 @@ The webview runs a lightweight game loop with canvas rendering, BFS pathfinding,
 
 ## Known Limitations
 
-- **Agent-terminal sync** — the way agents are connected to Claude Code terminal instances is not super robust and sometimes desyncs, especially when terminals are rapidly opened/closed or restored across sessions.
-- **Heuristic-based status detection** — Claude Code's JSONL transcript format does not provide clear signals for when an agent is waiting for user input or when it has finished its turn. The current detection is based on heuristics (idle timers, turn-duration events) and often misfires — agents may briefly show the wrong status or miss transitions.
-- **Windows-only testing** — the extension has only been tested on Windows 11. It may work on macOS or Linux, but there could be unexpected issues with file watching, paths, or terminal behavior on those platforms.
-
-## Roadmap
-
-There are several areas where contributions would be very welcome:
-
-- **Improve agent-terminal reliability** — more robust connection and sync between characters and Claude Code instances
-- **Better status detection** — find or propose clearer signals for agent state transitions (waiting, done, permission needed)
-- **Community assets** — freely usable pixel art tilesets or characters that anyone can use without purchasing third-party assets
-- **Agent creation and definition** — define agents with custom skills, system prompts, names, and skins before launching them
-- **Desks as directories** — click on a desk to select a working directory, drag and drop agents or click-to-assign to move them to specific desks/projects
-- **Claude Code agent teams** — native support for [agent teams](https://code.claude.com/docs/en/agent-teams), visualizing multi-agent coordination and communication
-- **Git worktree support** — agents working in different worktrees to avoid conflict from parallel work on the same files
-- **Support for other agentic frameworks** — [OpenCode](https://github.com/nichochar/opencode), or really any kind of agentic experiment you'd want to run inside a pixel art interface (see [simile.ai](https://simile.ai/) for inspiration)
-
-If any of these interest you, feel free to open an issue or submit a PR.
-
-## Contributions
-
-See [CONTRIBUTORS.md](CONTRIBUTORS.md) for instructions on how to contribute to this project.
-
-Please read our [Code of Conduct](CODE_OF_CONDUCT.md) before participating.
-
-## Supporting the Project
+- **Shell integration required for VS Code Terminal agents** — if shell integration isn't active in a terminal, command detection won't work
+- **Agent-terminal sync** — connecting agents to terminal instances can desync when terminals are rapidly opened/closed or restored across sessions
+- **Heuristic-based status detection** — Claude Code/Opencode status detection uses timers and heuristics that can occasionally misfire
+- **Windows-focused testing** — primarily tested on Windows 11; may work on macOS/Linux but could have issues
 
-If you find Pixel Agents useful, consider supporting its development:
+## Upstream
 
-<a href="https://github.com/sponsors/pablodelucca">
-  <img src="https://img.shields.io/badge/Sponsor-GitHub-ea4aaa?logo=github" alt="GitHub Sponsors">
-</a>
-<a href="https://ko-fi.com/pablodelucca">
-  <img src="https://img.shields.io/badge/Support-Ko--fi-ff5e5b?logo=ko-fi" alt="Ko-fi">
-</a>
+This is a fork of [pablodelucca/pixel-agents](https://github.com/pablodelucca/pixel-agents). The multi-agent support (Opencode, VS Code Terminal, terminal activity tracking) was added in this fork.
 
 ## License
 

src/PixelAgentsViewProvider.ts

@@ -2,9 +2,10 @@ import * as fs from 'fs';
 import * as os from 'os';
 import * as path from 'path';
 import * as vscode from 'vscode';
-import type { AgentState } from './types.js';
+import type { AgentState, AgentType } from './types.js';
 import {
 	launchNewTerminal,
+	adoptExistingTerminal,
 	removeAgent,
 	restoreAgents,
 	persistAgents,
@@ -17,6 +18,7 @@ import { loadFurnitureAssets, sendAssetsToWebview, loadFloorTiles, sendFloorTile
 import { WORKSPACE_KEY_AGENT_SEATS, GLOBAL_KEY_SOUND_ENABLED } from './constants.js';
 import { writeLayoutToFile, readLayoutFromFile, watchLayoutFile } from './layoutPersistence.js';
 import type { LayoutWatcher } from './layoutPersistence.js';
+import { startTerminalActivityTracking, cleanupTerminalActivity } from './terminalActivityTracker.js';
 
 export class PixelAgentsViewProvider implements vscode.WebviewViewProvider {
 	nextAgentId = { current: 1 };
@@ -42,6 +44,9 @@ export class PixelAgentsViewProvider implements vscode.WebviewViewProvider {
 	// Cross-window layout sync
 	layoutWatcher: LayoutWatcher | null = null;
 
+	// Terminal activity tracking for vscode-terminal agents
+	terminalActivityDisposable: vscode.Disposable | null = null;
+
 	constructor(private readonly context: vscode.ExtensionContext) {}
 
 	private get extensionUri(): vscode.Uri {
@@ -61,15 +66,31 @@ export class PixelAgentsViewProvider implements vscode.WebviewViewProvider {
 		webviewView.webview.options = { enableScripts: true };
 		webviewView.webview.html = getWebviewContent(webviewView.webview, this.extensionUri);
 
+		// Start terminal activity tracking for vscode-terminal agents
+		if (!this.terminalActivityDisposable) {
+			this.terminalActivityDisposable = startTerminalActivityTracking(
+				this.agents,
+				() => this.webview,
+			);
+		}
+
 		webviewView.webview.onDidReceiveMessage(async (message) => {
 			if (message.type === 'openClaude') {
+				const agentType = (message.agentType as AgentType) || 'claude-code';
 				await launchNewTerminal(
 					this.nextAgentId, this.nextTerminalIndex,
 					this.agents, this.activeAgentId, this.knownJsonlFiles,
 					this.fileWatchers, this.pollingTimers, this.waitingTimers, this.permissionTimers,
 					this.jsonlPollTimers, this.projectScanTimer,
 					this.webview, this.persistAgents,
 					message.folderPath as string | undefined,
+					agentType,
+				);
+			} else if (message.type === 'adoptTerminal') {
+				await adoptExistingTerminal(
+					this.nextAgentId,
+					this.agents, this.activeAgentId,
+					this.webview, this.persistAgents,
 				);
 			} else if (message.type === 'focusAgent') {
 				const agent = this.agents.get(message.id);
@@ -283,8 +304,7 @@ export class PixelAgentsViewProvider implements vscode.WebviewViewProvider {
 				if (agent.terminalRef === closed) {
 					if (this.activeAgentId.current === id) {
 						this.activeAgentId.current = null;
-					}
-					removeAgent(
+					}				cleanupTerminalActivity(id);					removeAgent(
 						id, this.agents,
 						this.fileWatchers, this.pollingTimers, this.waitingTimers, this.permissionTimers,
 						this.jsonlPollTimers, this.persistAgents,
@@ -322,6 +342,8 @@ export class PixelAgentsViewProvider implements vscode.WebviewViewProvider {
 	}
 
 	dispose() {
+		this.terminalActivityDisposable?.dispose();
+		this.terminalActivityDisposable = null;
 		this.layoutWatcher?.dispose();
 		this.layoutWatcher = null;
 		for (const id of [...this.agents.keys()]) {