Skip to content

feat: integrate d-back browser runtime for live mock actors#7

Closed
NNTin wants to merge 4 commits intomainfrom
feat/integrate-d-back-browser-runtime
Closed

feat: integrate d-back browser runtime for live mock actors#7
NNTin wants to merge 4 commits intomainfrom
feat/integrate-d-back-browser-runtime

Conversation

@NNTin
Copy link
Owner

@NNTin NNTin commented Mar 17, 2026

Summary

  • Integrates d-back WebSocket client into the browser dev runtime so the pixel office populates with live mock agents instead of showing an empty map
  • Brings in the full browser runtime infrastructure from the feat/stubbing-vscode-comments feature branch (asset loading pipeline, Vite dev middleware, runtime detection, browserMock.ts)
  • Adds d-back/ to .gitignore (reference clone, not project source)

How it works

In browser mode (npm run dev in webview-ui/), the app:

  1. Loads assets via initBrowserMock() in main.tsx (sprites, layouts — pre-decoded by Vite middleware in dev, decoded in-browser in prod)
  2. Connects to d-back at wss://hermes.nntin.xyz/dzone (6 s timeout)
  3. Auto-joins the default server, receives the initial user list
  4. Dispatches existingAgents before layoutLoaded so users are buffered and seated correctly
  5. Ongoing events from d-back map to Pixel Agents messages:
    • presence: online/dndagentStatus: 'active' (agent walks to desk, types)
    • presence: idle/offlineagentStatus propagated (agent wanders)
    • message event → agentToolStart + agentToolDone (3 s activity window)

Falls back gracefully to an empty office if d-back is unreachable.

New file

  • webview-ui/src/dbackRuntime.ts — WebSocket client, uid→agentId mapping, presence + message handlers

Test plan

  • cd webview-ui && npm run dev opens browser at http://localhost:3000 with agents from the live d-back Hermes instance visible in the pixel office
  • Agents change active/idle states every ~4 s (d-back periodic presence updates)
  • Chat bubbles/tool activity appear for agents every ~5 s (d-back periodic messages)
  • VS Code extension mode is unaffected (d-back code is tree-shaken, isBrowserRuntime guard)
  • npm run build succeeds with dbackRuntime in its own chunk
  • npm test passes (2/2)

🤖 Generated with Claude Code

@vercel
Copy link

vercel bot commented Mar 17, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
pixel-agents Ready Ready Preview, Comment Mar 18, 2026 5:18pm

@NNTin NNTin changed the base branch from main to feat/stubbing-vscode March 17, 2026 17:40
@NNTin NNTin changed the base branch from feat/stubbing-vscode to feat/stubbing-vscode-comments March 17, 2026 17:44
@NNTin
Copy link
Owner Author

NNTin commented Mar 17, 2026

Experimental PR, not suited for production. It uses the backend of https://nntin.xyz/d-zone/ to spawn the agents and have them send messages.

image

This code is AI slop pure. It works but comes with large technical debt. It's better when proper interfaces have been established.

NNTin and others added 3 commits March 18, 2026 17:59
- Add src/plugin/types.ts: IPixelAgentsPlugin, IAgentProvider, IAgentHandle,
  IMessageBridge, IRuntimeUI, PostMessage, and related interfaces
- Add src/plugin/registry.ts: registerPlugin/getPlugin singleton
- Add src/vscode/VSCodePlugin.ts: WebviewViewProvider + IPixelAgentsPlugin impl
- Add src/vscode/VSCodeAgentProvider.ts: terminal lifecycle via IAgentProvider
- Add src/vscode/VSCodeMessageBridge.ts: webview ↔ extension bridge
- Add src/vscode/VSCodeRuntimeUI.ts: VS Code dialog/state APIs via IRuntimeUI
- Add src/core/agentLifecycle.ts: runtime-agnostic agent + message coordination
- Replace vscode.Webview param with PostMessage in fileWatcher, transcriptParser,
  timerManager, assetLoader
- Replace ExtensionContext param with IRuntimeUI in layoutPersistence
- Replace AgentState.terminalRef with AgentState.handle (IAgentHandle)
- Remove PersistedAgent from types.ts (replaced by PersistedAgentHandle)
- Delete PixelAgentsViewProvider.ts and agentManager.ts (replaced by new modules)
- Slim extension.ts to only wire up the plugin and lifecycle

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The /api/dev-assets endpoint was Vite dev-server only (configureServer hook),
causing 404 in the production static build on Vercel. Added a generateBundle
hook to emit dev-assets.json alongside the JS/CSS artifacts so browser mock
asset loading works in production. Also renamed the dev middleware path to
/dev-assets.json for consistency. Removed broken /vite.svg favicon reference.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@NNTin NNTin changed the base branch from feat/stubbing-vscode-comments to feat/plugin-system-for-runtime-extensibility March 18, 2026 17:09
Adds DBackRuntime — a browser-side plugin that connects to a d-back
WebSocket server (default: wss://hermes.nntin.xyz/dzone) and maps
Discord users to pixel agents in real time.

- src/dback/DBackRuntime.ts: WebSocket client that handles the d-back
  protocol (server-list → connect → server-join, presence, message).
  Each Discord user becomes a pixel agent; presence updates drive agent
  status; chat messages trigger short Write-tool animations.
  Reconnects automatically on disconnect.
- main.tsx: starts DBackRuntime in browser mode alongside asset loading
- .gitignore: ignore the cloned d-back/ directory

The live d-back instance at wss://hermes.nntin.xyz/dzone (D-World server,
id: dworld) is used by default. Override via VITE_DBACK_WS_URL /
VITE_DBACK_SERVER env vars.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@NNTin NNTin force-pushed the feat/integrate-d-back-browser-runtime branch from d6c2fcf to 9593feb Compare March 18, 2026 17:17
@NNTin NNTin changed the base branch from feat/plugin-system-for-runtime-extensibility to main March 18, 2026 17:20
@NNTin
Copy link
Owner Author

NNTin commented Mar 18, 2026

succeeded by #9

@NNTin NNTin closed this Mar 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant