feat: preview pixel agents in web browser (additional support tool for dev and review) by NNTin · Pull Request #143 · pablodelucca/pixel-agents

NNTin · 2026-03-15T18:10:09Z

Description

Browser preview mode for faster asset development and design iteration — run cd webview-ui && npm run dev to see the office in a browser without
launching the VS Code extension.

Originally contributed by @NNTin (reference PR), then extended with:

Browser mock system:

Vite dev server serves pre-decoded asset JSON via middleware endpoints (/assets/decoded/*.json)
browserMock.ts is just a thin wrapper, it fetches pre-decoded JSON and dispatches messages, while all the heavy lifting (PNG decoding, catalog building) lives in the shared modules
Hot-reload on any asset file change (PNGs, manifests, layouts)
Dev-mode debug logging for outbound postMessage calls
build:browser mode for static deployable builds (e.g. Vercel preview)

Shared module layer (shared/assets/):

Extracted pure, VS Code-free modules into shared/assets/ domain directory, preparing the codebase for future standalone backends (Electron, web app)
pngDecoder.ts — PNG Buffer → SpriteData conversion (single implementation, was duplicated across extension + browser mock)
manifestUtils.ts — manifest tree flattening + furniture types
loader.ts — scan directories, read PNGs, return decoded asset collections
build.ts — build furniture catalog + asset index from manifests
types.ts — shared asset pipeline types (AssetIndex, CatalogEntry, CharacterDirectionSprites)
constants.ts — asset parsing constants (single source of truth, was duplicated in src/constants.ts + webview-ui/src/constants.ts)

Docs:

Added browser preview instructions to CONTRIBUTING.md

Direct link to Preview: https://pixel-agents-git-feat-stubbing-vscode-nntins-projects.vercel.app/

Further mocks could be added to spawn and despawn agents and have them make mock messages.

Type of change

Related issues

Faster feedback for sprite/asset editing: Contributing by drawing sprites #135
Lays groundwork for standalone backend: shared modules with zero VS Code dependency

Screenshots / GIFs

Test plan

PR targets main branch
npm run build passes locally
Tested in Extension Development Host (F5) — verify agents spawn, walk, type, and display tools correctly
cd webview-ui && npm run dev — verify browser preview renders office with all assets (characters, floors, walls, furniture)
cd webview-ui && npm run build:browser — verify dist/browser/assets/decoded/ contains characters.json, floors.json, walls.json, furniture.json
Hot-reload works — modify a PNG or manifest, browser auto-refreshes with updated assets

daniel-dallimore · 2026-03-15T21:27:39Z

I think this has already been solved by #120

florintimbuc · 2026-03-16T00:38:31Z

Thanks @NNTin for the efforts! Great addition, is nice to have a browser preview for asset development, I also added hot-reload on asset changes, the Vite dev server now watches public/assets/ and triggers a full reload whenever PNGs or manifests change, so you see updates instantly.

I refactored a bit and added server-side asset decoding and added a shared module layer with zero VS Code dependency, the browser mock (browserMock.ts) is now just a thin wrapper. This also give us the chance to abstract the architecture a bit and lay the groundwork for supporting multiple apps/backends down the road.

florintimbuc · 2026-03-16T00:42:56Z

I think this has already been solved by #120

@daniel-dallimore #120 goes further by adding Express + WebSockets for a full standalone app. This PR is focused on browser-based asset preview to speed up development and design iteration, but a standalone backend is something we're considering soon

florintimbuc

Hey @NNTin Would love to also hear how you'd approach mocking agent spawn/despawn, I would keep it as minimal as possible

Also, would you mind adding a small section to CONTRIBUTING.md on how to run the browser preview? Something short covering:

cd webview-ui && npm run dev to preview locally
npm run build:browser + deploy dist/browser/ to Vercel (or similar) for a shareable preview link, like you did in your demo

Would be really helpful for anyone working on pixel art or layouts to quickly preview and share their work

NNTin · 2026-03-16T11:52:34Z

Vercel is a one time setup for the repo owner.

There are alternatives to Vercel like Cloudflare Pages, Netlify etc. You need to evaluate if Vercel is something you want to use for this project. I've personally switched so a custom self-host solution but that also comes with its own trade-offs. The Vercel plan is pretty generous and I think you won't run into issues there. If you do Allure Test reports with playwright + video artifact you quickly come to its limit. According to ChatGPT Vercel has: 100 deployments per day, 32 builds per hour, build time limit 45 minutes per deployment.

I updated the code to fix the build. On my Vercel dashboard I can see the pipeline was failing in build due to missing dependency and dependency not found.

I need to re-evaluate a bit how I would mock the agents. I'll do this in a different PR / someone else can also take this over.

NNTin · 2026-03-16T17:26:03Z

Put to draft. I am testing something and may optimize the code.

NNTin · 2026-03-16T17:44:11Z

Vercel configuration is changed to:

Last 2 commits improved the code slighly. Previously we had in webview-ui npm run build and npm run build:browser producing 2 different distributions dist/webview and dist/browser. Maintaining 2 different distribution may drift and introduce future headache so this was merged into a single distribution.

Removed an unnecessary file + not introducing a new script in webview-ui/package.json.

florintimbuc · 2026-03-16T20:47:20Z

webview-ui/src/App.tsx

+  // useExtensionMessages listener has been registered.
  useEffect(() => {
-    if (import.meta.env.DEV) {
+    const isBrowserRuntime =


isBrowserRuntime is duplicated in main.tsx and App.tsx with verbose casts and acquireVsCodeApi === 'undefined' is hardcoded to VS Code, which is not future-proof for Cursor, Windsurf, etc. - soon enough we go provider agnostic

My suggestion, create in webview-ui/src/runtime.ts:

/** * Runtime detection, provider-agnostic * * Single source of truth for determining whether the webview is running * inside an IDE extension (VS Code, Cursor, Windsurf, etc.) or standalone * in a browser. */ declare function acquireVsCodeApi(): unknown; export type Runtime = 'vscode' | 'browser'; // Future: 'cursor' | 'windsurf' | 'electron' | etc. export const runtime: Runtime = typeof acquireVsCodeApi !== 'undefined' ? 'vscode' : 'browser'; export const isBrowserRuntime = runtime === 'browser';

Then main.tsx, App.tsx, and vscodeApi.ts all import runtime from one place

created runtime.ts for single source of truth

florintimbuc · 2026-03-16T20:48:16Z

webview-ui/src/main.tsx


 async function main() {
-  if (import.meta.env.DEV) {
+  const isBrowserRuntime =


Implement isBrowserRuntime deduplication mentioned before, please

florintimbuc · 2026-03-16T20:59:46Z

webview-ui/vite.config.ts

+    },
+    // Build output must include decoded JSON so dist/webview can run in plain browsers.
+    closeBundle() {
+      fs.mkdirSync(distAssetsDir, { recursive: true });


Suggested change

fs.mkdirSync(distAssetsDir, { recursive: true });

if (process.env.BROWSER_BUILD !== 'true') return;

fs.mkdirSync(distAssetsDir, { recursive: true });

What do you think about adding a gate on BROWSER_BUILD env var to avoid adding ~1-2MB decoded JSON to dist/webview/ which gets packaged into the .vsix?

and restore build:browser script in webview-ui/package.json like this:
"build:browser": "BROWSER_BUILD=true tsc -b && vite build"

I solved it a bit differently, browserMock was extended, the decoded JSON files is decoded at runtime and don't exist as files. This ensures less magic code in production environment and only in the mock.

florintimbuc · 2026-03-16T21:13:23Z

webview-ui/src/browserMock.ts

+    ]);
+
+  const layout = assetIndex.defaultLayout
+    ? await fetch(`/assets/${assetIndex.defaultLayout}`).then((r) => r.json())


Suggested change

? await fetch(`/assets/${assetIndex.defaultLayout}`).then((r) => r.json())

? await fetch(`${import.meta.env.BASE_URL}assets/${assetIndex.defaultLayout}`).then((r) => r.json())

Good catch. Implemented change and added tests that explicitely tests that frontend serving work from base root and subpath

florintimbuc · 2026-03-16T21:15:26Z

webview-ui/src/browserMock.ts

+      fetch('/assets/asset-index.json').then((r) => r.json()) as Promise<AssetIndex>,
+      fetch('/assets/furniture-catalog.json').then((r) => r.json()) as Promise<CatalogEntry[]>,
+      fetch('/assets/decoded/characters.json').then((r) => r.json()) as Promise<
+        CharacterDirectionSprites[]
+      >,
+      fetch('/assets/decoded/floors.json').then((r) => r.json()) as Promise<string[][][]>,
+      fetch('/assets/decoded/walls.json').then((r) => r.json()) as Promise<string[][][][]>,
+      fetch('/assets/decoded/furniture.json').then((r) => r.json()) as Promise<


Suggested change

fetch('/assets/asset-index.json').then((r) => r.json()) as Promise<AssetIndex>,

fetch('/assets/furniture-catalog.json').then((r) => r.json()) as Promise<CatalogEntry[]>,

fetch('/assets/decoded/characters.json').then((r) => r.json()) as Promise<

CharacterDirectionSprites[]

>,

fetch('/assets/decoded/floors.json').then((r) => r.json()) as Promise<string[][][]>,

fetch('/assets/decoded/walls.json').then((r) => r.json()) as Promise<string[][][][]>,

fetch('/assets/decoded/furniture.json').then((r) => r.json()) as Promise<

fetch(`${import.meta.env.BASE_URL}assets/asset-index.json`).then((r) => r.json()) as Promise<AssetIndex>,

fetch(`${import.meta.env.BASE_URL}assets/furniture-catalog.json`).then((r) => r.json()) as Promise<CatalogEntry[]>,

fetch(`${import.meta.env.BASE_URL}assets/decoded/characters.json`).then((r) => r.json()) as Promise<

CharacterDirectionSprites[]

>,

fetch(`${import.meta.env.BASE_URL}assets/decoded/floors.json`).then((r) => r.json()) as Promise<string[][][]>,

fetch(`${import.meta.env.BASE_URL}assets/decoded/walls.json`).then((r) => r.json()) as Promise<string[][][][]>,

fetch(`${import.meta.env.BASE_URL}assets/decoded/furniture.json`).then((r) => r.json()) as Promise<

Better we get rid of absolute paths here, wdyt?

Good catch. Implemented change and added tests that explicitely tests that frontend serving work from base root and subpath

florintimbuc · 2026-03-16T21:24:34Z

webview-ui/src/vscodeApi.ts


-export const vscode = acquireVsCodeApi();
+export const vscode: { postMessage(msg: unknown): void } =
+  typeof acquireVsCodeApi !== 'undefined'


import isBrowserRuntime defined above instead of using acquireVsCodeApi

NNTin · 2026-03-17T16:51:26Z

Addressed comments, diff to it: https://github.com/NNTin/pixel-agents/pull/6/changes

…ule pngjs

This reverts commit 6cd4f85.

…browser mode

…tests

NNTin · 2026-03-17T17:03:34Z

I rebased the branch since I noticed the git history is pretty linear. A note should be added in Contributing what the merge strategy is here. CI can also enforce conventional commit message and merge strategy

NNTin · 2026-03-17T18:06:31Z

@florintimbuc regarding the message how to mock agent spawning and also sending message, see PR NNTin#7

However this code is AI slop pure and is not meant for production. I use d-back as a mock + as an integration to Discord. There I already get a list of actors -> agents and discord messages are bridged to the frontend as well as presence updates.

It is good to see that in order to create agents, send message and status not much code is involved. Basically getOrCreateAgentId() and run dispatch events. Payload is not overly complicated.

florintimbuc requested changes Mar 16, 2026

View reviewed changes

NNTin marked this pull request as draft March 16, 2026 17:25

NNTin marked this pull request as ready for review March 16, 2026 17:44

NNTin requested a review from florintimbuc March 16, 2026 17:57

florintimbuc requested changes Mar 16, 2026

View reviewed changes

NNTin marked this pull request as draft March 17, 2026 06:57

NNTin force-pushed the feat/stubbing-vscode branch from 7a3c016 to d4b9237 Compare March 17, 2026 16:50

NNTin and others added 16 commits March 17, 2026 17:59

feat: debug mode in browser

6233180

refactor: reduce unnecessary files

af9b24d

fix: browser mock is not injected

4475a4c

feat: serving under root not subpath

f8d464a

refactor: deduplicate manifest types, constants, and browser mock types

e1f9ad0

refactor: move asset pipeline modules to shared/assets/ domain directory

33c63b0

fix: cd webview-ui && npm run build:browser reporting cannot find mod…

7e8e3ba

…ule pngjs

fix: pngDecoder is outside of webview-ui

1ad7e1f

docs: onboard usage of mocked pixel agents

402a79a

refactor: rename shared/assets/decode.ts to loader.ts for clarity

19b496f

revert: this part of the code overcomplicates it and may be unnecessary

95caab1

This reverts commit 6cd4f85.

feat: add missing asset files, differentiate between vscode mode and …

51af938

…browser mode

refactor: single source of truth for runtime detection

f412734

fix: assets being served under root and subpath and added regression …

ef1663a

…tests

fix: no longer bundling character, floor, wall and furniture JSONs

d42e12e

fix: URL were being build wrong for floors, walls and characters

d40e618

NNTin force-pushed the feat/stubbing-vscode branch from 81a13fe to d40e618 Compare March 17, 2026 17:00

NNTin marked this pull request as ready for review March 17, 2026 17:05

florintimbuc approved these changes Mar 17, 2026

View reviewed changes

florintimbuc merged commit b1124e8 into pablodelucca:main Mar 17, 2026
1 check passed

NNTin mentioned this pull request Mar 18, 2026

feat: preview pixel agents in web browser (additional support tool for dev and review) NNTin/pixel-agents#3

Closed

11 tasks

	fs.mkdirSync(distAssetsDir, { recursive: true });
	if (process.env.BROWSER_BUILD !== 'true') return;
	fs.mkdirSync(distAssetsDir, { recursive: true });

	? await fetch(`/assets/${assetIndex.defaultLayout}`).then((r) => r.json())
	? await fetch(`${import.meta.env.BASE_URL}assets/${assetIndex.defaultLayout}`).then((r) => r.json())

Conversation

NNTin commented Mar 15, 2026 • edited by florintimbuc Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Browser mock system:

Shared module layer (shared/assets/):

Type of change

Related issues

Screenshots / GIFs

Test plan

Uh oh!

daniel-dallimore commented Mar 15, 2026

Uh oh!

florintimbuc commented Mar 16, 2026

Uh oh!

florintimbuc commented Mar 16, 2026

Uh oh!

florintimbuc left a comment

Choose a reason for hiding this comment

Uh oh!

NNTin commented Mar 16, 2026

Uh oh!

NNTin commented Mar 16, 2026

Uh oh!

NNTin commented Mar 16, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

NNTin commented Mar 17, 2026

Uh oh!

NNTin commented Mar 17, 2026

Uh oh!

NNTin commented Mar 17, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

NNTin commented Mar 15, 2026 •

edited by florintimbuc

Loading