Skip to content

feat: Playwright e2e tests for VS Code extension#10

Open
NNTin wants to merge 7 commits intomainfrom
feat/playwright-e2e-tests-for-vscode-extension
Open

feat: Playwright e2e tests for VS Code extension#10
NNTin wants to merge 7 commits intomainfrom
feat/playwright-e2e-tests-for-vscode-extension

Conversation

@NNTin
Copy link
Owner

@NNTin NNTin commented Mar 18, 2026

Summary

  • Adds a complete Playwright end-to-end test infrastructure that validates the Pixel Agents extension inside a real VS Code/Electron instance
  • First test: clicking + Agent in the Pixel Agents webview spawns mock claude, creates a JSONL session file, and shows a Claude Code ci: code quality #1 terminal tab
  • No real Claude CLI is required — a deterministic bash shim handles invocation

What's included

File Purpose
e2e/playwright.config.ts Playwright config — video on, trace on retry, 1 worker
e2e/global-setup.ts Downloads VS Code once via @vscode/test-electron, caches to .vscode-test/
e2e/helpers/launch.ts Launches VS Code with --extensionDevelopmentPath, isolated HOME+userDataDir, mock claude in PATH, records video
e2e/helpers/webview.ts Waits for the Pixel Agents webview iframe, clicks + Agent
e2e/fixtures/mock-claude Bash shim — parses --session-id, writes invocation log, creates .jsonl file
e2e/tests/agent-spawn.spec.ts First spec: asserts mock invoked, JSONL created, terminal tab visible
e2e/tsconfig.json Strict TypeScript config for e2e code

How to run

npm run build   # required first — tests load compiled dist/
npm run e2e     # headless (Linux: wraps with xvfb-run)
npm run e2e:headed
npm run e2e:debug

Test result

✓  clicking + Agent spawns mock claude and creates a JSONL session file (24.6s)
1 passed (26.0s)

Video recorded at test-results/e2e/videos/<test-name>/*.webm

Flakiness notes & mitigations

Risk Mitigation
VS Code slow to start 60 s workbench timeout + waitForSelector('.monaco-workbench')
Webview frame not yet attached getPixelAgentsFrame polls all frames every 500 ms for up to 30 s
Terminal tab rendering lag Text locator /Claude Code #\d+/ is resilient to aria-label changes
JSONL not created expect.poll retries for 20 s at 500 ms / 1 s intervals
Display unavailable on Linux xvfb-run --auto-servernum in the e2e script

🤖 Generated with Claude Code

Adds a complete end-to-end test setup using Playwright's Electron API to
validate the Pixel Agents extension inside a real VS Code instance.

- e2e/playwright.config.ts     — config with video, trace, 1-worker isolation
- e2e/global-setup.ts          — downloads VS Code via @vscode/test-electron
- e2e/helpers/launch.ts        — launches VS Code with isolated HOME + mock PATH
- e2e/helpers/webview.ts       — waits for the webview frame, clicks + Agent
- e2e/fixtures/mock-claude     — deterministic claude shim: creates JSONL file,
                                  logs invocation; no real Claude CLI required
- e2e/tests/agent-spawn.spec.ts — first spec: click + Agent → assert mock called,
                                  JSONL session file created, terminal tab visible
- e2e/tsconfig.json            — strict TypeScript config for test code

NPM scripts added: e2e (xvfb-run headless), e2e:headed, e2e:debug
CONTRIBUTING.md updated with running instructions, artifact paths, mock docs
test-results/ and playwright-report/ added to .gitignore

Test passes locally in 25 s; video recording confirmed at
test-results/e2e/videos/<test-name>/*.webm

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@vercel
Copy link

vercel bot commented Mar 18, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
pixel-agents Ready Ready Preview, Comment Mar 18, 2026 8:16pm

@NNTin NNTin force-pushed the feat/playwright-e2e-tests-for-vscode-extension branch from 226e993 to 52096c0 Compare March 18, 2026 19:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant