diff --git a/e2e/BROWSERBASE.md b/e2e/BROWSERBASE.md new file mode 100644 index 0000000..2f8d049 --- /dev/null +++ b/e2e/BROWSERBASE.md @@ -0,0 +1,182 @@ +# Running the web-client smokes on Browserbase + +Optional transport for the browser-based clients — **`claude-web`** (default) +and `chatgpt-web`. Instead of attaching to a real Chrome on the Mac Studio, the +driver connects to a cloud browser, which lets that CI job leave the self-hosted +runner. + +`claude-web` drives claude.ai. It is the browser sibling of `claude-desktop`, +which drives the signed Electron app through Appium and macOS Accessibility +(CDP is fused off) and therefore cannot run anywhere but the Mac Studio. Same +product, two clients, only one of which is portable. + +**Read the caveat at the bottom before investing time.** Whether this works at +all is decided by Cloudflare, not by us. + +## What changes, and what doesn't + +Only the transport. `runSmokeTest.ts`, all 18 smoke tests, the gate manifest, +fixtures, prompt templates and the whole `setup/` scratch layer are untouched — +`setup/` talks to Google/Slack/ClickUp APIs and never opens a browser. + +Transport lives in `connect.ts` and is shared by both web drivers, so adding a +third web client is a selectors-only job and a transport fix lands for every +client at once. Only the site-specific selectors live in each driver. + +`claude-desktop` **cannot** move here, ever. It drives a signed Electron app +through Appium and macOS Accessibility because CDP is fused off; Browserbase +runs browsers, not desktop apps. That job stays on the Mac Studio — and it is +the blocking gate signal, while `chatgpt-web` is `continue-on-error: true`. +So this buys CI decoupling for the advisory half, not gate independence. + +## Setup + +### 1. Account and credentials + +Sign up at , then from **Settings**: + +- `BROWSERBASE_API_KEY` +- `BROWSERBASE_PROJECT_ID` + +The Free tier (1 browser-hour, 3 concurrent sessions) is enough to answer the +Cloudflare question. It is *not* enough to run a gate — see Cost. + +```bash +export BROWSERBASE_API_KEY=bb_live_… +export BROWSERBASE_PROJECT_ID=… +``` + +### 2. Seed a Context with a logged-in session + +Pick the client first — one context per client, since claude.ai and chatgpt.com +cookies are unrelated: + +```bash +export CLIENT=claude-web # default; use chatgpt-web for the other +``` + +A Context is Browserbase's persistent cookie jar. This replaces the warmed +`$HOME/e2e-chrome-profile` the local transport relies on. + +```bash +cd e2e +npm install +npm run seed:browserbase +``` + +The script prints a **Live View URL**. Open it, log in by hand — 2FA included, +since you are driving a real browser — then press Enter in the terminal. It verifies a composer is present before saving, so an incomplete +login is reported rather than silently stored. + +It prints the context id to keep: + +```bash +export BROWSERBASE_CONTEXT_ID=ctx_… +``` + +Confirm the MCP connector is configured **on that account** while you are in +there. Connectors are account-side, not browser-side, so this is once only. + +Re-run the same command whenever ChatGPT expires the session. Pass an existing +`BROWSERBASE_CONTEXT_ID` to refresh in place instead of creating a new one. + +### 3. Run a test + +```bash +cd e2e +E2E_BROWSER=browserbase CLIENT=claude-web \ + node --import tsx --test tests/readGoogleDoc.smoke.ts +``` + +`CLIENT` selects the driver; the tests themselves are client-agnostic. + +On `main` there is exactly one smoke test, at that path. The 18-test suite under +`tests/read/` and `tests/write/` arrives with PR #45 — until that merges, the +paths in the Cost section below describe the future state, not this branch. + +Each run logs a session id and a replay URL. The replay is a full video of the +run — considerably better than the local harness's screenshot when a selector +drifts. + +Anything other than `E2E_BROWSER=browserbase` keeps the existing local-Chrome +behaviour, so the Mac Studio path is unaffected by default. + +### 4. CI (once the spike passes) + +In `.github/workflows/e2e-smoke.yml`, the `chatgpt-web` job becomes: + +```yaml +runs-on: ubuntu-latest # was: [self-hosted, macOS, mac-studio] +env: + CLIENT: claude-web + E2E_BROWSER: browserbase + BROWSERBASE_API_KEY: ${{ secrets.BROWSERBASE_API_KEY }} + BROWSERBASE_PROJECT_ID: ${{ secrets.BROWSERBASE_PROJECT_ID }} + BROWSERBASE_CONTEXT_ID: ${{ secrets.BROWSERBASE_CONTEXT_ID }} + # CHATGPT_CDP_ENDPOINT no longer needed +``` + +Leave `claude-desktop` on the Mac Studio. + +## Two design decisions worth knowing + +**Tests run with `persist: false`.** Node's test runner parallelises across +files, so a gate run opens ~18 sessions at once. If each wrote its cookie jar +back to the shared context on close they would race, and whichever finished last +would define everyone's auth state. Tests read the seeded auth and write +nothing. Only `seed:browserbase` uses `persist: true`, and it runs alone. + +**`dispose()` means two different things.** Locally it only disconnects, leaving +Chrome warm for the next run. On Browserbase it *ends the session*, which is +what stops it billing. Same call, opposite intent. + +## Cost + +18 smoke tests, one session each, roughly 2 min per session ≈ **36 +browser-minutes per gate run**. + +| Plan | Price | Browser hours | Concurrent | +|---|---|---|---| +| Free | $0 | 1 | 3 | +| Developer | $20/mo | 100 | 25 | +| Startup | $99/mo | 500 | 100 | +| Scale | custom | flexible | 250+ | + +Developer covers ~165 gate runs/month and its 25 concurrent sessions clear the +18 a parallel gate needs. Free cannot run a gate — 3 concurrent — but is fine +for a single test. + +## Selector status + +**Every selector in `claude-web.ts` is unverified against a live claude.ai +session.** They are written with ordered fallbacks, and completion is detected +by watching the reply text stop changing rather than by a stop-button selector — +so a DOM rename produces a named error listing what was tried, not a silent +wrong answer. Watch the first run; the replay URL logged at session start is +exactly the tool for it. + +`chatgpt-web.ts`'s selectors carry the same SELECTOR-TODO caveat and predate +this work. + +## The caveat that decides this + +`chatgpt-web.ts` opens with: + +> Chrome is launched outside Playwright with a warmed persistent profile so +> Cloudflare doesn't see Playwright's bundled Chromium fingerprint or a +> webdriver flag set by Playwright. + +That is exactly the fight a cloud browser re-opens — for ChatGPT. Whether +claude.ai is as aggressive is unknown and worth measuring separately; it is a +different site with different protections. Browserbase's advanced +anti-detection ("Verified" identity) is **Scale plan only**; Developer and +Startup get basic identity plus automatic captcha solving. + +So the outcomes are: works on $20/mo, works only at Scale pricing, or does not +work reliably at all. **Spike it on the Free tier before committing** — seed a +context, run one read smoke, and you have a binary answer for an hour's work and +no spend. + +Separately: automating `chatgpt.com` in a cloud browser is worth checking +against OpenAI's terms of service. That is a judgement call for whoever owns the +account, not a technical blocker. diff --git a/e2e/drivers/browserbase.ts b/e2e/drivers/browserbase.ts new file mode 100644 index 0000000..b31d2d0 --- /dev/null +++ b/e2e/drivers/browserbase.ts @@ -0,0 +1,88 @@ +// Opens a Browserbase cloud browser and hands back a CDP endpoint. +// +// This module is deliberately ONLY the connection. Everything about driving +// ChatGPT — selectors, the streaming-complete heuristic, response extraction — +// stays in chatgpt-web.ts and is shared by both transports. A forked driver +// would drift the moment ChatGPT changed its DOM, and the whole point of the +// SELECTOR-TODO comments there is that they change often. +// +// Why this exists: the local transport attaches to a real Chrome started +// outside Playwright with a warmed profile, which pins the ChatGPT job to the +// self-hosted Mac Studio. A cloud browser lets that job run on ubuntu-latest. +// See BROWSERBASE.md for setup and for the Cloudflare caveat, which is the +// thing most likely to decide whether this is usable at all. + +import Browserbase from '@browserbasehq/sdk'; + +export interface BrowserbaseSession { + /** Pass to chromium.connectOverCDP(). */ + connectUrl: string; + sessionId: string; + /** Dashboard replay of the whole run — worth attaching to a failure bundle. */ + replayUrl: string; +} + +function required(name: string): string { + const value = process.env[name]; + if (!value) { + throw new Error( + `${name} is not set. Browserbase transport needs BROWSERBASE_API_KEY, ` + + 'BROWSERBASE_PROJECT_ID and BROWSERBASE_CONTEXT_ID — see e2e/BROWSERBASE.md.', + ); + } + return value; +} + +/** True when the harness has been asked to run against a cloud browser. */ +export function usingBrowserbase(): boolean { + return (process.env.E2E_BROWSER ?? '').toLowerCase() === 'browserbase'; +} + +export function browserbaseClient(): Browserbase { + return new Browserbase({ apiKey: required('BROWSERBASE_API_KEY') }); +} + +/** + * Start a session carrying the seeded ChatGPT login. + * + * `persist` defaults to FALSE, and that default is load-bearing. Node's test + * runner parallelises across files, so a gate run opens ~18 sessions at once; + * if each wrote its cookie jar back to the shared context on close, they would + * race and the last one to finish would define everyone's auth state. Tests + * read the seeded auth and write nothing. Only the seeding script sets + * persist: true, and it runs alone. + */ +export async function createBrowserbaseSession( + opts: { persist?: boolean; timeoutSeconds?: number; keepAlive?: boolean } = {}, +): Promise { + const bb = browserbaseClient(); + const projectId = required('BROWSERBASE_PROJECT_ID'); + const contextId = required('BROWSERBASE_CONTEXT_ID'); + + const session = await bb.sessions.create({ + projectId, + browserSettings: { + context: { id: contextId, persist: opts.persist ?? false }, + }, + // `api_timeout`, not `timeout` — the Node SDK carries the Python parameter + // name here (SessionCreateParams in @browserbasehq/sdk), and `timeout` is + // silently rejected as an unknown property. Bounded so a hung ChatGPT + // stream cannot burn browser-minutes up to the 6h ceiling; comfortably + // above RESPONSE_TIMEOUT_MS (120s). + api_timeout: opts.timeoutSeconds ?? 300, + ...(opts.keepAlive ? { keepAlive: true } : {}), + }); + + return { + connectUrl: session.connectUrl, + sessionId: session.id, + replayUrl: `https://browserbase.com/sessions/${session.id}`, + }; +} + +/** Live View URL — a human can watch, or drive, a running session from here. */ +export async function liveViewUrl(sessionId: string): Promise { + const bb = browserbaseClient(); + const links = await bb.sessions.debug(sessionId); + return links.debuggerFullscreenUrl; +} diff --git a/e2e/drivers/chatgpt-web.ts b/e2e/drivers/chatgpt-web.ts index 0818a12..4f81cab 100644 --- a/e2e/drivers/chatgpt-web.ts +++ b/e2e/drivers/chatgpt-web.ts @@ -14,16 +14,22 @@ // app refresh; surface failures clearly in forensics rather than silently // matching the wrong element. -import { chromium, type Browser, type Page } from 'playwright'; +import { type Page } from 'playwright'; import type { Driver } from './driver.ts'; +import { connectBrowser } from './connect.ts'; const CDP_ENDPOINT = process.env.CHATGPT_CDP_ENDPOINT ?? 'http://127.0.0.1:9222'; const CHATGPT_URL = process.env.CHATGPT_URL ?? 'https://chatgpt.com/'; const RESPONSE_TIMEOUT_MS = Number(process.env.RESPONSE_TIMEOUT_MS ?? 120_000); export async function createChatGptWebDriver(): Promise { - const browser: Browser = await chromium.connectOverCDP(CDP_ENDPOINT); - const context = browser.contexts()[0] ?? (await browser.newContext()); + // Only the TRANSPORT differs between local Chrome and a Browserbase cloud + // browser — everything below is shared, so the selectors that change most + // often live in exactly one place. E2E_BROWSER=browserbase picks the cloud + // path; anything else keeps the local warmed-profile behaviour that is the + // default on the Mac Studio. + const conn = await connectBrowser(CDP_ENDPOINT); + const context = conn.browser.contexts()[0] ?? (await conn.browser.newContext()); let page: Page = context.pages()[0] ?? (await context.newPage()); if (!page.url().includes('chatgpt.com')) { @@ -64,14 +70,19 @@ export async function createChatGptWebDriver(): Promise { async appVersion() { const ua = await page.evaluate(() => navigator.userAgent); - return `chatgpt-web userAgent=${ua}`; + return `chatgpt-web ${conn.describe()} userAgent=${ua}`; }, async dispose() { - // Chrome is a long-lived process started outside Playwright. Disconnect - // without closing the underlying browser so the next run reuses the - // warmed profile. - await browser.close(); + // The same call means two different things, which is worth stating + // rather than inferring: + // - local: Chrome is a long-lived process started outside Playwright, + // so this only DISCONNECTS and the warmed profile survives for the + // next run. + // - browserbase: this ENDS the cloud session, which is what stops it + // billing. Leaving it open would burn browser-minutes until the + // session timeout expires. + await conn.browser.close(); }, }; } diff --git a/e2e/drivers/claude-web.ts b/e2e/drivers/claude-web.ts new file mode 100644 index 0000000..530d79c --- /dev/null +++ b/e2e/drivers/claude-web.ts @@ -0,0 +1,197 @@ +// Drives Claude on the web (claude.ai) — the browser sibling of the +// claude-desktop driver, and the only Claude client that can run on a cloud +// browser. claude-desktop drives a signed Electron app through Appium and +// macOS Accessibility because CDP is fused off, which pins it to the Mac +// Studio; this one runs anywhere Playwright can reach a Chrome, including +// Browserbase. +// +// Transport (local Chrome vs Browserbase) is shared with chatgpt-web via +// connect.ts. Only the selectors below are Claude-specific. +// +// SELECTOR-TODO — EVERY selector in this file is UNVERIFIED against a live +// claude.ai session. They are written with fallbacks and a DOM-independent +// completion heuristic so they degrade into a clear error rather than a silent +// wrong answer, but the first run must be watched. With Browserbase that is +// easy: each run logs a replay URL, and `bb.sessions.debug()` gives a Live +// View you can drive by hand while inspecting the DOM. + +import type { Page } from 'playwright'; +import type { Driver } from './driver.ts'; +import { connectBrowser } from './connect.ts'; + +const CDP_ENDPOINT = process.env.CLAUDE_CDP_ENDPOINT ?? 'http://127.0.0.1:9222'; +const CLAUDE_URL = process.env.CLAUDE_URL ?? 'https://claude.ai/new'; +const RESPONSE_TIMEOUT_MS = Number(process.env.RESPONSE_TIMEOUT_MS ?? 120_000); + +/** How long the reply text must stop changing before it counts as finished. */ +const SETTLE_MS = Number(process.env.CLAUDE_SETTLE_MS ?? 2_500); +const POLL_MS = 500; + +// Ordered by how specific they are. The first one that matches anything wins, +// so a future rename only needs a new entry at the front. +// SELECTOR-TODO: confirm against a live DOM. +const COMPOSER_SELECTORS = [ + 'div[contenteditable="true"].ProseMirror', + '[data-testid="chat-input"]', + 'div[contenteditable="true"]', +]; + +// SELECTOR-TODO: confirm. `.font-claude-message` has been the assistant-turn +// marker historically; the data-testid variants are defensive. +const ASSISTANT_SELECTORS = [ + '[data-testid="chat-message-content"]', + '.font-claude-message', + '[data-is-streaming] .font-claude-message', +]; + +export async function createClaudeWebDriver(): Promise { + const conn = await connectBrowser(CDP_ENDPOINT); + const context = conn.browser.contexts()[0] ?? (await conn.browser.newContext()); + let page: Page = context.pages()[0] ?? (await context.newPage()); + + if (!page.url().includes('claude.ai')) { + await page.goto(CLAUDE_URL, { waitUntil: 'domcontentloaded' }); + } + + return { + async newConversation() { + await page.goto(CLAUDE_URL, { waitUntil: 'domcontentloaded' }); + await firstMatching(page, COMPOSER_SELECTORS, 'composer', 30_000); + }, + + async sendAndWait(prompt) { + const composer = await firstMatching(page, COMPOSER_SELECTORS, 'composer', 30_000); + await composer.click(); + + // Claude's composer is a ProseMirror contenteditable, not a textarea, so + // fill() is unreliable — type through the keyboard instead. The prompts + // are multi-line and Enter SENDS, so newlines go in as Shift+Enter or the + // first line would submit on its own. + const lines = prompt.split('\n'); + for (const [i, line] of lines.entries()) { + if (i > 0) await page.keyboard.press('Shift+Enter'); + await page.keyboard.type(line); + } + await page.keyboard.press('Enter'); + + return waitForResponseComplete(page); + }, + + async captureAccessibilitySnapshot() { + // Same as chatgpt-web: page.accessibility was removed in Playwright 1.49, + // so the full HTML is what forensics gets. + return page.content(); + }, + + async captureScreenshot() { + return page.screenshot({ fullPage: true }); + }, + + async appVersion() { + const ua = await page.evaluate(() => navigator.userAgent); + return `claude-web ${conn.describe()} userAgent=${ua}`; + }, + + async dispose() { + // Local: only disconnects, leaving Chrome warm for the next run. + // Browserbase: ENDS the session, which is what stops it billing. + await conn.browser.close(); + }, + }; +} + +/** + * Resolve the first selector in `candidates` that actually matches. + * + * Trying them in order and reporting all of them on failure is what turns a + * DOM rename into "none of these matched, go look" instead of a bare Playwright + * timeout naming one selector that was only ever a guess. + */ +async function firstMatching( + page: Page, + candidates: string[], + label: string, + timeoutMs: number, +) { + const deadline = Date.now() + timeoutMs; + while (Date.now() < deadline) { + for (const selector of candidates) { + const locator = page.locator(selector).first(); + if (await locator.count().then((n) => n > 0).catch(() => false)) { + if (await locator.isVisible().catch(() => false)) return locator; + } + } + await page.waitForTimeout(POLL_MS); + } + // Naming the page is what separates "the selector is wrong" from "this is a + // sign-in page and no composer exists" — the first run hit the latter and the + // error blamed the selectors. + const url = page.url(); + const title = await page.title().catch(() => '(unknown)'); + const signedOut = /sign in|log in|sign up/i.test(title); + + const detail = signedOut + ? `The page is "${title}" (${url}) — this session is NOT LOGGED IN, so no ${label} exists. ` + + 'The selectors are probably fine. Re-run `npm run seed:browserbase` to save a working ' + + 'login into the Browserbase context, and note it now verifies the context afterwards.' + : `The page is "${title}" (${url}). Tried: ${candidates.join(', ')}. These are ` + + 'SELECTOR-TODO guesses — inspect the live DOM (Browserbase Live View or the replay ' + + 'URL logged at session start) and update claude-web.ts.'; + + throw new Error(`claude-web: no ${label} found after ${timeoutMs}ms. ${detail}`); +} + +/** + * Wait for the reply to finish by watching the text stop changing. + * + * Deliberately NOT keyed on a stop-button selector the way chatgpt-web is. + * That selector is the single most brittle thing in the ChatGPT driver, and + * every selector here is unverified, so a heuristic that only needs to find + * the assistant turn at all is a better bet: text that has not changed for + * SETTLE_MS is done, whatever the button markup does next. + * + * The trade-off is honest — a long pause mid-stream could settle early. Hence + * SETTLE_MS is generous and tunable, and it still refuses to return empty. + */ +async function waitForResponseComplete(page: Page): Promise { + const deadline = Date.now() + RESPONSE_TIMEOUT_MS; + let lastText = ''; + let stableSince = 0; + + while (Date.now() < deadline) { + const text = await lastAssistantText(page); + + if (text && text === lastText) { + if (stableSince === 0) stableSince = Date.now(); + if (Date.now() - stableSince >= SETTLE_MS) return text; + } else { + lastText = text; + stableSince = 0; + } + + await page.waitForTimeout(POLL_MS); + } + + if (lastText) { + // Timed out mid-stream: returning what we have lets the assertion say what + // was actually wrong, instead of masking it as a timeout. + console.error(`[e2e] claude-web: response still changing after ${RESPONSE_TIMEOUT_MS}ms — asserting on partial text`); + return lastText; + } + throw new Error( + `claude-web: no assistant message after ${RESPONSE_TIMEOUT_MS}ms. Tried: ${ASSISTANT_SELECTORS.join(', ')}. ` + + 'Either the reply never arrived or these SELECTOR-TODO guesses are wrong — check the session replay.', + ); +} + +async function lastAssistantText(page: Page): Promise { + for (const selector of ASSISTANT_SELECTORS) { + const messages = page.locator(selector); + const count = await messages.count().catch(() => 0); + if (count > 0) { + const text = await messages.nth(count - 1).innerText().catch(() => ''); + if (text.trim()) return text; + } + } + return ''; +} diff --git a/e2e/drivers/connect.ts b/e2e/drivers/connect.ts new file mode 100644 index 0000000..7d5818b --- /dev/null +++ b/e2e/drivers/connect.ts @@ -0,0 +1,32 @@ +// Shared browser transport for the web-based drivers (chatgpt-web, claude-web). +// +// Both connect the same two ways — a local Chrome over CDP, or a Browserbase +// cloud session — and only the site they drive differs. Keeping the switch here +// means adding a third web client is a selectors-only job, and that a fix to +// the transport lands for every client at once. + +import { chromium, type Browser } from 'playwright'; +import { createBrowserbaseSession, usingBrowserbase, type BrowserbaseSession } from './browserbase.ts'; + +export interface Connection { + browser: Browser; + /** Null when running against a local Chrome. */ + remote: BrowserbaseSession | null; + /** For appVersion() / forensics, so a failure says where it ran. */ + describe(): string; +} + +export async function connectBrowser(localCdpEndpoint: string): Promise { + const remote = usingBrowserbase() ? await createBrowserbaseSession() : null; + if (remote) { + console.error(`[e2e] browserbase session ${remote.sessionId} — replay: ${remote.replayUrl}`); + } + + const browser = await chromium.connectOverCDP(remote ? remote.connectUrl : localCdpEndpoint); + + return { + browser, + remote, + describe: () => (remote ? `browserbase session=${remote.sessionId}` : 'local-cdp'), + }; +} diff --git a/e2e/drivers/driver.ts b/e2e/drivers/driver.ts index c5548c2..0e1b3ea 100644 --- a/e2e/drivers/driver.ts +++ b/e2e/drivers/driver.ts @@ -1,4 +1,7 @@ -export type ClientName = 'claude-desktop' | 'chatgpt-web'; +// claude-desktop drives the signed Electron app via Appium/macOS Accessibility +// (CDP is fused off) and therefore only runs on the Mac Studio. claude-web and +// chatgpt-web are Playwright over CDP and run anywhere, including Browserbase. +export type ClientName = 'claude-desktop' | 'claude-web' | 'chatgpt-web'; export interface Driver { newConversation(): Promise; diff --git a/e2e/package-lock.json b/e2e/package-lock.json index 40f969b..d4f30e2 100644 --- a/e2e/package-lock.json +++ b/e2e/package-lock.json @@ -8,6 +8,7 @@ "name": "google-docs-mcp-e2e", "version": "0.0.0", "dependencies": { + "@browserbasehq/sdk": "^2.19.0", "playwright": "^1.49.0", "webdriverio": "^9.5.0" }, @@ -17,6 +18,36 @@ "typescript": "^5.8.3" } }, + "node_modules/@browserbasehq/sdk": { + "version": "2.19.0", + "resolved": "https://registry.npmjs.org/@browserbasehq/sdk/-/sdk-2.19.0.tgz", + "integrity": "sha512-Fiehnkt0q9ItPnkVyi16fQLNtzUMi5KsAFYLzLcDXKWQV6kXhBNVHOa+v9BqqzSxHIS2Nbzk4E1XRKvphLh1eA==", + "license": "Apache-2.0", + "dependencies": { + "@types/node": "^18.11.18", + "@types/node-fetch": "^2.6.4", + "abort-controller": "^3.0.0", + "agentkeepalive": "^4.2.1", + "form-data-encoder": "1.7.2", + "formdata-node": "^4.3.2", + "node-fetch": "^2.6.7" + } + }, + "node_modules/@browserbasehq/sdk/node_modules/@types/node": { + "version": "18.19.130", + "resolved": "https://registry.npmjs.org/@types/node/-/node-18.19.130.tgz", + "integrity": "sha512-GRaXQx6jGfL8sKfaIDD6OupbIHBr9jv7Jnaml9tB7l4v068PAOXqfcujMMo5PhbIs6ggR1XODELqahT2R8v0fg==", + "license": "MIT", + "dependencies": { + "undici-types": "~5.26.4" + } + }, + "node_modules/@browserbasehq/sdk/node_modules/undici-types": { + "version": "5.26.5", + "resolved": "https://registry.npmjs.org/undici-types/-/undici-types-5.26.5.tgz", + "integrity": "sha512-JlCMO+ehdEIKqlFxk6IfVoAUVmgz7cU7zD/h9XZ0qzeosSHmUJVOzSQvvYSYWXkFXC+IfLKSIffhv0sVZup6pA==", + "license": "MIT" + }, "node_modules/@esbuild/aix-ppc64": { "version": "0.28.0", "resolved": "https://registry.npmjs.org/@esbuild/aix-ppc64/-/aix-ppc64-0.28.0.tgz", @@ -553,6 +584,16 @@ "undici-types": "~6.21.0" } }, + "node_modules/@types/node-fetch": { + "version": "2.6.13", + "resolved": "https://registry.npmjs.org/@types/node-fetch/-/node-fetch-2.6.13.tgz", + "integrity": "sha512-QGpRVpzSaUs30JBSGPjOg4Uveu384erbHBoT1zeONvyCfwQxIkUshLAOqN/k9EjGviPRmWTTe6aH2qySWKTVSw==", + "license": "MIT", + "dependencies": { + "@types/node": "*", + "form-data": "^4.0.4" + } + }, "node_modules/@types/sinonjs__fake-timers": { "version": "8.1.5", "resolved": "https://registry.npmjs.org/@types/sinonjs__fake-timers/-/sinonjs__fake-timers-8.1.5.tgz", @@ -723,6 +764,18 @@ "node": ">= 14" } }, + "node_modules/agentkeepalive": { + "version": "4.6.0", + "resolved": "https://registry.npmjs.org/agentkeepalive/-/agentkeepalive-4.6.0.tgz", + "integrity": "sha512-kja8j7PjmncONqaTsB8fQ+wE2mSU2DJ9D4XKoJ5PFWIdRMa6SLSN1ff4mOr4jCbfRSsxR4keIiySJU0N9T5hIQ==", + "license": "MIT", + "dependencies": { + "humanize-ms": "^1.2.1" + }, + "engines": { + "node": ">= 8.0.0" + } + }, "node_modules/ansi-regex": { "version": "6.2.2", "resolved": "https://registry.npmjs.org/ansi-regex/-/ansi-regex-6.2.2.tgz", @@ -810,6 +863,12 @@ "integrity": "sha512-htCUDlxyyCLMgaM3xXg0C0LW2xqfuQ6p05pCEIsXuyQ+a1koYKTuBMzRNwmybfLgvJDMd0r1LTn4+E0Ti6C2AA==", "license": "MIT" }, + "node_modules/asynckit": { + "version": "0.4.0", + "resolved": "https://registry.npmjs.org/asynckit/-/asynckit-0.4.0.tgz", + "integrity": "sha512-Oei9OH4tRh0YqU3GxhX79dM/mwVgvbZJaSNaRk+bshkj0S5cfHcgYakreBjrHwatXKbz+IoIdYLxrKim2MjW0Q==", + "license": "MIT" + }, "node_modules/b4a": { "version": "1.8.1", "resolved": "https://registry.npmjs.org/b4a/-/b4a-1.8.1.tgz", @@ -998,6 +1057,19 @@ "node": ">=8.0.0" } }, + "node_modules/call-bind-apply-helpers": { + "version": "1.0.2", + "resolved": "https://registry.npmjs.org/call-bind-apply-helpers/-/call-bind-apply-helpers-1.0.2.tgz", + "integrity": "sha512-Sp1ablJ0ivDkSzjcaJdxEunN5/XvksFJ2sMBFfq6x0ryhQV/2b/KwFe21cMpmHtPOSij8K99/wSfoEuTObmuMQ==", + "license": "MIT", + "dependencies": { + "es-errors": "^1.3.0", + "function-bind": "^1.1.2" + }, + "engines": { + "node": ">= 0.4" + } + }, "node_modules/chalk": { "version": "5.6.2", "resolved": "https://registry.npmjs.org/chalk/-/chalk-5.6.2.tgz", @@ -1157,6 +1229,18 @@ "integrity": "sha512-dOy+3AuW3a2wNbZHIuMZpTcgjGuLU/uBL/ubcZF9OXbDo8ff4O8yVp5Bf0efS8uEoYo5q4Fx7dY9OgQGXgAsQA==", "license": "MIT" }, + "node_modules/combined-stream": { + "version": "1.0.8", + "resolved": "https://registry.npmjs.org/combined-stream/-/combined-stream-1.0.8.tgz", + "integrity": "sha512-FQN4MRfuJeHf7cBbBMJFXhKSDq+2kAArBlmRBvcvFE5BB1HZKXtSFASDhdlz9zOYwxh8lDdnvmMOe/+5cdoEdg==", + "license": "MIT", + "dependencies": { + "delayed-stream": "~1.0.0" + }, + "engines": { + "node": ">= 0.8" + } + }, "node_modules/commander": { "version": "9.5.0", "resolved": "https://registry.npmjs.org/commander/-/commander-9.5.0.tgz", @@ -1348,6 +1432,15 @@ "node": ">= 14" } }, + "node_modules/delayed-stream": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/delayed-stream/-/delayed-stream-1.0.0.tgz", + "integrity": "sha512-ZySD7Nf91aLB0RxL4KGrKHBXl7Eds1DAmEdcoVawXnLD7SDhpNgtuII2aAkg7a7QS41jxPSZ17p4VdGnMHk3MQ==", + "license": "MIT", + "engines": { + "node": ">=0.4.0" + } + }, "node_modules/dom-serializer": { "version": "2.0.0", "resolved": "https://registry.npmjs.org/dom-serializer/-/dom-serializer-2.0.0.tgz", @@ -1403,6 +1496,20 @@ "url": "https://github.com/fb55/domutils?sponsor=1" } }, + "node_modules/dunder-proto": { + "version": "1.0.1", + "resolved": "https://registry.npmjs.org/dunder-proto/-/dunder-proto-1.0.1.tgz", + "integrity": "sha512-KIN/nDJBQRcXw0MLVhZE9iQHmG68qAVIBg9CqmUYjmQIhgij9U5MFvrqkUL5FbtyyzZuOeOt0zdeRe4UY7ct+A==", + "license": "MIT", + "dependencies": { + "call-bind-apply-helpers": "^1.0.1", + "es-errors": "^1.3.0", + "gopd": "^1.2.0" + }, + "engines": { + "node": ">= 0.4" + } + }, "node_modules/eastasianwidth": { "version": "0.2.0", "resolved": "https://registry.npmjs.org/eastasianwidth/-/eastasianwidth-0.2.0.tgz", @@ -1509,6 +1616,51 @@ "url": "https://github.com/fb55/entities?sponsor=1" } }, + "node_modules/es-define-property": { + "version": "1.0.1", + "resolved": "https://registry.npmjs.org/es-define-property/-/es-define-property-1.0.1.tgz", + "integrity": "sha512-e3nRfgfUZ4rNGL232gUgX06QNyyez04KdjFrF+LTRoOXmrOgFKDg4BCdsjW8EnT69eqdYGmRpJwiPVYNrCaW3g==", + "license": "MIT", + "engines": { + "node": ">= 0.4" + } + }, + "node_modules/es-errors": { + "version": "1.3.0", + "resolved": "https://registry.npmjs.org/es-errors/-/es-errors-1.3.0.tgz", + "integrity": "sha512-Zf5H2Kxt2xjTvbJvP2ZWLEICxA6j+hAmMzIlypy4xcBg1vKVnx89Wy0GbS+kf5cwCVFFzdCFh2XSCFNULS6csw==", + "license": "MIT", + "engines": { + "node": ">= 0.4" + } + }, + "node_modules/es-object-atoms": { + "version": "1.1.2", + "resolved": "https://registry.npmjs.org/es-object-atoms/-/es-object-atoms-1.1.2.tgz", + "integrity": "sha512-HWcBoN6NileqtSydK2FqHbS/LoDd2pqrnQHLyJzBj4kOp/ky2MWMN694xOfkK8/SnUsW2DH7EfyVlydKCsm1Zw==", + "license": "MIT", + "dependencies": { + "es-errors": "^1.3.0" + }, + "engines": { + "node": ">= 0.4" + } + }, + "node_modules/es-set-tostringtag": { + "version": "2.1.0", + "resolved": "https://registry.npmjs.org/es-set-tostringtag/-/es-set-tostringtag-2.1.0.tgz", + "integrity": "sha512-j6vWzfrGVfyXxge+O0x5sh6cvxAog0a/4Rdd2K36zCMV5eJ+/+tOAngRO8cODMNWbVRdVlmGZQL2YS3yR8bIUA==", + "license": "MIT", + "dependencies": { + "es-errors": "^1.3.0", + "get-intrinsic": "^1.2.6", + "has-tostringtag": "^1.0.2", + "hasown": "^2.0.2" + }, + "engines": { + "node": ">= 0.4" + } + }, "node_modules/esbuild": { "version": "0.28.0", "resolved": "https://registry.npmjs.org/esbuild/-/esbuild-0.28.0.tgz", @@ -1734,6 +1886,41 @@ "url": "https://github.com/sponsors/isaacs" } }, + "node_modules/form-data": { + "version": "4.0.6", + "resolved": "https://registry.npmjs.org/form-data/-/form-data-4.0.6.tgz", + "integrity": "sha512-vKatAh4SlVfgbv+YtmhiRjhEMJsYpsG1Y2rMQtR+SVSbytsSD1YGzDIcrAJmdFec88u/+VoGmxnl+80gL1tRCQ==", + "license": "MIT", + "dependencies": { + "asynckit": "^0.4.0", + "combined-stream": "^1.0.8", + "es-set-tostringtag": "^2.1.0", + "hasown": "^2.0.4", + "mime-types": "^2.1.35" + }, + "engines": { + "node": ">= 6" + } + }, + "node_modules/form-data-encoder": { + "version": "1.7.2", + "resolved": "https://registry.npmjs.org/form-data-encoder/-/form-data-encoder-1.7.2.tgz", + "integrity": "sha512-qfqtYan3rxrnCk1VYaA4H+Ms9xdpPqvLZa6xmMgFvhO32x7/3J/ExcTd6qpxM0vH2GdMI+poehyBZvqfMTto8A==", + "license": "MIT" + }, + "node_modules/formdata-node": { + "version": "4.4.1", + "resolved": "https://registry.npmjs.org/formdata-node/-/formdata-node-4.4.1.tgz", + "integrity": "sha512-0iirZp3uVDjVGt9p49aTaqjk84TrglENEDuqfdlZQ1roC9CWlPk6Avf8EEnZNcAqPonwkG35x4n3ww/1THYAeQ==", + "license": "MIT", + "dependencies": { + "node-domexception": "1.0.0", + "web-streams-polyfill": "4.0.0-beta.3" + }, + "engines": { + "node": ">= 12.20" + } + }, "node_modules/fsevents": { "version": "2.3.2", "resolved": "https://registry.npmjs.org/fsevents/-/fsevents-2.3.2.tgz", @@ -1748,6 +1935,15 @@ "node": "^8.16.0 || ^10.6.0 || >=11.0.0" } }, + "node_modules/function-bind": { + "version": "1.1.2", + "resolved": "https://registry.npmjs.org/function-bind/-/function-bind-1.1.2.tgz", + "integrity": "sha512-7XHNxH7qX9xG5mIwxkhumTox/MIRNcOgDrxWsMt2pAr23WHp6MrRlN7FBSFpCpr+oVO0F744iUgR82nJMfG2SA==", + "license": "MIT", + "funding": { + "url": "https://github.com/sponsors/ljharb" + } + }, "node_modules/geckodriver": { "version": "6.1.0", "resolved": "https://registry.npmjs.org/geckodriver/-/geckodriver-6.1.0.tgz", @@ -1778,6 +1974,30 @@ "node": "6.* || 8.* || >= 10.*" } }, + "node_modules/get-intrinsic": { + "version": "1.3.0", + "resolved": "https://registry.npmjs.org/get-intrinsic/-/get-intrinsic-1.3.0.tgz", + "integrity": "sha512-9fSjSaos/fRIVIp+xSJlE6lfwhES7LNtKaCBIamHsjr2na1BiABJPo0mOjjz8GJDURarmCPGqaiVg5mfjb98CQ==", + "license": "MIT", + "dependencies": { + "call-bind-apply-helpers": "^1.0.2", + "es-define-property": "^1.0.1", + "es-errors": "^1.3.0", + "es-object-atoms": "^1.1.1", + "function-bind": "^1.1.2", + "get-proto": "^1.0.1", + "gopd": "^1.2.0", + "has-symbols": "^1.1.0", + "hasown": "^2.0.2", + "math-intrinsics": "^1.1.0" + }, + "engines": { + "node": ">= 0.4" + }, + "funding": { + "url": "https://github.com/sponsors/ljharb" + } + }, "node_modules/get-port": { "version": "7.2.0", "resolved": "https://registry.npmjs.org/get-port/-/get-port-7.2.0.tgz", @@ -1790,6 +2010,19 @@ "url": "https://github.com/sponsors/sindresorhus" } }, + "node_modules/get-proto": { + "version": "1.0.1", + "resolved": "https://registry.npmjs.org/get-proto/-/get-proto-1.0.1.tgz", + "integrity": "sha512-sTSfBjoXBp89JvIKIefqw7U2CCebsc74kiY6awiGogKtoSGbgjYE/G/+l9sF3MWFPNc9IcoOC4ODfKHfxFmp0g==", + "license": "MIT", + "dependencies": { + "dunder-proto": "^1.0.1", + "es-object-atoms": "^1.0.0" + }, + "engines": { + "node": ">= 0.4" + } + }, "node_modules/get-stream": { "version": "5.2.0", "resolved": "https://registry.npmjs.org/get-stream/-/get-stream-5.2.0.tgz", @@ -1840,6 +2073,18 @@ "url": "https://github.com/sponsors/isaacs" } }, + "node_modules/gopd": { + "version": "1.2.0", + "resolved": "https://registry.npmjs.org/gopd/-/gopd-1.2.0.tgz", + "integrity": "sha512-ZUKRh6/kUFoAiTAtTYPZJ3hw9wNxx+BIBOijnlG9PnrJsCcSjs1wyyD6vJpaYtgnzDrKYRSqf3OO6Rfa93xsRg==", + "license": "MIT", + "engines": { + "node": ">= 0.4" + }, + "funding": { + "url": "https://github.com/sponsors/ljharb" + } + }, "node_modules/graceful-fs": { "version": "4.2.11", "resolved": "https://registry.npmjs.org/graceful-fs/-/graceful-fs-4.2.11.tgz", @@ -1861,6 +2106,45 @@ "node": ">=8" } }, + "node_modules/has-symbols": { + "version": "1.1.0", + "resolved": "https://registry.npmjs.org/has-symbols/-/has-symbols-1.1.0.tgz", + "integrity": "sha512-1cDNdwJ2Jaohmb3sg4OmKaMBwuC48sYni5HUw2DvsC8LjGTLK9h+eb1X6RyuOHe4hT0ULCW68iomhjUoKUqlPQ==", + "license": "MIT", + "engines": { + "node": ">= 0.4" + }, + "funding": { + "url": "https://github.com/sponsors/ljharb" + } + }, + "node_modules/has-tostringtag": { + "version": "1.0.2", + "resolved": "https://registry.npmjs.org/has-tostringtag/-/has-tostringtag-1.0.2.tgz", + "integrity": "sha512-NqADB8VjPFLM2V0VvHUewwwsw0ZWBaIdgo+ieHtK3hasLz4qeCRjYcqfB6AQrBggRKppKF8L52/VqdVsO47Dlw==", + "license": "MIT", + "dependencies": { + "has-symbols": "^1.0.3" + }, + "engines": { + "node": ">= 0.4" + }, + "funding": { + "url": "https://github.com/sponsors/ljharb" + } + }, + "node_modules/hasown": { + "version": "2.0.4", + "resolved": "https://registry.npmjs.org/hasown/-/hasown-2.0.4.tgz", + "integrity": "sha512-T2UbfbBEF32wiepXIsMlTW9+dDYC6wMh/t/vYA4tuOMKqWz/n3vr1NFSxQiyP+zk2mXsoMA/i/7qV6LKut1t1A==", + "license": "MIT", + "dependencies": { + "function-bind": "^1.1.2" + }, + "engines": { + "node": ">= 0.4" + } + }, "node_modules/htmlfy": { "version": "0.8.1", "resolved": "https://registry.npmjs.org/htmlfy/-/htmlfy-0.8.1.tgz", @@ -1924,6 +2208,15 @@ "node": ">= 14" } }, + "node_modules/humanize-ms": { + "version": "1.2.1", + "resolved": "https://registry.npmjs.org/humanize-ms/-/humanize-ms-1.2.1.tgz", + "integrity": "sha512-Fl70vYtsAFb/C06PTS9dZBo7ihau+Tu/DNCk/OyHhea07S+aeMWpFFkUaXRa8fI+ScZbEI8dfSxwY7gxZ9SAVQ==", + "license": "MIT", + "dependencies": { + "ms": "^2.0.0" + } + }, "node_modules/iconv-lite": { "version": "0.6.3", "resolved": "https://registry.npmjs.org/iconv-lite/-/iconv-lite-0.6.3.tgz", @@ -2216,6 +2509,36 @@ "integrity": "sha512-JNAzZcXrCt42VGLuYz0zfAzDfAvJWW6AfYlDBQyDV5DClI2m5sAmK+OIO7s59XfsRsWHp02jAJrRadPRGTt6SQ==", "license": "ISC" }, + "node_modules/math-intrinsics": { + "version": "1.1.0", + "resolved": "https://registry.npmjs.org/math-intrinsics/-/math-intrinsics-1.1.0.tgz", + "integrity": "sha512-/IXtbwEk5HTPyEwyKX6hGkYXxM9nbj64B+ilVJnC/R6B0pH5G4V3b0pVbL7DBj4tkhBAppbQUlf6F6Xl9LHu1g==", + "license": "MIT", + "engines": { + "node": ">= 0.4" + } + }, + "node_modules/mime-db": { + "version": "1.52.0", + "resolved": "https://registry.npmjs.org/mime-db/-/mime-db-1.52.0.tgz", + "integrity": "sha512-sPU4uV7dYlvtWJxwwxHD0PuihVNiE7TyAbQ5SWxDCB9mUYvOgroQOwYQQOKPJ8CIbE+1ETVlOoK1UC2nU3gYvg==", + "license": "MIT", + "engines": { + "node": ">= 0.6" + } + }, + "node_modules/mime-types": { + "version": "2.1.35", + "resolved": "https://registry.npmjs.org/mime-types/-/mime-types-2.1.35.tgz", + "integrity": "sha512-ZDY+bPm5zTTF+YpCrAU9nK0UgICYPT0QtT1NZWFv4s++TNkcgVaT0g6+4R2uI4MjQjzysHB1zxuWL50hzaeXiw==", + "license": "MIT", + "dependencies": { + "mime-db": "1.52.0" + }, + "engines": { + "node": ">= 0.6" + } + }, "node_modules/minimatch": { "version": "9.0.9", "resolved": "https://registry.npmjs.org/minimatch/-/minimatch-9.0.9.tgz", @@ -2270,6 +2593,46 @@ "node": ">= 0.4.0" } }, + "node_modules/node-domexception": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/node-domexception/-/node-domexception-1.0.0.tgz", + "integrity": "sha512-/jKZoMpw0F8GRwl4/eLROPA3cfcXtLApP0QzLmUT/HuPCZWyB7IY9ZrMeKw2O/nFIqPQB3PVM9aYm0F312AXDQ==", + "deprecated": "Use your platform's native DOMException instead", + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/jimmywarting" + }, + { + "type": "github", + "url": "https://paypal.me/jimmywarting" + } + ], + "license": "MIT", + "engines": { + "node": ">=10.5.0" + } + }, + "node_modules/node-fetch": { + "version": "2.7.0", + "resolved": "https://registry.npmjs.org/node-fetch/-/node-fetch-2.7.0.tgz", + "integrity": "sha512-c4FRfUm/dbcWZ7U+1Wq0AwCyFL+3nt2bEw05wfxSz+DWpWsitgmSgYmy2dQdWyKC1694ELPqMs/YzUSNozLt8A==", + "license": "MIT", + "dependencies": { + "whatwg-url": "^5.0.0" + }, + "engines": { + "node": "4.x || >=6.0.0" + }, + "peerDependencies": { + "encoding": "^0.1.0" + }, + "peerDependenciesMeta": { + "encoding": { + "optional": true + } + } + }, "node_modules/normalize-path": { "version": "3.0.0", "resolved": "https://registry.npmjs.org/normalize-path/-/normalize-path-3.0.0.tgz", @@ -3005,6 +3368,12 @@ "b4a": "^1.6.4" } }, + "node_modules/tr46": { + "version": "0.0.3", + "resolved": "https://registry.npmjs.org/tr46/-/tr46-0.0.3.tgz", + "integrity": "sha512-N3WMsuqV66lT30CrXNbEjx4GEwlow3v6rr4mCcv6prnfwhS01rkgyFdjPNBYd9br7LpXV1+Emh01fHnq2Gdgrw==", + "license": "MIT" + }, "node_modules/tslib": { "version": "2.8.1", "resolved": "https://registry.npmjs.org/tslib/-/tslib-2.8.1.tgz", @@ -3155,6 +3524,15 @@ "url": "https://github.com/chalk/chalk?sponsor=1" } }, + "node_modules/web-streams-polyfill": { + "version": "4.0.0-beta.3", + "resolved": "https://registry.npmjs.org/web-streams-polyfill/-/web-streams-polyfill-4.0.0-beta.3.tgz", + "integrity": "sha512-QW95TCTaHmsYfHDybGMwO5IJIM93I/6vTRk+daHTWFPhwh+C8Cg7j7XyKrwrj8Ib6vYXe0ocYNrmzY4xAAN6ug==", + "license": "MIT", + "engines": { + "node": ">= 14" + } + }, "node_modules/webdriver": { "version": "9.27.1", "resolved": "https://registry.npmjs.org/webdriver/-/webdriver-9.27.1.tgz", @@ -3248,6 +3626,12 @@ "undici-types": "~6.21.0" } }, + "node_modules/webidl-conversions": { + "version": "3.0.1", + "resolved": "https://registry.npmjs.org/webidl-conversions/-/webidl-conversions-3.0.1.tgz", + "integrity": "sha512-2JAn3z8AR6rjK8Sm8orRC0h/bcl/DqL7tRPdGZ4I1CjdF+EaMLmYxBHyXuKL849eucPFhvBoxMsflfOb8kxaeQ==", + "license": "BSD-2-Clause" + }, "node_modules/whatwg-encoding": { "version": "3.1.1", "resolved": "https://registry.npmjs.org/whatwg-encoding/-/whatwg-encoding-3.1.1.tgz", @@ -3270,6 +3654,16 @@ "node": ">=18" } }, + "node_modules/whatwg-url": { + "version": "5.0.0", + "resolved": "https://registry.npmjs.org/whatwg-url/-/whatwg-url-5.0.0.tgz", + "integrity": "sha512-saE57nupxk6v3HY35+jzBwYa0rKSy0XR8JSxZPwgLr7ys0IBzhGviA1/TUGJLmSVqs8pb9AnvICXEuOHLprYTw==", + "license": "MIT", + "dependencies": { + "tr46": "~0.0.3", + "webidl-conversions": "^3.0.0" + } + }, "node_modules/which": { "version": "6.0.1", "resolved": "https://registry.npmjs.org/which/-/which-6.0.1.tgz", diff --git a/e2e/package.json b/e2e/package.json index a7f28e8..d1660db 100644 --- a/e2e/package.json +++ b/e2e/package.json @@ -5,9 +5,11 @@ "type": "module", "scripts": { "test": "node --import tsx --test tests/*.smoke.ts", - "typecheck": "tsc --noEmit" + "typecheck": "tsc --noEmit", + "seed:browserbase": "node --import tsx scripts/seedBrowserbaseContext.ts" }, "dependencies": { + "@browserbasehq/sdk": "^2.19.0", "playwright": "^1.49.0", "webdriverio": "^9.5.0" }, diff --git a/e2e/runSmokeTest.ts b/e2e/runSmokeTest.ts index e8c3461..8f8a783 100644 --- a/e2e/runSmokeTest.ts +++ b/e2e/runSmokeTest.ts @@ -49,6 +49,10 @@ async function loadDriver(client: ClientName): Promise { const { createClaudeDesktopDriver } = await import('./drivers/claude-desktop.ts'); return createClaudeDesktopDriver(); } + if (client === 'claude-web') { + const { createClaudeWebDriver } = await import('./drivers/claude-web.ts'); + return createClaudeWebDriver(); + } const { createChatGptWebDriver } = await import('./drivers/chatgpt-web.ts'); return createChatGptWebDriver(); } diff --git a/e2e/scripts/seedBrowserbaseContext.ts b/e2e/scripts/seedBrowserbaseContext.ts new file mode 100644 index 0000000..7738adb --- /dev/null +++ b/e2e/scripts/seedBrowserbaseContext.ts @@ -0,0 +1,266 @@ +// One-time setup: log into ChatGPT inside a Browserbase Context so the smoke +// tests can reuse that session without ever handling a password. +// +// This replaces the warmed `$HOME/e2e-chrome-profile` the local transport +// depends on — the thing that currently pins the ChatGPT job to the Mac Studio. +// +// Run it, open the printed Live View URL, log in by hand (2FA included — you +// are driving a real browser), then press Enter here. The cookies are written +// back to the Context on close, and every later session reads them. +// +// npm run seed:browserbase # creates a new context +// BROWSERBASE_CONTEXT_ID=ctx_… npm run seed:browserbase # refresh an existing one +// +// Re-run whenever ChatGPT expires the session; nothing else needs touching. + +import { createInterface } from 'node:readline/promises'; +import { stdin, stdout } from 'node:process'; +import { chromium } from 'playwright'; +import { browserbaseClient, createBrowserbaseSession, liveViewUrl } from '../drivers/browserbase.ts'; + +// Which site's login this context is for. One context per client — cookies for +// claude.ai and chatgpt.com are unrelated, and mixing them in one context would +// mean re-seeding both whenever either expires. +const CLIENT = process.env.CLIENT ?? 'claude-web'; +const SEED_TARGETS: Record = { + 'claude-web': { url: process.env.CLAUDE_URL ?? 'https://claude.ai/new', label: 'Claude' }, + 'chatgpt-web': { url: process.env.CHATGPT_URL ?? 'https://chatgpt.com/', label: 'ChatGPT' }, +}; + +/** + * Strip quotes and whitespace off an env value. + * + * `export KEY="bb_live_…"` in some shells, or a copy-paste that grabs a + * trailing newline, leaves the quotes/whitespace IN the value. The API then + * returns a bare 401 that looks like a bad key rather than a bad paste. + */ +function envValue(name: string): string { + return (process.env[name] ?? '').trim().replace(/^['"]|['"]$/g, ''); +} + +/** Turn a 401 into something that says what to check. */ +function explainAuthFailure(apiKey: string, projectId: string): void { + console.error(''); + console.error('[seed] Browserbase rejected the credentials (HTTP 401).'); + console.error(''); + console.error(` BROWSERBASE_API_KEY = ${mask(apiKey)}`); + console.error(` BROWSERBASE_PROJECT_ID = ${mask(projectId)}`); + console.error(''); + // Both live next to each other on the Settings page and neither is labelled + // in a way that survives a hurried copy-paste, so this is the usual cause. + if (projectId.startsWith('bb_') && !apiKey.startsWith('bb_')) { + console.error(' These look SWAPPED — the project id starts with "bb_", which is the'); + console.error(' API key prefix. Try exchanging the two values.'); + } else if (!apiKey.startsWith('bb_')) { + console.error(' The API key does not start with "bb_", so this may be the project id'); + console.error(' (a UUID) pasted into the key. Both are on the same Settings page.'); + } else { + console.error(' The key looks well-formed, so check it has not been revoked or'); + console.error(' regenerated, and that it belongs to this project.'); + } + console.error(''); + console.error(' Verify the key on its own:'); + console.error(` curl -s -o /dev/null -w '%{http_code}\\n' \\`); + console.error(` -H "X-BB-API-Key: $BROWSERBASE_API_KEY" \\`); + console.error(' https://api.browserbase.com/v1/projects'); + console.error(' 200 = key is good; 401 = the key itself is the problem.'); + console.error(''); +} + +function mask(value: string): string { + if (!value) return '(unset)'; + if (value.length <= 8) return `${value.slice(0, 2)}… (${value.length} chars)`; + return `${value.slice(0, 6)}…${value.slice(-4)} (${value.length} chars)`; +} + +async function main(): Promise { + const apiKey = envValue('BROWSERBASE_API_KEY'); + let projectId = envValue('BROWSERBASE_PROJECT_ID'); + if (!apiKey) { + console.error('Set BROWSERBASE_API_KEY first. See e2e/BROWSERBASE.md.'); + process.exit(1); + } + // Write the cleaned key back so the SDK sees the trimmed form rather than the + // raw paste. The project id is written back after it has been validated. + process.env.BROWSERBASE_API_KEY = apiKey; + + const target = SEED_TARGETS[CLIENT]; + if (!target) { + console.error(`[seed] CLIENT=${CLIENT} has no browser login to seed.`); + console.error(`[seed] Expected one of: ${Object.keys(SEED_TARGETS).join(', ')}.`); + console.error('[seed] (claude-desktop authenticates in the app itself, not in a browser context.)'); + process.exit(1); + } + console.error(`[seed] seeding a ${target.label} login (CLIENT=${CLIENT})`); + + const bb = browserbaseClient(); + + // Cheapest authenticated call there is, and it doubles as the project check. + // Failing here means the credentials are wrong, and saying so beats a stack + // trace out of contexts.create(). + let projects: Array<{ id: string; name: string }>; + try { + projects = await bb.projects.list(); + } catch (err: any) { + if (err?.status === 401) { + explainAuthFailure(apiKey, projectId); + process.exit(1); + } + throw err; + } + + // A key that authenticates fine will still 401 on contexts.create() when the + // projectId belongs to someone else — the error says "Unauthorized" and + // names nothing, which is indistinguishable from a bad key. Since we already + // know every project this key owns, check it here and say which is wrong. + const owned = projects.map((p) => p.id); + if (projectId && !owned.includes(projectId)) { + console.error(''); + console.error(`[seed] BROWSERBASE_PROJECT_ID (${mask(projectId)}) is not a project this API key owns.`); + console.error('[seed] That is what produces a 401 from contexts.create() even though the key is valid.'); + console.error(''); + console.error(' Projects this key can use:'); + for (const p of projects) console.error(` ${p.id} ${p.name}`); + console.error(''); + process.exit(1); + } + if (!projectId) { + if (projects.length === 1) { + projectId = projects[0].id; + console.error(`[seed] BROWSERBASE_PROJECT_ID not set — using the only project: ${projectId} (${projects[0].name})`); + } else { + console.error('[seed] BROWSERBASE_PROJECT_ID is not set and this key owns several projects:'); + for (const p of projects) console.error(` ${p.id} ${p.name}`); + process.exit(1); + } + } + process.env.BROWSERBASE_PROJECT_ID = projectId; + + let contextId = envValue('BROWSERBASE_CONTEXT_ID') || undefined; + if (contextId) { + console.error(`[seed] refreshing existing context ${contextId}`); + } else { + const context = await bb.contexts.create({ projectId }); + contextId = context.id; + console.error(`[seed] created context ${contextId}`); + } + + // persist: true is correct HERE and nowhere else — this is the one run whose + // job is to WRITE the cookie jar. The tests read it with persist: false so a + // parallel gate run cannot race 18 sessions writing back at once. + process.env.BROWSERBASE_CONTEXT_ID = contextId; + const session = await createBrowserbaseSession({ + persist: true, + // Generous: a human is logging in, possibly hunting for a 2FA code. + timeoutSeconds: 1800, + // NOT keepAlive. `persist` writes the cookie jar back when the SESSION + // ends, and keepAlive's whole purpose is to keep it running after the + // client disconnects — so browser.close() would detach without ending + // anything and the login would never be saved. That produced a context + // that looked seeded and yielded "Sign in - Claude" on the first test run. + }); + + const browser = await chromium.connectOverCDP(session.connectUrl); + const ctx = browser.contexts()[0] ?? (await browser.newContext()); + const page = ctx.pages()[0] ?? (await ctx.newPage()); + await page.goto(target.url, { waitUntil: 'domcontentloaded' }); + + const live = await liveViewUrl(session.sessionId); + console.error(''); + console.error(` Open this and log into ${target.label}:`); + console.error(` ${live}`); + console.error(''); + console.error(` Confirm the MCP connector is configured on the ${target.label} account`); + console.error(' too — connectors are account-side, so once set there is nothing'); + console.error(' per-session to redo.'); + console.error(''); + + const rl = createInterface({ input: stdin, output: stdout }); + await rl.question('Press Enter once you are logged in and the chat UI is loaded… '); + rl.close(); + + // Prove the login actually took before saving, rather than persisting an + // anonymous jar that fails confusingly on the first test run. + const loggedIn = await page + .locator('#prompt-textarea, div[contenteditable="true"]') + .first() + .isVisible() + .catch(() => false); + + // Disconnect, then ask Browserbase to END the session explicitly. Closing the + // CDP connection alone is not a guarantee the session completed, and `persist` + // only writes on completion — so this is the step that actually saves the + // login. REQUEST_RELEASE is also what stops it billing until the timeout. + await browser.close(); + try { + await bb.sessions.update(session.sessionId, { status: 'REQUEST_RELEASE', projectId }); + } catch (err: any) { + console.error(`[seed] could not release the session: ${err?.message ?? err}`); + console.error('[seed] the context may not have been written — re-run and check.'); + } + await new Promise((r) => setTimeout(r, 5_000)); // docs: allow a few seconds to sync + + // Prove the cookies actually landed, rather than trusting the write. This is + // the check whose absence let a silently-empty context reach a test run. + await verifyContextPersisted(contextId, target.url); + + if (!loggedIn) { + console.error(''); + console.error('[seed] WARNING: no composer found on the page, so the login may not have'); + console.error('[seed] completed. The context was still saved — re-run to try again.'); + } + + console.error(''); + console.error('[seed] done. Add to CI secrets / your shell:'); + console.error(` BROWSERBASE_CONTEXT_ID=${contextId}`); + console.error(''); + console.error('Then run a test against it:'); + console.error(` E2E_BROWSER=browserbase CLIENT=${CLIENT} \\`); + console.error(' node --import tsx --test tests/readGoogleDoc.smoke.ts'); +} + +/** + * Re-open the context in a fresh session and confirm the login survived. + * + * Without this the script's "done" only means "we asked it to save". The first + * evidence of failure was otherwise a smoke test landing on a sign-in page, + * which reads as a broken selector rather than a broken context. + */ +async function verifyContextPersisted(contextId: string, url: string): Promise { + console.error('[seed] verifying the saved login by reopening the context…'); + process.env.BROWSERBASE_CONTEXT_ID = contextId; + const check = await createBrowserbaseSession({ persist: false, timeoutSeconds: 120 }); + const browser = await chromium.connectOverCDP(check.connectUrl); + try { + const ctx = browser.contexts()[0] ?? (await browser.newContext()); + const page = ctx.pages()[0] ?? (await ctx.newPage()); + await page.goto(url, { waitUntil: 'domcontentloaded' }); + await page.waitForTimeout(4_000); // let any client-side redirect settle + + const title = await page.title().catch(() => ''); + const signedOut = /sign in|log in/i.test(title); + if (signedOut) { + console.error(''); + console.error(`[seed] FAILED: the reopened context lands on "${title}".`); + console.error('[seed] The login did not persist. Re-run and make sure you are fully'); + console.error('[seed] signed in — the chat UI loaded, not just the password submitted —'); + console.error('[seed] before pressing Enter.'); + console.error(`[seed] Replay of this check: ${check.replayUrl}`); + process.exitCode = 1; + return; + } + console.error(`[seed] verified — reopened context loads "${title}".`); + } finally { + await browser.close(); + try { + // Release rather than letting it idle to timeout — an unreleased check + // session bills for its full duration. + browserbaseClient().sessions.update(check.sessionId, { status: 'REQUEST_RELEASE' }); + } catch { /* best-effort */ } + } +} + +main().catch((err) => { + console.error('[seed] failed:', err); + process.exit(1); +});