MCP tools (snapshot/read/grep/run) never return payload inline; plus interaction, crash, and MCP-registration bugs

[debpramanik]

Summary

Testing the BrowserOS MCP server (mcp__browseros__* tools, accessed via a Claude Code MCP client over the HTTP transport) surfaced a cluster of bugs across three areas: tools that never return their actual payload, several interaction/reporting inconsistencies, and infrastructure instability (server port drift + a recurring MCP client config regression). Confirmed reproducible across two testing sessions and two server versions (0.0.115 and 0.0.124, i.e. after upgrading specifically to check whether these were already fixed).

Environment

• BrowserOS app version: 0.46.25.0
• Agent server version: reproduced on both 0.0.115 and 0.0.124 (upgraded mid-testing via browseros-cli install specifically to check whether these were fixed in the latest release — they were not)
• Chromium: 148.0.7948.97
• macOS (Darwin 25.5.0, arm64)
• browseros-cli 0.2.2 initially, 0.4.0 available
• Client: Claude Code, MCP over HTTP ({"type":"http","url":"http://127.0.0.1:<port>/mcp"})

Data-return bugs (core pattern)

1. snapshot, read, grep, and run never return their actual payload inline — only metadata ({"page":N}, {"count":N}, {"ok":true}, {"contentLength":N,"writtenToFile":false}). Confirmed the underlying data is real and retrievable via evaluate (which works correctly), so this isn't a case of the data not existing — it's just never surfaced in the tool response.
2. Refinement of Needs steps/guide on how to run the project locally in README.md #1: snapshot/read do return a real, usable file path once content crosses a size threshold (roughly tens of KB). Below that threshold, both the file path and the inline text are missing — the content isn't just hidden behind a path, it's lost entirely for small pages.
3. run's return value and console.log output have no such size-based fallback — the response is always exactly {"ok":true} regardless of payload size. Only thrown exceptions surface (as the error message). This makes run effectively non-functional for its stated purpose (multi-step JS + data extraction).
4. grep's match count is accurate but it never returns the matched lines themselves — same metadata-only pattern as Needs steps/guide on how to run the project locally in README.md #1. Reproduced with a simple case-sensitive literal pattern known to match visible page text.

This looks related to #1376 (structured results for screenshot/grep/diff) and #1417 (screenshot missing structured image data, fixed in agent-server 0.0.124) — but unlike screenshot, snapshot/read/grep/run are still broken on 0.0.124, so whatever fixed screenshot's structured-content handling didn't extend to these.

Interaction / reporting bugs

5. Keyboard input (type/type_at/press) silently no-ops on background (non-focused) tabs — no error is raised, changed:false is returned, and the target field never receives the text.
6. act's changed flag and diff are unreliable in both directions: a verified real DOM change (a checkbox toggle) was reported as changed:false/"no change"; conversely, under a corrupted ref-map condition, fill reported changed:true when nothing had actually happened.
7. press with key "Enter" doesn't trigger native form submission, even on a correctly focused/filled field. Clicking the submit button (by ref, or via click_at) works fine as a workaround.
8. act kind scroll reliably timed out in original testing. A later retest didn't hang, but that retest was on a page with nothing to scroll, so it's inconclusive whether this is actually fixed.
9. drag_at did not perform HTML5-native drag-and-drop. Plausibly a generic pointer-event automation limitation rather than a BrowserOS-specific bug (a known hard case for automation generally).
10. Minor: tab_groups action:"create" ignores the color parameter (always returns "grey"); a follow-up action:"update" with the same color does apply correctly.

Infrastructure / stability bugs

11. The local BrowserOS agent server can crash under heavy synthetic DOM load — injecting ~3000 filler DOM elements (used as a stress-test workaround while investigating Using Releases instead of a private beta download portal #2) preceded the local server refusing all connections, reproduced twice.
12. The registered MCP client config's "type":"http" field gets dropped, and the server's port changes, recurringly. Hit 3 times in one day across two testing sessions: on the original re-add, again after the server crash/restart in Does it support file uploading? #11, and then a third time spontaneously mid-session with no heavy load in progress (port cycled 9200 → 9202 → 9200 with no action on our end). Because it recurred with no corresponding client-side action, this doesn't look like purely a crash-recovery artifact — something appears to intermittently regenerate or touch the MCP client registration. Not certain what's writing it; flagging in case BrowserOS (the app, browseros-cli launch/init, or an auto-configure-MCP-clients feature) is the one touching third-party AI client configs like ~/.claude.json, and doing so without the type field.
13. Fixing the client registration mid-session never takes effect in that same session — claude mcp list/curl confirm the server is healthy and the config is correct, but the already-running session's tool transport stays bound to whatever was configured at session start. A brand-new client session is required every time the port/config changes. (This may be more a property of the MCP client than BrowserOS itself, but it makes the port-drift bug in "Google API keys are missing. Some functionality of Chromium will be disabled." error/warning message #12 considerably more disruptive in practice, since every drift requires restarting the AI client session, not just re-pointing config.)

Separately, this also relates to open issue #1514 (server drops mid-session, rebinds between ports 9200/9201) — we hit the same class of instability, but observed a 3-port cycle (9200→9202→9200) rather than the 2-port cycle described there, and tied it to the registration's type field going missing.

Not a bug (documented quirk, for context)

search_documentation's keyword matching is literal, not semantic (as it discloses in its own description) — a query like "create issue" surfaced repo/org-creation actions before the actual GITHUB_ISSUES.create action. Not filing this as a bug, just noting it since it came up during the same testing pass.

Why this matters

Together, #1-4 make the snapshot → act(ref) loop — the documented core automation pattern — unusable as designed: no [ref=eN] handles are ever returned, so ref-based act kinds and ref-based download/upload can't be driven at all. The only reliable workaround found is evaluate (CDP Runtime.evaluate) for reads plus coordinate-based act kinds (click_at, type_at, etc.) for interaction — screenshot and evaluate were the only two tools that behaved correctly and consistently across all testing.

Happy to provide raw request/response logs for any of these if useful — let me know which ones would help most.