Remove built-in Playwright MCP support - #57521
Conversation
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
|
🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅
|
|
✅ Test Quality Sentinel completed test quality analysis. Test Quality Sentinel skipped because pre-fetch PR data was unavailable: unable to fetch test file diff
|
|
✅ PR Code Quality Reviewer completed the code quality review. Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
To allow these domains, add them to the network:
allowed:
- defaults
- "github.com"See Network Configuration for more information.
|
|
Lean already. Ship. Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
To allow these domains, add them to the network:
allowed:
- defaults
- "ab.chatgpt.com"See Network Configuration for more information.
|
|
✅ Design Decision Gate 🏗️ completed the design decision gate check. See the comment below for the result and any generated ADR draft. Warning Firewall blocked 2 domainsThe following domains were blocked by the firewall during workflow execution:
To allow these domains, add them to the network:
allowed:
- defaults
- "github.com"
- "proxy.golang.org"See Network Configuration for more information.
|
Comment MemoryNote This comment is managed by comment memory.It stores persistent context for this thread in the code block at the top of this comment. Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
To allow these domains, add them to the network:
allowed:
- defaults
- "github.com"See Network Configuration for more information.
|
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
This removes the built-in Playwright MCP path consistently and leaves the CLI path as the only first-party integration. I did not find a blocking regression in the changed lines.
Checked themes
- Playwright mode validation now rejects
mode: mcpwith migration guidance. - MCP server detection/rendering no longer treats built-in Playwright as an MCP server.
- Copilot/Codex permission plumbing still auto-allows
playwright-cli:*only when restricted bash needs it. - The remaining
@playwright/mcpreferences I found are outside this diff and belong to generic dependency/runtime parsing tests, not the removed built-in integration.
Warning
Firewall blocked 1 domain
The following domain was blocked by the firewall during workflow execution:
github.com
To allow these domains, add them to the network.allowed list in your workflow frontmatter:
network:
allowed:
- defaults
- "github.com"See Network Configuration for more information.
🔎 Code quality review by PR Code Quality Reviewer · copilot · gpt54 · 68.7 AIC · ⌖ 7.36 AIC · ⊞ 21.8K
Comment /review to run again
🏗️ ADR required — draft added for PR #57521I added a draft ADR at Evidence used
Inferred decision captured in the draft
Next actionPlease review and refine the ADR’s rationale/trade-offs, then keep it with the PR before merge. Warning Firewall blocked 2 domainsThe following domains were blocked by the firewall during workflow execution:
To allow these domains, add them to the network:
allowed:
- defaults
- "github.com"
- "proxy.golang.org"See Network Configuration for more information.
|
There was a problem hiding this comment.
Skills-Based Review 🧠
Applied /codebase-design — commenting on two correctness issues and one minor cleanup opportunity. Overall this is a clean, well-scoped removal with good test hygiene and excellent migration documentation.
📋 Key Themes & Highlights
Issues Found
-
Schema/compiler mismatch (
pkg/parser/schemas/main_workflow_schema.jsonL4777):"enum": ["cli", "mcp"]still allows"mcp"— IDE autocompletion will suggest a value the compiler then rejects. Remove"mcp"from the enum. -
Flag inconsistency in migration docs (
docs/src/content/docs/reference/playwright.mdL26 vs L72):browser_evaluateis documented with--expressionin the code example but--functionin the migration table. One of them is wrong and will cause immediate failures for migrating users. -
Minor redundant dispatch (
pkg/parser/mcp.go):extractBuiltinMCPToolscallsprocessBuiltinMCPToolwhich re-checkstoolName == "github"— double dispatch left over from theplaywrightremoval. Low risk but worth cleaning up for readability.
Positive Highlights
- ✅ Compiler rejects
mode: mcpwith a detailed, actionable error message including a documentation link — excellent UX for migration. - ✅ 2564 deletions vs 360 additions: extremely clean removal with no orphaned stubs.
- ✅
playwright_validation_test.gocovers all meaningful cases (nil, false, empty map,cli,CLI,mcp, expressions) after the refactor. - ✅ The custom MCP server snippet in the docs (
mcp-servers.playwright) gives a copy-pasteable escape hatch for users who genuinely need MCP. - ✅
(nolint/redacted):largefuncannotations on the unchanged inspection flow are accurate and add no technical debt.
🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · copilot · sonnet46 · 95.3 AIC · ⌖ 15.3 AIC · ⊞ 7.6K
Comment /matt to run again
Comments that could not be inline-anchored
pkg/parser/schemas/main_workflow_schema.json:4777
[/codebase-design] The mode enum still includes "mcp" even though the compiler now rejects it — the schema advertises a value that is illegal at compile time, causing IDE autocompletion to suggest a broken option.
<details>
<summary>💡 Suggested fix</summary>
Change:
"enum": ["cli", "mcp"]to:
"enum": ["cli"]This makes the schema the source of truth and prevents IDE tools from suggesting the removed value.
</details>
@copilot please address this.
docs/src/content/docs/reference/playwright.md:72
[/codebase-design] The migration table in browser_evaluate uses --function but the code example 10 lines above uses --expression — these flags are inconsistent and at least one is wrong.
<details>
<summary>💡 Detail</summary>
Line 26: playwright-cli browser_evaluate --expression "document.title"
Line 72: | browser_evaluate | playwright-cli browser_evaluate --function <javascript> |
Verify the correct flag against playwright-cli browser_evaluate --help and align both occurrenc…
pkg/parser/mcp.go:256
[/codebase-design] The extractBuiltinMCPTools function now only handles github, but still iterates over all tools calling processBuiltinMCPTool. The processBuiltinMCPTool call for non-github tool names returns (nil, nil) — dead code path that was only needed for playwright. Consider simplifying by removing the dispatch through processBuiltinMCPTool for non-github tools or tightening the guard condition.
<details>
<summary>💡 Suggested simplification</summary>
In `pkg/par…
There was a problem hiding this comment.
Impeccable Review: Remove built-in Playwright MCP support
Change type: refactor_cleanup — removes MCP mode entirely, makes CLI the only supported integration.
Overall assessment: The removal is well-structured. The compile-time error for mode: mcp with actionable migration guidance is the right pattern. Test updates correctly reflect that all playwright configs now map to CLI mode (since IsCLIMode() returns true for empty or "cli" mode). The docker.go cleanup is clean. The copilot_engine_tools.go fix (using effectiveWorkflowData instead of workflowData) is a correctness improvement bundled naturally with this change.
One non-blocking issue found: browser_evaluate flag inconsistency in the migration docs (inline comment above). The bash usage example uses --expression while the migration table uses --function — one is wrong, and users following the migration guide will encounter a flag error.
No other actionable issues found.
🧵 Reviewed using Impeccable skills by Impeccable Skills Reviewer · copilot · sonnet46 · 94.7 AIC · ⌖ 14.7 AIC · ⊞ 6.2K
| | `browser_snapshot` | `playwright-cli browser_snapshot` | | ||
| | `browser_take_screenshot` | `playwright-cli browser_take_screenshot --filename <path>` | | ||
| | `browser_click` | `playwright-cli browser_click --element <description> --ref <ref>` | | ||
| | `browser_evaluate` | `playwright-cli browser_evaluate --function <javascript>` | |
There was a problem hiding this comment.
browser_evaluate is documented with two different flags in this same file:
- Line 26 (bash usage example):
playwright-cli browser_evaluate --expression "document.title" - Line 72 (migration table):
playwright-cli browser_evaluate --function <javascript>
One of these is wrong, and users migrating from MCP mode will follow the migration table at line 72 and get a flag error. Please verify the correct flag name against the @playwright/cli package and fix the inconsistency.
@copilot please address this.
There was a problem hiding this comment.
🟡 Changes recommended
Migration commands are invalid, and custom mcp-servers.playwright configurations are misclassified and removed for Codex.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
Removes built-in Playwright MCP support and makes @playwright/cli the sole built-in browser integration.
Changes:
- Removes Playwright MCP rendering, registration, containers, arguments, and version metadata.
- Adds CLI-default behavior and migration validation.
- Updates schemas, documentation, workflows, fixtures, and tests.
File summaries
| File | Description |
|---|---|
pkg/workflow/version_field_test.go |
Tests CLI version parsing. |
pkg/workflow/tools_types.go |
Defaults Playwright to CLI mode. |
pkg/workflow/tools_types_test.go |
Updates config parsing tests. |
pkg/workflow/tools_parser.go |
Removes MCP argument parsing. |
pkg/workflow/testdata/TestWasmGolden_CompileFixtures/smoke-copilot.golden |
Updates compiled fixture. |
pkg/workflow/safe_update_manifest_test.go |
Removes built-in MCP manifest expectations. |
pkg/workflow/repo_config_test.go |
Replaces Playwright image fixture. |
pkg/workflow/README.md |
Updates generated API reference. |
pkg/workflow/playwright_validation.go |
Rejects removed MCP mode. |
pkg/workflow/playwright_validation_test.go |
Tests migration errors. |
pkg/workflow/playwright_tools.go |
Removes MCP tool inventory. |
pkg/workflow/playwright_mcp_integration_test.go |
Removes MCP integration tests. |
pkg/workflow/playwright_cli.go |
Makes CLI the built-in integration. |
pkg/workflow/playwright_cli_test.go |
Tests omitted-mode behavior. |
pkg/workflow/playwright_allowed_domains_secrets_test.go |
Removes MCP argument tests. |
pkg/workflow/non_github_mcp_guard_policy_test.go |
Removes Playwright guard-policy cases. |
pkg/workflow/neutral_tools_test.go |
Removes MCP expansion expectations. |
pkg/workflow/mcp_setup_generator.go |
Stops MCP registration. |
pkg/workflow/mcp_renderer.go |
Removes Playwright rendering dispatch. |
pkg/workflow/mcp_renderer_types.go |
Removes renderer callback. |
pkg/workflow/mcp_renderer_test.go |
Removes renderer tests. |
pkg/workflow/mcp_renderer_factory.go |
Removes renderer wiring. |
pkg/workflow/mcp_renderer_builtin.go |
Deletes built-in renderer. |
pkg/workflow/mcp_playwright_config.go |
Deletes MCP argument helpers. |
pkg/workflow/mcp_manifest.go |
Removes built-in manifest entry. |
pkg/workflow/mcp_logs_upload_test.go |
Verifies CLI installation. |
pkg/workflow/mcp_environment.go |
Removes MCP-specific environment handling. |
pkg/workflow/mcp_detection.go |
Excludes Playwright from MCP detection. |
pkg/workflow/mcp_detection_test.go |
Updates detection expectations. |
pkg/workflow/mcp_config_shared_test.go |
Removes shared renderer tests. |
pkg/workflow/mcp_config_refactor_test.go |
Removes TOML renderer tests. |
pkg/workflow/mcp_config_playwright_renderer.go |
Deletes MCP configuration renderer. |
pkg/workflow/mcp_cli_mount_test.go |
Tests CLI command mounting. |
pkg/workflow/mcp_benchmark_test.go |
Removes obsolete benchmark. |
pkg/workflow/importable_tools_test.go |
Updates imported-tool expectations. |
pkg/workflow/github_disabled_test.go |
Uses a custom MCP fixture. |
pkg/workflow/gemini_engine_test.go |
Updates CLI permission expectation. |
pkg/workflow/engine_helpers_shared_test.go |
Replaces renderer fixtures. |
pkg/workflow/engine_helpers_secrets_test.go |
Uses a custom MCP fixture. |
pkg/workflow/domains.go |
Updates browser-domain documentation. |
pkg/workflow/docker.go |
Stops collecting Playwright images. |
pkg/workflow/data/action_pins.json |
Removes Playwright image pin. |
pkg/workflow/copilot_engine_tools.go |
Enables CLI shell permission. |
pkg/workflow/copilot_engine_tool_arguments_test.go |
Updates permission tests. |
pkg/workflow/codex_playwright_test.go |
Tests CLI-only Codex behavior. |
pkg/workflow/codex_mcp.go |
Removes MCP rendering branch. |
pkg/workflow/codex_engine.go |
Removes Playwright MCP expansion. |
pkg/workflow/claude_tools.go |
Removes MCP permissions. |
pkg/workflow/claude_engine_tools_test.go |
Updates Claude tool expectations. |
pkg/workflow/claude_engine_test.go |
Updates CLI permission expectation. |
pkg/workflow/args.go |
Removes Playwright argument helpers. |
pkg/workflow/args_field_test.go |
Removes MCP argument tests. |
pkg/parser/schemas/main_workflow_schema.json |
Defines CLI-only configuration. |
pkg/parser/mcp.go |
Removes built-in MCP extraction. |
pkg/parser/mcp_test.go |
Removes extraction tests. |
pkg/constants/version_constants.go |
Removes MCP and browser versions. |
pkg/constants/constants.go |
Removes MCP log-directory constant. |
pkg/cli/workflows/test-playwright-args.md |
Deletes obsolete test workflow. |
pkg/cli/mcp_inspect.go |
Updates built-in descriptions. |
pkg/cli/mcp_inspect_playwright_live_integration_test.go |
Removes live MCP tests. |
pkg/cli/mcp_inspect_playwright_integration_test.go |
Removes inspect tests. |
pkg/actionpins/data/action_pins.json |
Removes mirrored image pin. |
docs/src/content/docs/troubleshooting/common-issues.md |
Replaces MCP troubleshooting guidance. |
docs/src/content/docs/reference/playwright.md |
Adds CLI migration documentation. |
docs/src/content/docs/reference/frontmatter-full.md |
Regenerates frontmatter reference. |
docs/public/editor/autocomplete-data.json |
Updates editor metadata. |
.github/workflows/smoke-codex.md |
Migrates smoke test to CLI. |
.github/workflows/smoke-codex.lock.yml |
Regenerates smoke workflow. |
.github/workflows/smoke-claude.lock.yml |
Removes MCP permissions. |
.github/workflows/dependabot-go-checker.md |
Tracks only the CLI package. |
.github/workflows/dependabot-go-checker.lock.yml |
Regenerates dependency checker. |
.github/workflows/cli-version-checker.md |
Removes MCP/browser tracking. |
.github/workflows/cli-version-checker.lock.yml |
Regenerates version checker. |
.github/workflows/blog-auditor.lock.yml |
Removes MCP permissions. |
.github/workflows/archivx-agentic-workflows-analyzer.lock.yml |
Removes MCP permissions. |
.github/aw/syntax-tools-imports.md |
Updates tool syntax guidance. |
.github/aw/actions-lock.json |
Removes Playwright image pin. |
.changeset/remove-playwright-mcp.md |
Records the breaking change. |
Review details
Suppressed comments (1)
docs/src/content/docs/reference/playwright.md:72
- These mappings retain MCP
browser_*names, which are not@playwright/clisubcommands. The migration table must use the CLI command surface or migrated workflows will fail immediately.
| `browser_navigate` | `playwright-cli browser_navigate --url <url>` |
| `browser_snapshot` | `playwright-cli browser_snapshot` |
| `browser_take_screenshot` | `playwright-cli browser_take_screenshot --filename <path>` |
| `browser_click` | `playwright-cli browser_click --element <description> --ref <ref>` |
| `browser_evaluate` | `playwright-cli browser_evaluate --function <javascript>` |
- Files reviewed: 79/79 changed files
- Comments generated: 7
- Review effort level: Balanced
| playwright-cli browser_navigate --url "https://example.com" | ||
| playwright-cli browser_snapshot |
| playwrightMCP["args"] = playwrightConfig.Args | ||
| } | ||
| result.raw["playwright"] = playwrightMCP | ||
| delete(result.raw, "playwright") |
| "mode": { | ||
| "description": "Integration mode: 'cli' (recommended) installs @playwright/cli via npm for token-efficient CLI invocations \u2014 use playwright-cli commands in bash and localhost to reach local servers; 'mcp' (deprecated) runs a Docker-based MCP server. Must be a literal value; GitHub Actions expressions are rejected.", | ||
| "description": "Integration mode. Only 'cli' is supported. The compiler rejects the removed 'mcp' value with migration guidance. Must be a literal value; GitHub Actions expressions are rejected.", |
| ``` | ||
|
|
||
| CLI mode installs `@playwright/cli` as a global npm package on the runner. The agent invokes `playwright-cli <command>` from bash: | ||
| The compiler installs `@playwright/cli` as a global npm package on the runner. The agent invokes `playwright-cli <command>` from bash: |
| // IsCLIMode returns true when the Playwright tool uses the supported CLI mode. | ||
| func (p *PlaywrightToolConfig) IsCLIMode() bool { | ||
| return p != nil && strings.EqualFold(p.Mode, "cli") | ||
| return p != nil && (p.Mode == "" || strings.EqualFold(p.Mode, "cli")) |
| "mode": { | ||
| "type": "string", | ||
| "desc": "Integration mode: 'cli' (recommended) installs @playwright/cli via npm for token-efficient CLI invocations — use play...", | ||
| "desc": "Integration mode.", |
|
@copilot this PR is ready for the next forward-progress pass. Please run the Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
To allow these domains, add them to the network:
allowed:
- defaults
- "github.com"See Network Configuration for more information.
|
…wright Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
Addressed the outstanding reviewer feedback: fixed the migration docs/troubleshooting/smoke test to use real |
There was a problem hiding this comment.
Warning
Firewall blocked 2 domains
The following domains were blocked by the firewall during workflow execution:
clients2.google.commtalk.google.com
To allow these domains, add them to the network.allowed list in your workflow frontmatter:
network:
allowed:
- defaults
- "clients2.google.com"
- "mtalk.google.com"See Network Configuration for more information.
💥 [THE END] — Illustrated by Smoke Claude · claude · sonnet46 · 64.2 AIC · ⌖ 18.7 AIC · ⊞ 7.2K
Comment /smoke-claude to run again
| @@ -0,0 +1,5 @@ | |||
| --- | |||
| "gh-aw": major | |||
There was a problem hiding this comment.
This changeset correctly marks this as a major change. Removing built-in Playwright MCP support is a breaking change for existing users who rely on tools.playwright with mode: mcp.
| @@ -0,0 +1,50 @@ | |||
| # ADR-57521: Remove Built-in Playwright MCP Support | |||
There was a problem hiding this comment.
Great addition of an ADR document for this architectural decision. Consider adding a section on rollback strategy for users who cannot migrate immediately.
|
@copilot add blog post explaining the reason for this: cli way is more secure and more token efficient |
|
@copilot review all agentic workflows in this repo and apply migration to cli. Add codemod that removes the tools.mode:cli since it is now the default. |
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Added |
…e repo workflows Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Added |
|
@copilot this PR is ready for the next forward-progress pass. You already replied after the last sous-chef nudge and pushed follow-up changes, but the PR is still blocked. Please run the Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
To allow these domains, add them to the network:
allowed:
- defaults
- "github.com"See Network Configuration for more information.
|
Verified again: all specific reviewer requests raised so far (invalid migration commands, schema |
|
@copilot this PR is ready for the next forward-progress pass. You already replied after the last sous-chef nudge and indicated the remaining work is done, but the PR is still blocked pending maintainer follow-up. Please run the Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
To allow these domains, add them to the network:
allowed:
- defaults
- "github.com"See Network Configuration for more information.
|
Documents the built-in tools.playwright integration as CLI-only following removal of built-in Playwright MCP support (#57521), and adds a Playwright CLI Mode glossary entry with links to the AWF sandbox policy behavior. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
🎉 This pull request is included in a new release. Release: |
Playwright now supports only the built-in CLI integration.
mode: mcpproduces actionable migration guidance; workflows requiring MCP must configure a custom server.Compiler
mode: mcp.Migration guidance
playwright-clicommands.localhost.Version tracking
@playwright/cli.Repository updates
✨ PR Review Safe Output Test - Run 33454421932
Warning
Firewall blocked 2 domains
The following domains were blocked by the firewall during workflow execution:
clients2.google.commtalk.google.comTo allow these domains, add them to the
network.allowedlist in your workflow frontmatter:See Network Configuration for more information.
Warning
Firewall blocked 1 domain
The following domain was blocked by the firewall during workflow execution:
github.comTo allow these domains, add them to the
network.allowedlist in your workflow frontmatter:See Network Configuration for more information.
Warning
Firewall blocked 1 domain
The following domain was blocked by the firewall during workflow execution:
github.comTo allow these domains, add them to the
network.allowedlist in your workflow frontmatter:See Network Configuration for more information.