Skip to content

Playwright test execution uses project's @playwright/test instead of Automaker's bundled version #756

@JasonBroderick

Description

@JasonBroderick

Related PR

#745 (fix(docker): Pre-install Playwright Chromium browsers for automated testing)

Problem

When Automaker runs verification tests using npx playwright test, the test execution incorrectly uses the project's @playwright/test package instead of Automaker's bundled Playwright.

Current behavior:

  1. Automaker's Docker image bundles Playwright 1.57.0 with chromium-1200
  2. Test command npx playwright test runs from the project's worktree directory
  3. npx resolves to the project's node_modules/.bin/playwright
  4. Test file imports (import { test, expect } from "@playwright/test") resolve to the project's @playwright/test
  5. If the project has a different Playwright version (e.g., 1.58.1 needing chromium-1208), tests fail with "browser not found"

Expected behavior:

Automaker's test execution should use Automaker's bundled Playwright and Chromium, regardless of what Playwright version (if any) the project has installed.

Root Cause

The test runner in apps/server/src/services/test-runner-service.ts spawns tests with:

spawn(finalCommand, [], {
  cwd: worktreePath,  // Project's directory
  ...
});

When npx playwright test runs from the project directory, Node's module resolution finds the project's @playwright/test first.

Architectural Constraint

  • Automaker writes verification tests to the project's tests/ directory
  • Test files contain import { test, expect } from "@playwright/test"
  • Node resolves imports relative to the file location, not the CLI binary location
  • Even using /app/node_modules/.bin/playwright directly doesn't fix imports in test files

Desired Solution

Automaker's test infrastructure should be completely decoupled from the project's dependencies:

  • Automaker's Playwright binary should be used for execution
  • Automaker's @playwright/test should be used for test file imports
  • The project's own Playwright installation (if any) should be irrelevant to Automaker's testing

Possible Approaches

  1. Module resolution override: Configure Node to resolve @playwright/test from Automaker's node_modules when running tests
  2. Test isolation: Store/execute Automaker's tests in a way that doesn't interact with project's node_modules
  3. Import rewriting: Generate test files with absolute imports to Automaker's Playwright
  4. Environment manipulation: Use NODE_PATH or similar to prioritize Automaker's modules

Need to determine the cleanest approach that doesn't invasively modify the project's files.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions