WIP: Test262 support #55997

o- · 2025-11-12T15:11:18Z

Add support for running Test262. To avoid polluting the outer environment the tests run in an iframe.

To do so we add:

A new minimal inner testharness-client which can collect Test262 results and post them to the parent site.
A test262 specific outer testharness which receives the results and reports them using the normal mechanism.
A harness-adapter which provides the minimally required infrastructure test262 tests expect from the global environment.

TBD: Currently the code assumes Test262 to be checked out in the root of the repo. It should be added as a dependency somehow.

This commit introduces the foundational infrastructure for the WPT manifest to recognize and process Test262 ECMA-262 conformance tests. This is the first step towards the goal of running the entire Test262 suite within the WPT infrastructure, as outlined in the approved RFC. The core of this change is a new `TestRecord` parser in `test262.py` which is responsible for extracting the YAML frontmatter from Test262 `.js` files. This metadata is then used by the manifest generation process to correctly identify and classify the tests. Key changes included in this commit: - A new `tools/manifest/test262.py` module containing the YAML frontmatter parser. - A corresponding `Test262Test` manifest item type defined in `tools/manifest/item.py` to represent these tests. - Updates to `tools/manifest/sourcefile.py` to identify Test262 files by their directory path (`/test262/`) and invoke the new parser. - The addition of `pyyaml` as a project dependency to handle the YAML parsing to tools/wpt/requirements.txt - **Note to reviewers:** PyYaml is lazily loaded. This is to prevent the modification of too many commands.json files. This may not be preferred since WPT does not do that often but open to other options. Please check failing checks for commits on the original PR for how I kept trying to track down the next commands.json file to fix. - New unit tests for the parser and the manifest generation logic for Test262 files. This work is part of the effort to add Test262 support as laid out in the RFC: web-platform-tests/rfcs#229 This commit is the first in a series of smaller PRs split from the larger, original implementation in #55997. Subsequent PRs will add the server-side logic to serve the tests and the wptrunner integration to execute them.

This commit introduces the necessary server-side handlers within `wptserve` to dynamically generate HTML wrappers for Test262 JavaScript tests. This is needed to enable Test262 execution within WPT. Key changes and their purpose: - Introduction of several new `HtmlWrapperHandler` and `WrapperHandler` subclasses (e.g., `Test262WindowHandler`, `Test262WindowTestHandler`, `Test262StrictHandler`). These handlers are responsible for: - Identifying Test262 test requests based on URL patterns. - Dynamically constructing an HTML page that loads the Test262 `.js` test file within an isolated `iframe`. - Injecting the required Test262 harness files (`assert.js`, `sta.js`) and the WPT-specific `testharness-client.js` and `harness-adapter.js` into the generated HTML. - Processing Test262-specific metadata (like `includes` and `negative` expectations) extracted by the manifest tooling from PR 1. - Updates to `RoutesBuilder` in `serve.py` to map incoming requests for Test262 test URLs to the appropriate new handler. - Unit tests in `test_serve.py` to validate the correct behavior of these new handlers, including URL rewriting, metadata processing, and the structure of the generated HTML wrappers. This work directly supports the integration of Test262 into WPT as detailed in the RFC: web-platform-tests/rfcs#229 This commit is the second in a series of smaller PRs split from the larger, original implementation in #55997.

…e tests This commit completes the core integration of Test262 into the WPT runner (`wptrunner`), enabling the execution of Test262 JavaScript conformance tests within the browser environment. This work builds upon the manifest tooling (PR 1 #56840) and server-side handling (PR 2 #56841) to provide a full end-to-end testing solution for Test262. Key components introduced and integrated: - **Client-Side Harness:** New JavaScript files (`harness-adapter.js`, `testharness-client.js`, `testharness.js`) under `resources/test262/` provide the necessary environment for Test262 tests to execute in an `iframe` and report results back to the main WPT testharness. - **Test262 Assertion Libraries (Vendored):** For the purpose of enabling these initial smoke tests, the standard Test262 assertion libraries (`assert.js` and `sta.js`) are directly placed into `third_party/test262/harness/`. This provides the immediate dependencies required by the smoke tests. An autoamted vendoring solution, including version tracking via `vendored.toml`, will be implemented in a subsequent PR focused on automated updates, as described in the RFC. - **Wptrunner Integration:** Modifications to `tools/wptrunner/wpttest.py` and `tools/wptrunner/browsers/*.py` enable `wptrunner` to recognize the `test262` test type and execute it using existing testharness executors. - **Smoke Tests and Metadata:** A suite of basic Test262 smoke tests (`infrastructure/test262/`) and their corresponding expected results (`infrastructure/metadata/infrastructure/test262/`) are added to verify the correct functioning of the entire integration. - **Linting Exemption:** `lint.ignore` is updated to exclude the vendored Test262 harness files from linting, respecting their upstream style. This integration allows browser vendors to run Test262 directly with `wptrunner`, streamlining conformance testing efforts as specified in the RFC: web-platform-tests/rfcs#229 This commit is the third in a series of smaller PRs split from the larger, original implementation in #55997.

This commit introduces a GitHub Actions workflow to automate the vendoring of the Test262 test suite into `third_party/test262`. This automation is the final piece of the integration, ensuring that the tests can be kept up-to-date with the upstream `tc39/test262` repository. The workflow is configured to run weekly or can be triggered manually. It performs the following steps: 1. Checks out both the WPT and `tc39/test262` repositories. 2. Selectively copies a subset of the Test262 tests into WPT. 3. Copies the essential Test262 harness files. 4. Creates a `vendored.toml` file at the root of the vendored directory, recording the specific commit SHA from `tc39/test262` that the files were sourced from. This ensures traceability, as described in the RFC. 5. Commits the updated files and opens a pull request with the changes. Initially, only a subset of Test262 tests related to Interop 2026 features (`Temporal` and `top-level-await`) are being imported. This is a temporary measure to manage the volume of new test results and to avoid overloading wpt.fyi, which currently has limitations in processing the full Test262 result set (see wpt-fyi/issues/4681). The workflow can be expanded to include more tests in the future. This work directly supports the integration of Test262 into WPT as detailed in the RFC: web-platform-tests/rfcs#229 This commit is part of a series of smaller PRs split from the larger, original implementation in #55997.

This commit integrates Test262 test execution into the WPT Taskcluster Continuous Integration system. Key changes and their purpose: - **Task Definition (`tools/ci/tc/tasks/test.yml`):** - A new `wpt-test262` component is defined, specifying the common properties and chunking strategy for Test262 tasks within Taskcluster. - The main Taskcluster task generation matrix is updated to include `test262` as a recognized test suite. This ensures that Taskcluster will automatically create jobs for Test262 across configured browsers and channels (e.g., Firefox Nightly, Chrome Canary). - **CI Validation (`tools/ci/tc/tests/test_valid.py`):** - The integration tests for Taskcluster's decision logic (`test_valid.py`) are updated to expect the new `wpt-*-test262-*` tasks in the generated task graph. This provides automated validation that the CI configuration is correctly interpreted and will schedule the Test262 tasks as intended. With these changes, Taskcluster will automatically detect and execute Test262 tests that are present in the `third_party/test262` directory (as populated by the automated vendoring workflow). This work is a fundamental part of the broader Test262 integration effort, as detailed in the RFC: web-platform-tests/rfcs#229 This commit is the final in a series of smaller PRs split from the larger, original implementation in #55997.