Skip to content

wpt: Use Webdriver for all WPT runs #56920

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jdm merged 5 commits into from
Jan 29, 2026
+792 −780

wpt: Fix requirements abstract method implementation

dd58a46
Select commit
Loading
Failed to load commit list.
Merged

wpt: Use Webdriver for all WPT runs #56920

wpt: Fix requirements abstract method implementation
dd58a46
Select commit
Loading
Failed to load commit list.
Community-TC Integration / infrastructure/ tests (firefox_android) succeeded Jan 28, 2026 in 23m 15s

Community-TC (pull_request)

Smoketests for wptrunner in firefox_android.

Details

View task in Taskcluster | View logs in Taskcluster | View task group in Taskcluster

Task Status

Started: 2026-01-28T17:51:31.652Z
Resolved: 2026-01-28T18:09:45.534Z
Task Execution Time: 18 minutes, 13 seconds, 882 milliseconds
Task Status: completed
Reason Resolved: completed
RunId: 0

Artifacts

- public/logs/live_backing.log
- public/logs/live.log
- public/results/checkrun.md
- public/results/emulator.log
- public/results/logcat-emulator-5554.log
- public/results/wptreport-firefox_android.json.gz

WPT Command: python3 ./wpt run --log-tbpl=- --log-wptreport=../artifacts/wptreport-firefox_android.json --logcat-dir=../artifacts/ --metadata=infrastructure/metadata/ --include=infrastructure/ --log-mach-level=info --log-mach=- -y --no-pause --no-restart-on-unexpected --install-fonts --verify-log-full --no-headless --install-browser --install-webdriver --logcat-dir /home/test/artifacts/ firefox_android

[taskcluster 2026-01-28 17:51:31.723Z] Task ID: V-KiL_KeQUKkbuGhsnxahg
[taskcluster 2026-01-28 17:51:31.723Z] Worker ID: 4080039460485959007
[taskcluster 2026-01-28 17:51:31.723Z] Worker Group: us-central1-b
[taskcluster 2026-01-28 17:51:31.723Z] Worker Node Type: projects/757942385826/machineTypes/n2-standard-4
[taskcluster 2026-01-28 17:51:31.723Z] Worker Pool: proj-wpt/ci
[taskcluster 2026-01-28 17:51:31.723Z] Worker Version: 46.1.0
[taskcluster 2026-01-28 17:51:31.723Z] Public IP: 34.172.16.87
[taskcluster 2026-01-28 17:51:31.723Z] Hostname: proj-wpt-ci-yoism-k8tvgxe8gmpvultw

[taskcluster 2026-01-28 17:51:32.701Z] === Task Starting ===
+ '[' '' ']'
+ '[' -d /etc/profile.d ']'
+ for i in /etc/profile.d/*.sh
+ '[' -r /etc/profile.d/01-locale-fix.sh ']'
+ . /etc/profile.d/01-locale-fix.sh
+++ /usr/bin/locale-check C.UTF-8
++ eval
+ for i in /etc/profile.d/*.sh
+ '[' -r /etc/profile.d/apps-bin-path.sh ']'
+ . /etc/profile.d/apps-bin-path.sh

...(4124 lines hidden)...

TEST-PASS | /infrastructure/testdriver/get_all_cookies.sub.html | Get all HTTP cookies 
16:22.58 TEST_END: Test OK. Subtests passed 1/1. Unexpected 0
TEST-OK | /infrastructure/testdriver/get_all_cookies.sub.html | took 499ms
16:22.58 TEST_START: /infrastructure/testdriver/get_all_cookies.sub.https.html
TEST-START | /infrastructure/testdriver/get_all_cookies.sub.https.html
16:22.58 INFO Closing window 223d536d-9087-45fb-b68d-6f2090c68665
Closing window 223d536d-9087-45fb-b68d-6f2090c68665
16:23.02 INFO Getting all cookies
Getting all cookies
16:23.04 INFO Deleting all cookies
Deleting all cookies
TEST-PASS | /infrastructure/testdriver/get_all_cookies.sub.https.html | Get all HTTPS cookies 
16:23.19 TEST_END: Test OK. Subtests passed 1/1. Unexpected 0
TEST-OK | /infrastructure/testdriver/get_all_cookies.sub.https.html | took 610ms
16:23.19 TEST_START: /infrastructure/testdriver/get_named_cookie.sub.html
TEST-START | /infrastructure/testdriver/get_named_cookie.sub.html
16:23.19 INFO Closing window 315054d8-b1a3-4a31-b4de-1e87a28eea1f
Closing window 315054d8-b1a3-4a31-b4de-1e87a28eea1f
16:23.70 INFO Getting cookie named test0
Getting cookie named test0
16:23.73 INFO Getting cookie named test1
Getting cookie named test1
16:23.76 INFO Getting cookie named test2
Getting cookie named test2
16:23.78 INFO Getting cookie named test3
Getting cookie named test3
16:23.79 INFO Getting cookie named test4
Getting cookie named test4
16:23.80 INFO Getting cookie named test5
Getting cookie named test5
16:23.81 INFO Getting cookie named test6
Getting cookie named test6
16:23.82 INFO Getting cookie named test7
Getting cookie named test7
16:23.84 INFO Deleting all cookies
Deleting all cookies
TEST-PASS | /infrastructure/testdriver/get_named_cookie.sub.html | Get Named HTTP cookie 
16:23.94 TEST_END: Test OK. Subtests passed 1/1. Unexpected 0
TEST-OK | /infrastructure/testdriver/get_named_cookie.sub.html | took 754ms
16:23.94 TEST_START: /infrastructure/testdriver/get_named_cookie.sub.https.html
TEST-START | /infrastructure/testdriver/get_named_cookie.sub.https.html
16:23.95 INFO Closing window b8a4de4f-6acc-4743-b350-2cdf6113c24d
Closing window b8a4de4f-6acc-4743-b350-2cdf6113c24d
16:24.52 INFO Getting cookie named test0
Getting cookie named test0
16:24.58 INFO Getting cookie named test1
Getting cookie named test1
16:24.59 INFO Getting cookie named test2
Getting cookie named test2
16:24.60 INFO Getting cookie named test3
Getting cookie named test3
16:24.61 INFO Getting cookie named test4
Getting cookie named test4
16:24.62 INFO Getting cookie named test5
Getting cookie named test5
16:24.63 INFO Getting cookie named test6
Getting cookie named test6
16:24.64 INFO Getting cookie named test7
Getting cookie named test7
16:24.65 INFO Getting cookie named test8
Getting cookie named test8
16:24.66 INFO Deleting all cookies
Deleting all cookies
TEST-PASS | /infrastructure/testdriver/get_named_cookie.sub.https.html | Get Named HTTPS cookie 
16:24.74 TEST_END: Test OK. Subtests passed 1/1. Unexpected 0
TEST-OK | /infrastructure/testdriver/get_named_cookie.sub.https.html | took 800ms
16:24.74 TEST_START: /infrastructure/testdriver/send_keys.html
TEST-START | /infrastructure/testdriver/send_keys.html
16:24.75 INFO Closing window 5a4248a1-0275-4ef9-a1fe-adc0684dfb97
Closing window 5a4248a1-0275-4ef9-a1fe-adc0684dfb97
TEST-PASS | /infrastructure/testdriver/send_keys.html | TestDriver send keys method 
16:25.46 TEST_END: Test OK. Subtests passed 1/1. Unexpected 0
TEST-OK | /infrastructure/testdriver/send_keys.html | took 717ms
16:25.46 TEST_START: /infrastructure/testdriver/set_get_window_rect.html
TEST-START | /infrastructure/testdriver/set_get_window_rect.html
16:25.47 INFO Closing window 32b80e19-50a4-4859-b36f-c4f6354f726b
Closing window 32b80e19-50a4-4859-b36f-c4f6354f726b
TEST-FAIL | /infrastructure/testdriver/set_get_window_rect.html | Window Position - promise_test: Unhandled rejection with value: "error: Action set_window_rect failed: Only supported in desktop applications\nstacktrace:\n\tRemoteError@chrome://remote/content/shared/RemoteError.sys.mjs:8:8\n\tWebDriverError@chrome://remote/content/shared/webdriver/Errors.sys.mjs:202:5\n\tUnsupportedOperationError@chrome://remote/content/shared/webdriver/Errors.sys.mjs:978:5\n\tassert.that/<@chrome://remote/content/shared/webdriver/Assert.sys.mjs:581:13\n\tassert.desktop@chrome://remote/content/shared/webdriver/Assert.sys.mjs:74:4\n\tGeckoDriver.prototype.setWindowRect@chrome://remote/content/marionette/driver.sys.mjs:1488:15\n\tdispatch@chrome://remote/content/marionette/server.sys.mjs:318:40\n\texecute@chrome://remote/content/marionette/server.sys.mjs:289:16\n\tonPacket/<@chrome://remote/content/marionette/server.sys.mjs:262:20\n\tonPacket@chrome://remote/content/marionette/server.sys.mjs:263:9\n\t_onJSONObjectReady/<@chrome://remote/content/marionette/transport.sys.mjs:494:20\n"
TEST-FAIL | /infrastructure/testdriver/set_get_window_rect.html | Window Size - promise_test: Unhandled rejection with value: "error: Action set_window_rect failed: Only supported in desktop applications\nstacktrace:\n\tRemoteError@chrome://remote/content/shared/RemoteError.sys.mjs:8:8\n\tWebDriverError@chrome://remote/content/shared/webdriver/Errors.sys.mjs:202:5\n\tUnsupportedOperationError@chrome://remote/content/shared/webdriver/Errors.sys.mjs:978:5\n\tassert.that/<@chrome://remote/content/shared/webdriver/Assert.sys.mjs:581:13\n\tassert.desktop@chrome://remote/content/shared/webdriver/Assert.sys.mjs:74:4\n\tGeckoDriver.prototype.setWindowRect@chrome://remote/content/marionette/driver.sys.mjs:1488:15\n\tdispatch@chrome://remote/content/marionette/server.sys.mjs:318:40\n\texecute@chrome://remote/content/marionette/server.sys.mjs:289:16\n\tonPacket/<@chrome://remote/content/marionette/server.sys.mjs:262:20\n\tonPacket@chrome://remote/content/marionette/server.sys.mjs:263:9\n\t_onJSONObjectReady/<@chrome://remote/content/marionette/transport.sys.mjs:494:20\n"
16:25.92 TEST_END: Test OK. Subtests passed 0/2. Unexpected 0
TEST-OK | /infrastructure/testdriver/set_get_window_rect.html | took 461ms
16:25.92 TEST_START: /infrastructure/testdriver/set_permission.https.html
TEST-START | /infrastructure/testdriver/set_permission.https.html
16:25.93 INFO Closing window 929f6cb9-f52b-4871-b4b6-5ebbb34bd901
Closing window 929f6cb9-f52b-4871-b4b6-5ebbb34bd901
TEST-PASS | /infrastructure/testdriver/set_permission.https.html | Grant Permission 
TEST-PASS | /infrastructure/testdriver/set_permission.https.html | Deny Permission 
16:26.59 TEST_END: Test OK. Subtests passed 2/2. Unexpected 0
TEST-OK | /infrastructure/testdriver/set_permission.https.html | took 667ms
16:26.59 TEST_START: /infrastructure/testdriver/virtual_authenticator.html
TEST-START | /infrastructure/testdriver/virtual_authenticator.html
16:26.60 INFO Closing window 65bb5be9-2af6-4c42-aa4d-55be519238ce
Closing window 65bb5be9-2af6-4c42-aa4d-55be519238ce
TEST-PASS | /infrastructure/testdriver/virtual_authenticator.html | Can create an authenticator 
TEST-PASS | /infrastructure/testdriver/virtual_authenticator.html | Can add a credential 
TEST-PASS | /infrastructure/testdriver/virtual_authenticator.html | Can get the credentials 
TEST-PASS | /infrastructure/testdriver/virtual_authenticator.html | Can remove a credential 
TEST-PASS | /infrastructure/testdriver/virtual_authenticator.html | Can remove all credentials 
TEST-PASS | /infrastructure/testdriver/virtual_authenticator.html | Can set user verified 
TEST-PASS | /infrastructure/testdriver/virtual_authenticator.html | Can remove a virtual authenticator 
16:27.30 TEST_END: Test OK. Subtests passed 7/7. Unexpected 0
TEST-OK | /infrastructure/testdriver/virtual_authenticator.html | took 714ms
16:27.30 TEST_START: /infrastructure/testharness/full.stop/full-stop.html
TEST-START | /infrastructure/testharness/full.stop/full-stop.html
16:27.31 INFO Closing window 2b3ddb1f-dc02-4fc3-9671-6441b985b26e
Closing window 2b3ddb1f-dc02-4fc3-9671-6441b985b26e
TEST-PASS | /infrastructure/testharness/full.stop/full-stop.html | full-stop 
16:27.67 TEST_END: Test OK. Subtests passed 1/1. Unexpected 0
TEST-OK | /infrastructure/testharness/full.stop/full-stop.html | took 366ms
16:27.67 TEST_START: /infrastructure/testharness/lone-surrogates.html
TEST-START | /infrastructure/testharness/lone-surrogates.html
16:27.68 INFO Closing window b050d68f-5f7c-496f-ac4a-85a2dd1d4ffb
Closing window b050d68f-5f7c-496f-ac4a-85a2dd1d4ffb
TEST-PASS | /infrastructure/testharness/lone-surrogates.html | passing test with lone surrogate U+d800 in name 
TEST-FAIL | /infrastructure/testharness/lone-surrogates.html | failing test with lone surrogate U+d800 in name - assert_true: expected true got false
@http://web-platform.test:8000/infrastructure/testharness/lone-surrogates.html:7:25
Test.prototype.step@http://web-platform.test:8000/resources/testharness.js:2869:25
test@http://web-platform.test:8000/resources/testharness.js:633:30
@http://web-platform.test:8000/infrastructure/testharness/lone-surrogates.html:7:7
TEST-PASS | /infrastructure/testharness/lone-surrogates.html | passing test with lone surrogate in assert 
TEST-FAIL | /infrastructure/testharness/lone-surrogates.html | failing test with lone surrogate in assert - assert_true: lone U+d800 surrogate expected true got false
@http://web-platform.test:8000/infrastructure/testharness/lone-surrogates.html:9:25
Test.prototype.step@http://web-platform.test:8000/resources/testharness.js:2869:25
test@http://web-platform.test:8000/resources/testharness.js:633:30
@http://web-platform.test:8000/infrastructure/testharness/lone-surrogates.html:9:7
16:28.06 TEST_END: Test OK. Subtests passed 2/4. Unexpected 0
TEST-OK | /infrastructure/testharness/lone-surrogates.html | took 391ms
16:28.06 TEST_START: /infrastructure/window/minimize-1.html
TEST-START | /infrastructure/window/minimize-1.html
16:28.07 INFO Closing window 98cbe35d-122f-4a43-8235-296eecf47e70
Closing window 98cbe35d-122f-4a43-8235-296eecf47e70
TEST-FAIL | /infrastructure/window/minimize-1.html | Minimize a window - promise_test: Unhandled rejection with value: "error: Action minimize_window failed: Only supported in desktop applications\nstacktrace:\n\tRemoteError@chrome://remote/content/shared/RemoteError.sys.mjs:8:8\n\tWebDriverError@chrome://remote/content/shared/webdriver/Errors.sys.mjs:202:5\n\tUnsupportedOperationError@chrome://remote/content/shared/webdriver/Errors.sys.mjs:978:5\n\tassert.that/<@chrome://remote/content/shared/webdriver/Assert.sys.mjs:581:13\n\tassert.desktop@chrome://remote/content/shared/webdriver/Assert.sys.mjs:74:4\n\tGeckoDriver.prototype.minimizeWindow@chrome://remote/content/marionette/driver.sys.mjs:3114:15\n\tdispatch@chrome://remote/content/marionette/server.sys.mjs:318:40\n\texecute@chrome://remote/content/marionette/server.sys.mjs:289:16\n\tonPacket/<@chrome://remote/content/marionette/server.sys.mjs:262:20\n\tonPacket@chrome://remote/content/marionette/server.sys.mjs:263:9\n\t_onJSONObjectReady/<@chrome://remote/content/marionette/transport.sys.mjs:494:20\n"
16:28.47 TEST_END: Test OK. Subtests passed 0/1. Unexpected 0
TEST-OK | /infrastructure/window/minimize-1.html | took 406ms
16:28.47 TEST_START: /infrastructure/window/minimize-2.html
TEST-START | /infrastructure/window/minimize-2.html
16:28.47 INFO Closing window 10c669d9-dc07-4a28-9a5f-a93c2d5fe2a9
Closing window 10c669d9-dc07-4a28-9a5f-a93c2d5fe2a9
TEST-PASS | /infrastructure/window/minimize-2.html | Use testdriver to check window is not minimized 
16:28.81 TEST_END: Test OK. Subtests passed 1/1. Unexpected 0
TEST-OK | /infrastructure/window/minimize-2.html | took 339ms
16:28.81 INFO No more tests
No more tests
16:29.41 INFO STDOUT: cleanup aborted: args: /home/test/web-platform-tests/_venv3/android/android-sdk-linux/platform-tools/adb wait-for-device remount, exitcode: 1, stdout: Device must be bootloader unlocked
STDOUT: cleanup aborted: args: /home/test/web-platform-tests/_venv3/android/android-sdk-linux/platform-tools/adb wait-for-device remount, exitcode: 1, stdout: Device must be bootloader unlocked
16:28.97 INFO Closing logging queue
Closing logging queue
16:28.97 INFO queue closed
queue closed
16:29.75 SUITE_END

web-platform-test
~~~~~~~~~~~~~~~~~
Ran 402 checks (223 subtests, 179 tests)
Expected results: 398 (1 known intermittents)
Skipped: 4 tests
Unexpected results: 0

Known Intermittent Results
--------------------------
/infrastructure/testdriver/actions/elementTiming.html
  KNOWN-INTERMITTENT-FAIL TestDriver actions: element timing - assert_array_equals: expected property 1 to be 150 but got 149 (expected array [50, 150] got [50, 149])
@http://web-platform.test:8000/infrastructure/testdriver/actions/elementTiming.html:67:22
OK
SUITE-END | took 702s
16:29.76 INFO Got 0 unexpected results, with 0 unexpected passes
Got 0 unexpected results, with 0 unexpected passes
16:29.75 INFO STDOUT: cleanup aborted: args: /home/test/web-platform-tests/_venv3/android/android-sdk-linux/platform-tools/adb wait-for-device remount, exitcode: 1, stdout: Device must be bootloader unlocked
STDOUT: cleanup aborted: args: /home/test/web-platform-tests/_venv3/android/android-sdk-linux/platform-tools/adb wait-for-device remount, exitcode: 1, stdout: Device must be bootloader unlocked
16:29.76 wptserve INFO Stopped http server on 127.0.0.1:8000
wptserve Stopped http server on 127.0.0.1:8000
16:29.76 wptserve INFO Stopped http server on 127.0.0.1:8003
wptserve Stopped http server on 127.0.0.1:8003
16:29.76 wptserve INFO Stopped http server on 127.0.0.1:8001
wptserve Stopped http server on 127.0.0.1:8001
16:29.77 wptserve INFO Stopped http server on 127.0.0.1:8002
wptserve Stopped http server on 127.0.0.1:8002
16:29.77 wptserve INFO Stopped http server on 127.0.0.1:8443
wptserve Stopped http server on 127.0.0.1:8443
16:29.78 wptserve INFO Stopped http server on 127.0.0.1:8444
wptserve Stopped http server on 127.0.0.1:8444
16:29.78 wptserve INFO Stopped http server on 127.0.0.1:8445
wptserve Stopped http server on 127.0.0.1:8445
16:29.78 wptserve INFO Stopped http server on 127.0.0.1:8446
wptserve Stopped http server on 127.0.0.1:8446
16:29.78 wptserve INFO Stopped http server on 127.0.0.1:9000
wptserve Stopped http server on 127.0.0.1:9000
16:29.91 wptserve INFO Close on: (<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('127.0.0.1', 8889))
wptserve Close on: (<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('127.0.0.1', 8889))
16:30.16 wptserve INFO Close on: (<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('127.0.0.1', 8888))
wptserve Close on: (<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('127.0.0.1', 8888))
16:30.21 INFO Removed font: Ahem.ttf
Removed font: Ahem.ttf
16:30.23 INFO Closing logging queue
Closing logging queue
16:30.23 INFO queue closed
queue closed
[taskcluster 2026-01-28 18:09:41.680Z] === Task Finished ===
[taskcluster 2026-01-28 18:09:43.704Z] Successful task run with exit code: 0 completed in 1091.981 seconds
View more details on Community-TC Integration