wpt: Use Webdriver for all WPT runs #41511
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
servo-highfive
added
the
S-awaiting-review
|
🔨 Triggering try run (#20491626788) for Linux (WPT) |
Contributor
Author
|
I took web-platform-tests/wpt@7496300 as inspiration as well as the other browser configurations to know what's the minimal set of configuration to make things work. |
Collaborator
|
🤖 Opened new upstream WPT pull request (web-platform-tests/wpt#56920) with upstreamable changes. |
All other browsers use a single configuration for their browser invocations on WPT. Servo historically had two: servo and servodriver. Now that we run WPT on Servo GitHub CI with Webdriver using the servodriver, we can align our configuration with theirs. To do so, we drop most of the custom implementation. The existing WPT configuration for the Servo binary duplicates some work in the executor that the browser configuration does instead. Therefore, we now configure preferences and such in the browser configuration rather than the executor. We also use the existing implementation for the servodriver configuration for the servo browser. The reason that we don't switch to servodriver always is that on wpt.fyi we use the Servo product and that's the one that also daily runs. Since we don't run servodriver externally and it isn't intended either, I think it's better to configure Servo probably. This does mean that in a follow-up PR we can remove the full servodriver WPT configuration. However, to make sure the existing setup works, we do that first and delete the servodriver WPT configuration once we are confident everything is stable. All in all, this ensures that both on local, Servo GitHub CI and on wpt.fyi we all use the exact same configuration. I tested this locally by running the following test: ``` ./mach test-wpt tests/wpt/tests/css/css-overflow/scrollbar-gutter-dynamic-004.html ``` This does times out with the servo binary and works with the servodriver binary. Fixes servo#40751 Signed-off-by: Tim van der Lippe <[email protected]>
TimvdLippe
force-pushed
the
use-webdriver-by-default
branch
from
07fec85 to
00e2eae
Compare
|
🔨 Triggering try run (#20491789965) for Linux (WPT) |
Collaborator
|
📝 Transplanted new upstreamable changes to existing upstream WPT pull request (web-platform-tests/wpt#56920). |
Member
|
As someone who regularly uses the servo product to make stepping through tests in debugger easier, I'm nervous about removing that configuration. |
Contributor
Author
|
Test results for linux-wpt from try job (#20491789965): Flaky unexpected result (36)
Stable unexpected results that are known to be intermittent (26)
|
|
✨ Try run (#20491789965) succeeded. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
All other browsers use a single configuration for their browser invocations on WPT. Servo historically had two: servo and servodriver. Now that we run WPT on Servo GitHub CI with Webdriver using the servodriver, we can align our configuration with theirs.
To do so, we drop most of the custom implementation. The existing WPT configuration for the Servo binary duplicates some work in the executor that the browser configuration does instead. Therefore, we now configure preferences and such in the browser configuration rather than the executor.
We also use the existing implementation for the servodriver configuration for the servo browser. The reason that we don't switch to servodriver always is that on wpt.fyi we use the Servo product and that's the one that also daily runs. Since we don't run servodriver externally and it isn't intended either, I think it's better to configure Servo probably.
This does mean that in a follow-up PR we can remove the full servodriver WPT configuration. However, to make sure the existing setup works, we do that first and delete the servodriver WPT configuration once we are confident everything is stable.
All in all, this ensures that both on local, Servo GitHub CI and on wpt.fyi we all use the exact same configuration. I tested this locally by running the following test:
This does times out with the servo binary and works with the servodriver binary.
Fixes #40751