Repository navigation
Commit bf8144b
Rahim Abdi
Automation: Add support for GetAccessibilityPropertiesForElement, GetAccessibilityPropertiesForAccessibilityNode commands
https://bugs.webkit.org/show_bug.cgi?id=299508
rdar://161303091
Reviewed by NOBODY (OOPS!).
This patch implements two new WebDriver automation commands used to support more robust WPT accessibility testing:
- GetAccessibilityPropertiesForElement
- GetAccessibilityPropertiesForAccessibilityNode
For this initial implementation, the Automation protocol returns an accessibility property bag (surfaced to WPT via
test_driver.get_accessibility_properties...()) containing the following computed properties:
- accessibilityNodeId
- role
- label
- checked ("true", "false", or "mixed"; omitted if the node does not support a checked state)
- pressed ("true", "false", or "mixed"; omitted if the node is not a toggle button)
- children (array of accessibility node handles)
- parent (accessibility node handle; omitted if the node has no parent)
As these new WebDriver commands are test-only interfaces, there's been in-depth discussion around how to treat computed
AX properties when a property is missing and supported, or present & unsupported; see WICG/aom#203
Some additional resources:
- WebDriver PR for the new commands: w3c/webdriver#1960
- ARIA PR for the initial set of computed accessibility properties, and treatment of undefined/missing/unsupported values: w3c/aria#2800
At a high level, the WebDriver client (i.e., Web Platform Tests' testdriver.js) initiates these new commands, which are then
dispatched to the UIProcess (WebAutomationSession). The WebAutomationSession then parses the payload and extracts the
protocol-specific element or accessibility node, after which it identifies the target (WebPageProxy), which invokes an
asynchronous IPC message to the corresponding WebProcess.
Since an AXID is only unique within a single WebProcess, accessibility nodes are identified by opaque handles that also
encode the hosting process and frame. The UIProcess validates the handle (frame still
exists, belongs to the page, and is still hosted by that process) and routes GetAccessibilityPropertiesForAccessibilityNode to
the process containing that frame; the WebProcess only resolves the AXID within that frame's document. Parent and children
are computed from the unignored tree using the cross-frame accessors (now exported from WebCore), so they can cross frames
hosted by the same process. With site isolation, a frame hosted by another process appears as a childless RemoteFrame node;
walking across process boundaries is not yet supported and will be addressed in a follow-up: https://bugs.webkit.org/show_bug.cgi?id=327631.
The WebProcess returns the computed accessibility properties to the UIProcess as a typed "ComputedAccessibilityProperties" struct
(Source/WebKit/Shared/Automation), using frame-qualified AXIDs and an AccessibilityTriState enum rather than strings; the UIProcess
converts these to strings when building the protocol object. WebCore::AXID is registered as an IPC-serialized identifier in messages.py.
Note that to fully support these new WebDriver commands, corresponding WebDriver HTTP endpoints must be implemented in Webdriver for Safari (safaridriver):
- /session/{session id}/element/{element id}/accessibilityproperties: Returns accessibility properties for DOM element {element id}
- /session/{session id}/accessibility/properties/{accessibility id}: Returns accessibility properties for accessibility node {accessibility id}
Due to the existence of two AccessibilityProperties protocol object definitions, documentation for each has also been updated:
- Source/WebKit/UIProcess/Automation/Automation.json — specification conformance and interoperability testing
- Source/JavaScriptCore/inspector/protocol/DOM.json — Web Inspector DOM element inspection
As an unrelated small fix, the return type for GetComputedLabel (WebAutomationSessionProxy IPC message) incorrectly names the first string parameter as "role"
instead of "label" so this patch also corrects the tuple to return (label, errorType).
New API tests (WebAutomationAccessibility) cover a cross-origin iframe with and without site isolation, checked/pressed states,
forged and stale handles, and malformed handles.
* Source/JavaScriptCore/inspector/protocol/DOM.json:
* Source/WebCore/accessibility/AXCoreObject.h:
* Source/WebKit/CMakeLists.txt:
* Source/WebKit/DerivedSources-input.xcfilelist:
* Source/WebKit/DerivedSources.make:
* Source/WebKit/Scripts/webkit/messages.py:
(serialized_identifiers):
(types_that_cannot_be_forward_declared):
(headers_for_type):
* Source/WebKit/Scripts/webkit/tests/MessageArgumentDescriptions.cpp:
(IPC::serializedIdentifiers):
* Source/WebKit/Shared/Automation/ComputedAccessibilityProperties.h: Added.
* Source/WebKit/Shared/Automation/ComputedAccessibilityProperties.serialization.in: Added.
* Source/WebKit/UIProcess/Automation/Automation.json:
* Source/WebKit/UIProcess/Automation/WebAutomationSession.cpp:
(WebKit::makeAccessibilityNodeHandle):
(WebKit::parseAccessibilityNodeHandle):
(WebKit::stringForAccessibilityTriState):
(WebKit::makeAccessibilityPropertiesCompletionHandler):
(WebKit::WebAutomationSession::getAccessibilityPropertiesForElement):
(WebKit::WebAutomationSession::getAccessibilityPropertiesForAccessibilityNode):
* Source/WebKit/UIProcess/Automation/WebAutomationSession.h:
* Source/WebKit/UIProcess/WebPageProxy.cpp:
* Source/WebKit/WebKit.xcodeproj/project.pbxproj:
* Source/WebKit/WebProcess/Automation/WebAutomationSessionProxy.cpp:
(WebKit::performDeferredAccessibilityCacheUpdates):
(WebKit::WebAutomationSessionProxy::getAccessibilityObjectForAXID):
(WebKit::toAccessibilityTriState):
(WebKit::accessibilityNodeReference):
(WebKit::getComputedAccessibilityProperties):
(WebKit::WebAutomationSessionProxy::getAccessibilityPropertiesForElement):
(WebKit::WebAutomationSessionProxy::getAccessibilityPropertiesForAccessibilityNode):
* Source/WebKit/WebProcess/Automation/WebAutomationSessionProxy.h:
* Source/WebKit/WebProcess/Automation/WebAutomationSessionProxy.messages.in:
* Tools/TestWebKitAPI/SourcesCocoa.txt:
* Tools/TestWebKitAPI/Tests/WebKit/WKWebView/WebAutomationAccessibility.mm: Added.
(TestWebKitAPI::decodeAccessibilityNodeHandle):
(TestWebKitAPI::encodeAccessibilityNodeHandle):
(TestWebKitAPI::frameIDForFrameInfo):
(TestWebKitAPI::WebAutomationAccessibility::WebAutomationAccessibility):
(TestWebKitAPI::WebAutomationAccessibility::~WebAutomationAccessibility):
(TestWebKitAPI::WebAutomationAccessibility::sendCommand):
(TestWebKitAPI::WebAutomationAccessibility::frameHandleForChildFrame):
(TestWebKitAPI::WebAutomationAccessibility::nodeHandleForElement):
(TestWebKitAPI::WebAutomationAccessibility::propertiesForElement):
(TestWebKitAPI::WebAutomationAccessibility::propertiesForAccessibilityNode):
(TestWebKitAPI::WebAutomationAccessibility::removeIframe):
(TestWebKitAPI::WebAutomationAccessibility::mainFrameID const):
(TestWebKitAPI::WebAutomationAccessibility::childFrameID const):
(TestWebKitAPI::WebAutomationAccessibility::mainFramePID const):
(TestWebKitAPI::WebAutomationAccessibility::childFramePID const):
(TestWebKitAPI::properties):
(TestWebKitAPI::errorMessage):
(TestWebKitAPI::expectError):
(TestWebKitAPI::walkAccessibilityTree):
(TestWebKitAPI::runRoundTripTest):
(TestWebKitAPI::TEST(WebAutomationAccessibility, CrossOriginIframe)):
(TestWebKitAPI::TEST(WebAutomationAccessibility, CrossOriginIframeWithSiteIsolation)):
(TestWebKitAPI::TEST(WebAutomationAccessibility, CheckedAndPressedStates)):
(TestWebKitAPI::TEST(WebAutomationAccessibility, MalformedAccessibilityNodeHandle)):1 parent 733b18f commit bf8144b
19 files changed
Lines changed: 855 additions & 4 deletions
File tree
- Source
- JavaScriptCore/inspector/protocol
- WebCore/accessibility
- WebKit
- Scripts/webkit
- tests
- UIProcess
- Automation
- WebKit.xcodeproj
- WebProcess/Automation
- Tools/TestWebKitAPI
- Tests/WebKit/WKWebView
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
101 | 101 | | |
102 | 102 | | |
103 | 103 | | |
104 | | - | |
| 104 | + | |
105 | 105 | | |
106 | 106 | | |
107 | 107 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
901 | 901 | | |
902 | 902 | | |
903 | 903 | | |
904 | | - | |
| 904 | + | |
905 | 905 | | |
906 | 906 | | |
907 | 907 | | |
908 | 908 | | |
909 | 909 | | |
910 | | - | |
| 910 | + | |
911 | 911 | | |
912 | 912 | | |
913 | 913 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
606 | 606 | | |
607 | 607 | | |
608 | 608 | | |
| 609 | + | |
| 610 | + | |
609 | 611 | | |
610 | 612 | | |
611 | 613 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
249 | 249 | | |
250 | 250 | | |
251 | 251 | | |
| 252 | + | |
252 | 253 | | |
253 | 254 | | |
254 | 255 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
705 | 705 | | |
706 | 706 | | |
707 | 707 | | |
| 708 | + | |
708 | 709 | | |
709 | 710 | | |
710 | 711 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
437 | 437 | | |
438 | 438 | | |
439 | 439 | | |
| 440 | + | |
440 | 441 | | |
441 | 442 | | |
442 | 443 | | |
| |||
620 | 621 | | |
621 | 622 | | |
622 | 623 | | |
| 624 | + | |
623 | 625 | | |
624 | 626 | | |
625 | 627 | | |
| |||
1388 | 1390 | | |
1389 | 1391 | | |
1390 | 1392 | | |
| 1393 | + | |
1391 | 1394 | | |
1392 | 1395 | | |
1393 | 1396 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
512 | 512 | | |
513 | 513 | | |
514 | 514 | | |
| 515 | + | |
515 | 516 | | |
516 | 517 | | |
517 | 518 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2 | 2 | | |
3 | 3 | | |
4 | 4 | | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
5 | 19 | | |
6 | 20 | | |
7 | 21 | | |
| |||
736 | 750 | | |
737 | 751 | | |
738 | 752 | | |
| 753 | + | |
| 754 | + | |
| 755 | + | |
| 756 | + | |
| 757 | + | |
| 758 | + | |
| 759 | + | |
| 760 | + | |
| 761 | + | |
| 762 | + | |
| 763 | + | |
| 764 | + | |
| 765 | + | |
| 766 | + | |
| 767 | + | |
| 768 | + | |
| 769 | + | |
| 770 | + | |
| 771 | + | |
| 772 | + | |
| 773 | + | |
| 774 | + | |
| 775 | + | |
| 776 | + | |
| 777 | + | |
739 | 778 | | |
740 | 779 | | |
741 | 780 | | |
| |||
0 commit comments