feat: Enable scanning of chromium content (#967)

#### Details

In #951, we intentionally excluded all elements inside a Chromium
document from scan results. This makes sense for most developers, since
the conversion from HTML to UIA is convoluted and outside the control of
the HTML developer. However, the Edge team relies on this functionality
to help improve their HTML-to-UIA conversion code. This PR defaults to
excluding the elements inside a Chromium document, but adds a way to
request that scan results include elements within a Chromium document.
This option is exposed in 3 places:
1. In the `Automation` layer
2. In the CLI
3. In the `SelectAction` class (for future use from Accessibility
Insights for Windows)

##### Motivation

Requested by the Edge team

##### Results

[Comparison.zip](https://github.com/microsoft/axe-windows/files/12480720/Comparison.zip)
contains `Default.a11ytest` and `WithChromiumContent.a11ytest`, which
were captured by running the CLI on the same instance of Edge. The rule
comparison is as follows:

Message | Default Count | WithChromiumContent count
--- | --- | ---
An on-screen element must not have a null BoundingRectangle property | 5
| 5
Chromium components should be scanned with an web-based scanner | 2 | 2
The Name must not include the same text as the LocalizedControlType | 2
| 2
The Name property must not include the element's control type | 2 | 2
An element must not have the same Name and LocalizedControlType as its
parent | 0 | 4
An element of the given ControlType must support the Text pattern | 0 |
1

The elements that are unique to the WithChromiumContent file are all
within the Chromium document

##### Screenshots 

Case | Snapshot (click to enlarge) | Results (click to enlarge)
--- | --- | ---
Default |
![image](https://github.com/microsoft/axe-windows/assets/45672944/1665bb43-a93b-44b8-aaf8-94d4af9680ee)
|![image](https://github.com/microsoft/axe-windows/assets/45672944/e2f6cd73-8628-46e7-b7db-3f89f0d12438)
WithChromiumContent |
![image](https://github.com/microsoft/axe-windows/assets/45672944/8702cb77-fda3-40eb-9fc2-a4f360f790d3)
|
![image](https://github.com/microsoft/axe-windows/assets/45672944/79b6bae7-afc5-4e6b-8b9f-37d1356bac9f)
Delta |
![image](https://github.com/microsoft/axe-windows/assets/45672944/aa68946e-ca5c-464e-826c-1914500517a3)
|
![image](https://github.com/microsoft/axe-windows/assets/45672944/6e7da53a-4f5d-4dd6-bc43-e23e957b5a90)

##### Context

<!-- Are there any parts that you've intentionally left out-of-scope for
a later PR to handle? -->
- The `Actions` project didn't previously reference the `Rules` project
directly, so the plumbing may be non-ideal. If we were to use the
existing dependencies, `Actions` would call into `Desktop`, which would
call into `RuleSelection`, which would call into `Rules`. We can add the
2 missing layers if we think there's value there.
- The flag to enable Chromium content is basically global. This will
work as expected in the CLI, in AI-Win, and in AccChecker. It will fail
if someone creates a scenario where concurrent scans want to have
different Chromium behavior. I've taken a YAGNI approach for now, and we
can always make it more complex at a future time, should that prove
necessary.

<!-- Were there any alternative approaches you considered? What
tradeoffs did you consider? -->

#### Pull request checklist
<!-- If a checklist item is not applicable to this change, write "n/a"
in the checkbox -->
- [n/a] Addresses an existing issue:

docs/AutomationReference.md

@@ -100,6 +100,14 @@ Cause each scan to save a test file, even if no errors are reported. The default
 
 The `WithAlwaysSaveTestFile` method returns the  `Config.Builder` configured to always save a test file.
 
+##### `WithTestAllChromiumContent`
+
+Cause all Chromium HTML content to be included in scan results. The default is to omit Chromium HTML content from the scan results. This is intended only for use by browser development teams and should be omitted in other scenarios.
+
+###### Return object
+
+The `WithTestAllChromiumContent` method returns the  `Config.Builder` configured to include all Chromium HTML content in the scan results..
+
 ##### Build
 
 Build an instance of `Config`.

src/Actions/Actions.csproj

@@ -32,6 +32,7 @@
   <ItemGroup>
     <ProjectReference Include="..\Core\Core.csproj" />
     <ProjectReference Include="..\Desktop\Desktop.csproj" />
+    <ProjectReference Include="..\Rules\Rules.csproj" />
     <ProjectReference Include="..\Win32\Win32.csproj" />
   </ItemGroup>
 

src/Actions/Actions/SelectAction.cs

@@ -10,6 +10,7 @@
 using Axe.Windows.Core.Misc;
 using Axe.Windows.Desktop.Settings;
 using Axe.Windows.Desktop.UIAutomation;
+using Axe.Windows.Rules;
 using System;
 using System.Linq;
 
@@ -391,6 +392,18 @@ public static void ClearDefaultInstance()
                 DefaultInstance = null;
             }
         }
+
+        /// <summary>
+        /// Allow testing of all Chromium content
+        /// </summary>
+        public static bool ShouldTestAllChromiumContent
+        {
+            get => RulesSettings.ShouldTestAllChromiumContent;
+            set
+            {
+                RulesSettings.ShouldTestAllChromiumContent = value;
+            }
+        }
         #endregion
 
         #region IDisposable Support

src/Automation/AxeWindowsActions.cs

@@ -64,5 +64,10 @@ public void RegisterCustomUIAPropertiesFromConfig(string path)
             Core.CustomObjects.Config conf = CustomUIAAction.ReadConfigFromFile(path);
             CustomUIAAction.RegisterCustomProperties(conf.Properties);
         }
+
+        public void SetShouldTestAllChromiumContent(bool shouldTestAllChromiumContent)
+        {
+            SelectAction.ShouldTestAllChromiumContent = shouldTestAllChromiumContent;
+        }
     } // class
 } // namespace

src/Automation/Data/Config.cs

@@ -42,6 +42,9 @@ public class Config
         /// <summary>Override the default behavior of only saving a11ytest files if errors are found.</summary>
         public bool AlwaysSaveTestFile { get; private set; }
 
+        /// <summary>Override the default behavior of not testing Chromium content.</summary>
+        public bool TestAllChromiumContent { get; private set; }
+
         /// <summary>
         /// Custom handling of DPI awareness. The default handling is to set the entire process as DPI-aware
         /// before running the scan, and to leave it in that state after the scan completes. If your process
@@ -130,6 +133,15 @@ public Builder WithAlwaysSaveTestFile()
                 return this;
             }
 
+            /// <summary>
+            /// Configure Axe.Windows to test all Chromium content. By default, Chromium content is not tested.
+            /// </summary>
+            public Builder WithTestAllChromiumContent()
+            {
+                _config.TestAllChromiumContent = true;
+                return this;
+            }
+
             /// <summary>
             /// Build an instance of <see cref="Config"/>
             /// </summary>
@@ -144,6 +156,7 @@ public Config Build()
                     CustomUIAConfigPath = _config.CustomUIAConfigPath,
                     DPIAwareness = _config.DPIAwareness,
                     AlwaysSaveTestFile = _config.AlwaysSaveTestFile,
+                    TestAllChromiumContent = _config.TestAllChromiumContent,
                 };
             }
         } // Builder

src/Automation/Interfaces/IAxeWindowsActions.cs

@@ -49,5 +49,12 @@ internal interface IAxeWindowsActions
         /// </summary>
         /// <param name="path"></param>
         void RegisterCustomUIAPropertiesFromConfig(string path);
+
+        /// <summary>
+        /// Specify how Chromium content should be evaluated
+        /// </summary>
+        /// <param name="shouldTestAllChromiumContent">This can be set to true to allow browser teams to test and debug code that converts from HTML to UIA.
+        /// It should be set to false for all other scenarios.</param>
+        void SetShouldTestAllChromiumContent(bool shouldTestAllChromiumContent);
     } // interface
 } // namespace

src/Automation/SnapshotCommand.cs

@@ -97,6 +97,7 @@ private static void ScanAndProcessResults(Config config, IScanTools scanTools, L
             var dpiAwarenessObject = scanTools.DpiAwareness.Enable();
             try
             {
+                scanTools.Actions.SetShouldTestAllChromiumContent(config.TestAllChromiumContent);
                 resultList.Add(scanTools.Actions.Scan(rootElement, (element, elementId) =>
                 {
                     return ProcessResults(element, elementId, config, scanTools, multipleTopLevelWindowsExist, targetIndex, rootElements.Count(), actionContext);

src/AutomationTests/SnapshotCommandTests.cs

@@ -80,7 +80,7 @@ private void SetupDpiAwarenessMock(object dataFromEnable)
             _scanToolsMock.Setup(x => x.DpiAwareness).Returns(_dpiAwarenessMock.Object);
         }
 
-        private void SetupScanToolsMock(bool withResultsAssembler = true, bool withOutputFileHelper = false)
+        private void SetupScanToolsMock(bool withResultsAssembler = true, bool withOutputFileHelper = false, bool withShouldTestAllChromiumContent = true)
         {
             _scanToolsMock.Setup(x => x.TargetElementLocator).Returns(_targetElementLocatorMock.Object);
             _scanToolsMock.Setup(x => x.Actions).Returns(_actionsMock.Object);
@@ -92,6 +92,10 @@ private void SetupScanToolsMock(bool withResultsAssembler = true, bool withOutpu
             {
                 _scanToolsMock.Setup(x => x.OutputFileHelper).Returns(_outputFileHelperMock.Object);
             }
+            if (withShouldTestAllChromiumContent)
+            {
+                _actionsMock.Setup(x => x.SetShouldTestAllChromiumContent(false));
+            }
         }
 
         private void SetupActionsMock(string expectedPath = "")
@@ -190,7 +194,7 @@ public async Task ExecuteAsync_NullAxeWindowsActions_ThrowsException()
         [Timeout(1000)]
         public async Task ExecuteAsync_NullDpiAwareness_ThrowsException()
         {
-            SetupScanToolsMock(withResultsAssembler: false);
+            SetupScanToolsMock(withResultsAssembler: false, withShouldTestAllChromiumContent: false);
             _scanToolsMock.Setup(x => x.DpiAwareness).Returns<IDPIAwareness>(null);
 
             var action = new Func<Task>(() => SnapshotCommand.ExecuteAsync(_minimalConfig, _scanToolsMock.Object, CancellationToken.None));
@@ -204,7 +208,7 @@ public async Task ExecuteAsync_NullDpiAwareness_ThrowsException()
         [Timeout(1000)]
         public async Task ExecuteAsync_TargetElementLocatorReturnsNull_PassesNullToScan()
         {
-            SetupScanToolsMock(withResultsAssembler: false);
+            SetupScanToolsMock(withResultsAssembler: false, withShouldTestAllChromiumContent: false);
             _scanToolsMock.Setup(x => x.DpiAwareness).Returns(_dpiAwarenessMock.Object);
             SetupTargetElementLocatorMock(overrideElements: true, elements: null);
 

src/CLI/IOptions.cs

@@ -13,5 +13,6 @@ internal interface IOptions
         int DelayInSeconds { get; }
         string CustomUia { get; }
         bool AlwaysSaveTestFile { get; }
+        bool TestAllChromiumContent { get; }
     }
 }

src/CLI/Options.cs

@@ -35,6 +35,9 @@ public class Options : IOptions
         [Option(Required = false, HelpText = "AlwaysSaveTestFile", ResourceType = typeof(Resources.OptionsHelpText))]
         public bool AlwaysSaveTestFile { get; set; }
 
+        [Option(Required = false, HelpText = "TestAllChromiumContent", ResourceType = typeof(Resources.OptionsHelpText))]
+        public bool TestAllChromiumContent { get; set; }
+
         // CommandLineParser will never set this value!
         public VerbosityLevel VerbosityLevel { get; set; } = VerbosityLevel.Default;
     }

src/CLI/OptionsEvaluator.cs

@@ -65,6 +65,7 @@ public static IOptions ProcessInputs(Options rawInputs, IProcessHelper processHe
                 DelayInSeconds = delayInSeconds,
                 CustomUia = rawInputs.CustomUia,
                 AlwaysSaveTestFile = rawInputs.AlwaysSaveTestFile,
+                TestAllChromiumContent = rawInputs.TestAllChromiumContent,
             };
         }
 

src/CLI/README.MD

@@ -19,30 +19,36 @@ If you run the tool without parameters, you'll be presented with the help screen
 AxeWindowsCLI 2.2.1
 Copyright c 2020
 
-  --processid                Process Id
+  --processid                 Process Id
 
-  --processname              Process Name
+  --processname               Process Name
 
-  --outputdirectory          Output directory
+  --outputdirectory           Output directory
 
-  --scanid                   Scan ID
+  --scanid                    Scan ID
 
-  --verbosity                Verbosity level (Quiet/Default/Verbose)
+  --verbosity                 Verbosity level (Quiet/Default/Verbose)
 
-  --showthirdpartynotices    Display Third Party Notices (opens file in browser
-                             without executing scan). If specified, all other
-                             options will be ignored.
+  --showthirdpartynotices     Display Third Party Notices (opens file in browser
+                              without executing scan). If specified, all other
+                              options will be ignored.
 
-  --delayinseconds           How many seconds to delay before triggering the
-                             scan. Valid range is 0 to 60 seconds, with a
-                             default of 0.
+  --delayinseconds            How many seconds to delay before triggering the
+                              scan. Valid range is 0 to 60 seconds, with a
+                              default of 0.
 
-  --customuia                The path to a configuration file specifying custom
-                             UI Automation attributes
+  --customuia                 The path to a configuration file specifying custom
+                              UI Automation attributes
 
-  --alwayssavetestfile       If specified, always save the test file. By
-                             default, the test file is saved only if errors are
-                             found.
+  --alwayssavetestfile        If specified, always save the test file. By
+                              default, the test file is saved only if errors are
+                              found.
+
+  --testallchromiumcontent    If specified, causes Chromium HTML content to be
+                              scanned. Chromium HTML content is ignored without
+                              this flag. This is intended primarily for browser
+                              development teams and is discouraged for other
+                              scenarios.
 ```
 
 To scan an application, you need to specify the application's process via either the `--processId` or `--processName` parameters
@@ -60,6 +66,7 @@ showThirdPartyNotices|If specified, displays the third party notices for compone
 delayInSeconds|Optionally inserts a delay before triggering the scan. This allows transient controls (menus, drop-down-lists, etc.) to be scanned.
 customUia|Optionally provides a path to a [custom UIA configuration file](../../docs/CustomUIA.md). By default, only system-defined UIA properties will be included in the scan.
 alwayssavetestfile|Optionally causes the test file to always be saved. By default, the test file is saved only if errors are found.
+testAllChromiumContent|Optionally causes Chromium HTML content to be scanned. Chromium HTML content is ignored without this flag. This is intended primarily for browser development teams and is discouraged for other scenarios.
 
 ### Scan results
 A summary of scan results will be displayed after the scan is run. In addition, an `.a11ytest` file will be generated if 1 or more errors were detected or if the `alwayssavetestfile` option was specified. The location of this file will be reported in the tool output (see the documentation of the `--outputDirectory` and `--scanId` parameters for ways to alter the name or location of the output file). This file can then be opened with **Accessibility Insights for Windows**, which is freely available at http://accessibilityinsights.io. If you scan an application with multiple top-level windows, an `.a11ytest` file will be generated for _each_ top-level window, even if no errors are detected.

src/CLI/Resources/OptionsHelpText.resx

@@ -141,6 +141,9 @@
   <data name="ShowThirdPartyNotices" xml:space="preserve">
     <value>Display Third Party Notices (opens file in browser without executing scan). If specified, all other options will be ignored.</value>
   </data>
+  <data name="TestAllChromiumContent" xml:space="preserve">
+    <value>If specified, causes Chromium HTML content to be scanned. Chromium HTML content is ignored without this flag. This is intended primarily for browser development teams and is discouraged for other scenarios.</value>
+  </data>
   <data name="Verbosity" xml:space="preserve">
     <value>Verbosity level (Quiet/Default/Verbose)</value>
   </data>

src/CLI/ScanRunner.cs

@@ -28,6 +28,9 @@ private static IScanner BuildScanner(IOptions options)
             if (options.AlwaysSaveTestFile)
                 builder = builder.WithAlwaysSaveTestFile();
 
+            if (options.TestAllChromiumContent)
+                builder = builder.WithTestAllChromiumContent();
+
             return ScannerFactory.CreateScanner(builder.Build());
         }
     }

src/CLITests/OptionsEvaluatorTests.cs

@@ -32,7 +32,7 @@ private void VerifyAllMocks()
         private static void ValidateOptions(IOptions options, string processName = TestProcessName,
             int processId = TestProcessId, string outputDirectory = null, string scanId = null,
             VerbosityLevel verbosityLevel = VerbosityLevel.Default, int delayInSeconds = 0,
-            bool alwaysWriteTestFile = false)
+            bool alwaysWriteTestFile = false, bool testAllChromiumContent = false)
         {
             Assert.AreEqual(processName, options.ProcessName);
             Assert.AreEqual(processId, options.ProcessId);
@@ -41,6 +41,7 @@ private static void ValidateOptions(IOptions options, string processName = TestP
             Assert.AreEqual(verbosityLevel, options.VerbosityLevel);
             Assert.AreEqual(delayInSeconds, options.DelayInSeconds);
             Assert.AreEqual(alwaysWriteTestFile, options.AlwaysSaveTestFile);
+            Assert.AreEqual(testAllChromiumContent, options.TestAllChromiumContent);
         }
 
         [TestMethod]
@@ -304,5 +305,20 @@ public void ProcessInputs_SpecifiesAlwaysSaveTestFile_SetsSaveTestFile()
                 processId: TestProcessId, alwaysWriteTestFile: true);
             VerifyAllMocks();
         }
+
+        [TestMethod]
+        [Timeout(1000)]
+        public void ProcessInputs_SpecifiesTestAllChromiumContent_SetsTestAllChromiumContent()
+        {
+            _processHelperMock.Setup(x => x.ProcessIdFromName(TestProcessName)).Returns(TestProcessId);
+            Options input = new Options
+            {
+                ProcessName = TestProcessName,
+                TestAllChromiumContent = true,
+            };
+            ValidateOptions(OptionsEvaluator.ProcessInputs(input, _processHelperMock.Object),
+                processId: TestProcessId, testAllChromiumContent: true);
+            VerifyAllMocks();
+        }
     }
 }

src/Rules/AssemblyInfo.cs

@@ -12,9 +12,11 @@
 [assembly: NeutralResourcesLanguage("en-US")]
 
 #if ENABLE_SIGNING
+[assembly: InternalsVisibleTo("Axe.Windows.Actions,PublicKey=002400000480000094000000060200000024000052534131000400000100010007d1fa57c4aed9f0a32e84aa0faefd0de9e8fd6aec8f87fb03766c834c99921eb23be79ad9d5dcc1dd9ad236132102900b723cf980957fc4e177108fc607774f29e8320e92ea05ece4e821c0a5efe8f1645c4c0c93c1ab99285d622caa652c1dfad63d745d6f2de5f17e5eaf0fc4963d261c8a12436518206dc093344d5ad293")]
 [assembly: InternalsVisibleTo("RulesTests,PublicKey=002400000480000094000000060200000024000052534131000400000100010007d1fa57c4aed9f0a32e84aa0faefd0de9e8fd6aec8f87fb03766c834c99921eb23be79ad9d5dcc1dd9ad236132102900b723cf980957fc4e177108fc607774f29e8320e92ea05ece4e821c0a5efe8f1645c4c0c93c1ab99285d622caa652c1dfad63d745d6f2de5f17e5eaf0fc4963d261c8a12436518206dc093344d5ad293")]
 [assembly:InternalsVisibleTo("DynamicProxyGenAssembly2,PublicKey=0024000004800000940000000602000000240000525341310004000001000100c547cac37abd99c8db225ef2f6c8a3602f3b3606cc9891605d02baa56104f4cfc0734aa39b93bf7852f7d9266654753cc297e7d2edfe0bac1cdcf9f717241550e0a7b191195b7667bb4f64bcb8e2121380fd1d9d46ad2d92d2d15605093924cceaf74c4861eff62abf69b9291ed0a340e113be11e6a7d3113e92484cf7045cc7")]
 #else
+[assembly: InternalsVisibleTo("Axe.Windows.Actions")]
 [assembly: InternalsVisibleTo("RulesTests")]
 [assembly: InternalsVisibleTo("DynamicProxyGenAssembly2")]
 #endif

src/Rules/PropertyConditions/ElementGroups.cs

@@ -19,6 +19,10 @@ namespace Axe.Windows.Rules.PropertyConditions
     /// </summary>
     static class ElementGroups
     {
+        /// <summary>
+        /// This can be set to true allow browser teams to test and debug code that converts from HTML to UIA.
+        /// It should be set to false for all other scenarios.
+        /// </summary>
         // the following occurs for xaml expand/collapse controls
         private static readonly Condition FocusableGroup = Group & IsKeyboardFocusable & (WPF | XAML);
 
@@ -48,6 +52,8 @@ static class ElementGroups
         public static Condition WinFormsEdit = Edit & WinForms;
         public static Condition IsChromiumDocument = Chrome & Document;
         public static Condition IsChromiumContent = (IsChromiumDocument | AnyAncestor(IsChromiumDocument))[ConditionDescriptions.IsChromiumContent];
+        public static ValueCondition<bool> ShouldTestAllChromiumContentValueCondition = new ValueCondition<bool>((e) => RulesSettings.ShouldTestAllChromiumContent, "TestAllChromiumContent");
+        public static Condition ShouldTestAllChromiumContent = ShouldTestAllChromiumContentValueCondition == true;
         public static Condition AllowSameNameAndControlType = CreateAllowSameNameAndControlTypeCondition();
 
         private static Condition CreateMinMaxCloseButtonCondition()

src/Rules/Rule.cs

@@ -20,7 +20,7 @@ abstract class Rule : IRule
     {
         public RuleInfo Info { get; private set; }
         public Condition Condition { get; }
-        protected static Condition DefaultExcludedCondition => IsChromiumContent;
+        protected static Condition DefaultExcludedCondition => IsChromiumContent - ShouldTestAllChromiumContent;
 
         protected Rule(Condition excludedCondition)
         {

src/Rules/RulesSettings.cs

@@ -0,0 +1,21 @@
+// Copyright (c) Microsoft. All rights reserved.
+// Licensed under the MIT license. See LICENSE file in the project root for full license information.
+
+namespace Axe.Windows.Rules
+{
+    /// <summary>
+    /// This class contains settings that influence the behavior of the rules engine. It is currently
+    /// static for simplicity. This is a potential problem in the scenario of concurrent scans where the
+    /// settings change in the middle of a scan. Given the very limited scope of the current settings,
+    /// we are choosing to accept this limitation for now.
+    /// </summary>
+    internal static class RulesSettings
+    {
+        /// <summary>
+        /// This setting controls whether the rules engine should test all Chromium content. This exists
+        /// to allow browser teams to debug code that converts from HTML to UIA. It should be set to false
+        /// in all other scenarios.
+        /// </summary>
+        internal static bool ShouldTestAllChromiumContent { get; set; }
+    }
+}