-
-
Notifications
You must be signed in to change notification settings - Fork 126
Open
Labels
bugSomething isn't workingSomething isn't working
Description
Describe the bug
When running iOS tests we have detected failed tests with no content in the failure tag:
...
<testcase name="testAccessibility" time="13.387" classname="TestTarget.Tests">
<failure></failure>
</testcase>
...
To Reproduce
Steps to reproduce the behaviour:
- Full configuration
# Check Marathon documentation in:
# https://docs.marathonlabs.io/intro/configure
# https://github.com/MarathonLabs
name: "E2E tests"
outputDir: "./reports"
screenRecordingPolicy: "ON_FAILURE"
deviceInitializationTimeoutMillis: 480000
debug: true
uncompletedTestRetryQuota: 0
batchingStrategy:
type: "fixed-size"
size: 10
shardingStrategy:
type: "count"
count: 1
executionStrategy:
mode: ALL_SUCCESS
fast: true
vendorConfiguration:
type: "iOS"
bundle:
application: "./App.app"
testApplication: "./Tests.xctest"
testType: "xcuitest"
xcresult:
pull: true
remoteClean: true
xcodebuildTestArgs:
"-test-timeouts-enabled": "YES"
"-maximum-test-execution-time-allowance": "180"
testParserConfiguration:
type: "xctest"
screenRecordConfiguration:
preferableRecorderType: "screenshot"
compactOutput: true
permissions:
grant:
- all
devices: "./Marathondevices"
outputConfiguration:
maxPath: 1024
filteringConfiguration:
allowlist:
- type: "method"
values:
- "testAccessibility"
- Any additional information about the device provider (devices that were stuck, etc.)
- The error is because the log from the xcrun does not indicate the class throwing / causing the test to fail, and it just prints
<unknown>:0
instead ofclass/path/:line
format. - It seems to happen consistently when an a11y test fails.
- The error is because the log from the xcrun does not indicate the class throwing / causing the test to fail, and it just prints
Expected behavior
The error message displayed on the logs and the xcresult instead of an empty tag.
Logs and reports
...
<DebugLogPrinter> <unknown>:0: error: -[TestTarget.Tests testAccessibility] : Element has no description
<DebugLogPrinter> t = 4.79s Find the Any (Descendant with identity binding to AX element pid: 62723, elementOrHash.elementID: 105553170498336.190)
<DebugLogPrinter> t = 4.80s Checking existence of `"empty state view" StaticText`
<DebugLogPrinter> t = 4.81s Find the "empty state view" StaticText
...
Devices (please complete the following information):
- Device: iPhone 16
- OS: iOS 18
Additional context
PR with an initial proposal to fix the issue: #1000
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't working