Skip to content

Test result updater fails when results xml file is not written by ctest. #9

@eap

Description

@eap

Description of issue

In some cases the ctest results xml fails to write (this is a known issue and affects a very small percentage of test runs but a high percentage of reported user issues). The runner bash script will correctly detect that there is no xml file even though one was expected, but our GitHub updater python program does not handle this case gracefully.

Expected behavior

The GitHub updater should note the issue and suggest a re-run.

Logs


Errors while running CTest
--
Output from these tests are in: /workdir/test_root/build/Testing/Temporary/LastTest.log
Use "--rerun-failed --output-on-failure" to re-run the failed cases verbosely.
+ ctest -C RelWithDebInfo -D ExperimentalSubmit -M Continuous -- --track Continuous --group Continuous
Site: AWSBatch
Build name: 1362;gcc11-Container;GNU_11.4.0;RelWithDebInfo
Submit files
Send to group: Experimental
SubmitURL: https://cdash.jcsda.org/submit.php?project=fv3-jedi
Uploaded: /workdir/test_root/build/Testing/20250724-1917/Done.xml
Submission successful
++ util.create_cdash_url /workdir/test_root/build/Testing
++ test_dir=/workdir/test_root/build/Testing
+++ head -1 /workdir/test_root/build/Testing/TAG
++ tag=20250724-1917
+++ cat /workdir/test_root/build/Testing/20250724-1917/Done.xml
++ Done='<?xml version="1.0" encoding="UTF-8"?>
<Done>
<buildId></buildId>
<time>1753384766</time>
</Done>'
+++ echo '<?xml' 'version="1.0"' 'encoding="UTF-8"?>' '<Done>' '<buildId></buildId>' '<time>1753384766</time>' '</Done>'
+++ grep -o -P '(?<=buildId>).*(?=</build)'
++ buildID=
++ echo 'https://cdash.jcsda.org/viewTest.php?buildid='
+ echo 'CDash URL: https://cdash.jcsda.org/viewTest.php?buildid='
CDash URL: https://cdash.jcsda.org/viewTest.php?buildid=
+ ALLOWED_UNIT_FAIL_RATE=0
+ '[' fv3-jedi = ufo ']'
+ util.check_run_end JCSDA-internal/fv3-jedi 46671148348 0
+ '[' no = yes ']'
+ repo=JCSDA-internal/fv3-jedi
+ run_id=46671148348
+ max_fail_ppc=0
++ util.create_cdash_url /workdir/test_root/build/Testing
++ test_dir=/workdir/test_root/build/Testing
+++ head -1 /workdir/test_root/build/Testing/TAG
++ tag=20250724-1917
+++ cat /workdir/test_root/build/Testing/20250724-1917/Done.xml
++ Done='<?xml version="1.0" encoding="UTF-8"?>
<Done>
<buildId></buildId>
<time>1753384766</time>
</Done>'
+++ echo '<?xml' 'version="1.0"' 'encoding="UTF-8"?>' '<Done>' '<buildId></buildId>' '<time>1753384766</time>' '</Done>'
+++ grep -o -P '(?<=buildId>).*(?=</build)'
++ buildID=
++ echo 'https://cdash.jcsda.org/viewTest.php?buildid='
+ cdash_url='https://cdash.jcsda.org/viewTest.php?buildid='
++ util.find_test_xml
+++ head -1 /workdir/test_root/build/Testing/TAG
++ test_tag=20250724-1917
++ expected_file=/workdir/test_root/build/Testing/20250724-1917/Test.xml
++ '[' -f /workdir/test_root/build/Testing/20250724-1917/Test.xml ']'
+++ find /workdir/test_root/build/Testing -type f -name Test.xml
+++ head -n1
++ found_file=
++ '[' -f '' ']'
++ echo no-file-found-for-Test.xml
+ test_xml=no-file-found-for-Test.xml
+ /workdir/CI/src/test_runner/github_api/check_run.py end --app-private-key=/tmp/tmp.ehvFZYbtWm --app-id=321361 --repo=JCSDA-internal/fv3-jedi --check-run-id=46671148348 --test-xml=no-file-found-for-Test.xml --max-failure-percentage 0 '--cdash-url=https://cdash.jcsda.org/viewTest.php?buildid=' --public-log-link=https://jedi-ci-build-public-logs.s3.us-east-2.amazonaws.com/fv3-jedi-1362-70c20f0-gcc11-6PVX3KP3.html --ecs-metadata-uri=http://169.254.170.2/v4/67ed041d-bb93-47b4-bd39-8d46bffdce7e --batch-task-id=3446f561-0a05-4afe-8ce4-e5761208ff71
/workdir/CI/src/test_runner/github_api/check_run.py:468: DeprecationWarning: Call to deprecated class AppAuthentication. (Use github.Auth.AppInstallationAuth instead)
auth = github.AppAuthentication(
Traceback (most recent call last):
File "/workdir/CI/src/test_runner/github_api/check_run.py", line 673, in <module>
args.func(args, app_id, app_key, repo_owner, repo_name)
File "/workdir/CI/src/test_runner/github_api/check_run.py", line 575, in check_run_end
results = TestOutput.from_test_xml(args.test_xml)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workdir/CI/src/test_runner/github_api/check_run.py", line 374, in from_test_xml
tree = xml.etree.ElementTree.parse(test_output_xml)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/software/linux-ubuntu24.04-x86_64/gcc-11.4.0/python-3.11.7-izgegfgwmaewrdtfdiq6fb4zcma2xqgz/lib/python3.11/xml/etree/ElementTree.py", line 1218, in parse
tree.parse(source, parser)
File "/opt/software/linux-ubuntu24.04-x86_64/gcc-11.4.0/python-3.11.7-izgegfgwmaewrdtfdiq6fb4zcma2xqgz/lib/python3.11/xml/etree/ElementTree.py", line 569, in parse
source = open(source, "rb")
^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: 'no-file-found-for-Test.xml'


~``

Metadata

Metadata

Assignees

Labels

bugSomething is not working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions