Repository navigation
subtests not itemized in Junit XML #13876
Description
Testing with tip-of-main to try out new subtests integration. Simple subtest example
def test_subtests(subtests):
"""Simple subtest example"""
for i in range(2):
with subtests.test(msg="is_even", i=i):
assert i % 2 == 0, f"{i} is odd"
I expect each subtest to appear as a in a generated JUnit XML. Instead, only the top level is:
<testcase classname="samples.test_sample" name="test_subtests" time="0.006">
<failure message="AssertionError: 1 is odd assert (1 % 2) == 0">subtests = <_pytest.subtests.Subtests object at 0x103b1d580>
def test_subtests(subtests):
"""Simple subtest example"""
for i in range(2):
with subtests.test(msg="is_even", i=i):
> assert i % 2 == 0, f"{i} is odd"
E AssertionError: 1 is odd
E assert (1 % 2) == 0
samples/test_sample.py:87: AssertionError</failure><system-out>--------------------------------- Captured Log ---------------------------------
</system-out>
<failure message="contains 1 failed subtest">contains 1 failed subtest</failure>
<system-out>--------------------------------- Captured Log ---------------------------------
</system-out>
</testcase>
- a detailed description of the bug or problem you are having
- output of
pip listfrom the virtual environment you are using - pytest and operating system versions
- minimal example if possible
Reacted by Thomas Hickman and Sergei Patiakin
Activity
codambro commentedon Nov 4, 2025
I can see in junitxml, there is some caching by nodeid
A SubtestReport will have the same nodeid as the parent. So when we call "_opentestcase" for each subtest, they all use the same cached NodeReporter
nicoddemus commentedon Nov 4, 2025
Hi @codambro,
Indeed, junitxml currently does not report subtests.
I wonder how would we report the subtests considering the schema? https://github.com/jenkinsci/xunit-plugin/blob/master/src/main/resources/org/jenkinsci/plugins/xunit/types/model/xsd/junit-10.xsd
pytest/src/_pytest/junitxml.py
Lines 2 to 9 in 76910ca
codambro commentedon Nov 10, 2025
Since this should be a drop-in replacement for unittest, I reviewed the unittest-xml-reporting plugin to see how it handles subtests in their JUnit XML:
I don't love their handling. They only list subtests if there is a failed one. So either:
xmlrunner README does call out that it is considered "limited support of subtests". So IMHO, this is a chance for pytest to improve, since there are open issues related to xmlrunner showing community desire for it. By reporting ALL subtests AND the parent test, pass or fail. This way reports are consistent between runs.
nicoddemus commentedon Nov 10, 2025
So if I understand correctly, it reports failed subtests as normal
testcasefailures? I guess that's the best you can do that still fits the schema.codambro commentedon Dec 2, 2025
currently, yes. I'd advocate reporting ALL subtests as normal "testcase" entries, pass or fail.
amit-gridware commentedon Jun 1, 2026
cjyhjy commentedon Oct 21, 2026
I would like to work on the JUnit XML reporting side of this. I plan to investigate separate, schema-valid testcase entries for subtests, with stable names and correct parent/subtest accounting, and add pytester coverage for passing, failing, skipped, and unittest-style subtests while keeping ordinary test output compatible.
I did not find an open PR for this issue in the linked activity or current JUnit/subtest PR search. Please let me know if someone is already implementing it or if there are naming/accounting constraints to follow. Development will use Codex assistance; the resulting patch will need human review before submission.