Skip to content

Conversation

@RoniKishner
Copy link

Description

This PR fixes the double-counting of subtest failures in pytest's final test summary. Previously, when a test with subtests had failures, both the subtest failure and the top-level test failure were counted as regular "failed" tests, leading to inflated failure counts.

Changes

  • Modified pytest_report_teststatus hook in src/_pytest/subtests.py to return "subtests failed" category instead of "failed" for subtest failures
  • Updated _determine_main_color in src/_pytest/terminal.py to consider "subtests failed" when determining the main color (red)
  • Enhanced parse_summary_nouns in src/_pytest/pytester.py:
    • Updated regex to handle multi-word outcomes like "subtests failed"
    • Added pluralization mapping for subtest outcomes
    • Added whitespace handling for parsed outcomes
  • Updated test expectations in testing/test_subtests.py to match the new summary format

Example

Before:
===================== 2 failed, 2 subtests passed in 0.02s =====================
After:
===================== 1 failed, 2 subtests passed, 1 subtests failed in 0.02s =====================

This provides clear visibility into:

  • How many top-level tests failed (1 failed)
  • How many subtests failed (1 subtests failed)
  • How many subtests passed (2 subtests passed)

Closes #13986

@psf-chronographer psf-chronographer bot added the bot:chronographer:provided (automation) changelog entry is part of PR label Nov 20, 2025
@RoniKishner RoniKishner force-pushed the subtests-failure-info branch from 80ecce2 to 7fa74de Compare November 20, 2025 19:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bot:chronographer:provided (automation) changelog entry is part of PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Fix double-counting of subtest failures in test summary

1 participant