Skip to content

Commit

Permalink
List all non-passing tests in test summaries
Browse files Browse the repository at this point in the history
This changes the default in pyproject.toml so that pytest lists
a line for each non-passing test at the end of a run, showing the
test name and, where available, condensed information about the
status, such as the "reason" argument for an xfailing or skipped
test.

Previously only failed and errored tests were listed in the
summary. Now skipped, xfailed, and xpassed tests are listed too.
The benefit is in keeping track of the status of tests. Although
showing the full failure output with stack trace and relevant code
under test would be too distracting for tests marked xfail, it is
valuable to not merely run those tests but be able to see a line
showing their names and statuses. Likewise, a number of tests are
currently marked skipped, and while some of them are skipped on a
particular platform because they don't make sense to run on that
platform, a number of others are skipped by raising SkipTest in
response to a failure condition on Windows. (Those consist mostly
of the tests skipped as a result of code discussed in #790.)

This also has the more specific benefit of making it easier to mark
tests as xfail in order to add CI jobs for native Windows, and more
importantly to allow information about their status to later be
used to understand and fix bugs on Windows.)
  • Loading branch information
EliahKagan committed Nov 14, 2023
1 parent 74cc671 commit 359116b
Showing 1 changed file with 3 additions and 3 deletions.
6 changes: 3 additions & 3 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ requires = ["setuptools"]
build-backend = "setuptools.build_meta"

[tool.pytest.ini_options]
addopts = "--cov=git --cov-report=term --disable-warnings"
addopts = "--cov=git --cov-report=term --disable-warnings -ra"
filterwarnings = "ignore::DeprecationWarning"
python_files = "test_*.py"
tmp_path_retention_policy = "failed"
Expand All @@ -14,8 +14,8 @@ testpaths = "test" # Space separated list of paths from root e.g test tests doc
# --cov-report html:path # html file at path
# --maxfail # number of errors before giving up
# -disable-warnings # Disable pytest warnings (not codebase warnings)
# -rf # increased reporting of failures
# -rE # increased reporting of errors
# -rfE # default test summary: list fail and error
# -ra # test summary: list all non-passing (fail, error, skip, xfail, xpass)
# --ignore-glob=**/gitdb/* # ignore glob paths
# filterwarnings ignore::WarningType # ignores those warnings

Expand Down

0 comments on commit 359116b

Please sign in to comment.