Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve stepwise to not forget failed tests #13122

Merged
merged 7 commits into from
Jan 17, 2025
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Add --stepwise-reset to restart the stepwise workflow
nicoddemus committed Jan 14, 2025
commit 40b38f97c977e51f165aa32ccfe95ec3a0e5f8c8
4 changes: 3 additions & 1 deletion changelog/13122.improvement.rst
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
Improve the ``--stepwise`` flag to not forget the last failed test in case pytest is executed later without the flag.
Improve the ``--stepwise``/``--sw`` flag to not forget the last failed test in case pytest is executed later without the flag.

This enables the following workflow:

@@ -9,3 +9,5 @@ This enables the following workflow:
failed test, and if it passes, continue with the next tests.

Previously, at step 3, pytest would start from the beginning, forgetting the failed tests.

Also added the new ``--stepwise-reset``/``--sw-reset``, allowing the user to explicitly reset the stepwise state and restart the workflow from the beginning.
16 changes: 15 additions & 1 deletion src/_pytest/stepwise.py
Original file line number Diff line number Diff line change
@@ -30,11 +30,23 @@ def pytest_addoption(parser: Parser) -> None:
help="Ignore the first failing test but stop on the next failing test. "
"Implicitly enables --stepwise.",
)
group.addoption(
"--sw-reset",
"--stepwise-reset",
action="store_true",
default=False,
dest="stepwise_reset",
help="Resets stepwise state, restarting the stepwise workflow. "
"Implicitly enables --stepwise.",
)


def pytest_configure(config: Config) -> None:
# --stepwise-skip implies stepwise.
if config.option.stepwise_skip:
# allow --stepwise-skip to work on its own merits.
config.option.stepwise = True
# --stepwise-clear implies stepwise.
if config.option.stepwise_reset:
config.option.stepwise = True
if config.getoption("stepwise"):
config.pluginmanager.register(StepwisePlugin(config), "stepwiseplugin")
@@ -58,6 +70,8 @@ def __init__(self, config: Config) -> None:
self.cache: Cache = config.cache
self.lastfailed: str | None = self.cache.get(STEPWISE_CACHE_DIR, None)
self.skip: bool = config.getoption("stepwise_skip")
if config.getoption("stepwise_reset"):
self.lastfailed = None

def pytest_sessionstart(self, session: Session) -> None:
self.session = session
31 changes: 31 additions & 0 deletions testing/test_stepwise.py
Original file line number Diff line number Diff line change
@@ -396,3 +396,34 @@ def test_3():
result.stdout.fnmatch_lines(
["*::test_2 - assert False*", "*failed, continuing from this test next run*"]
)


def test_stepwise_reset(pytester: Pytester) -> None:
pytester.makepyfile(
"""
def test_1():
pass
def test_2():
assert False
def test_3():
pass
"""
)
result = pytester.runpytest("--stepwise", "-v")
result.stdout.fnmatch_lines(
[
"*::test_1 *PASSED*",
"*::test_2 *FAILED*",
"*failed, continuing from this test next run*",
]
)

# Running with --stepwise-reset restarts the stepwise workflow.
result = pytester.runpytest("-v", "--stepwise-reset")
result.stdout.fnmatch_lines(
[
"*::test_1 *PASSED*",
"*::test_2 *FAILED*",
"*failed, continuing from this test next run*",
]
)