Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a way to run a test multiple times in a row #832

Closed
ahoppen opened this issue May 31, 2024 · 7 comments
Closed

Add a way to run a test multiple times in a row #832

ahoppen opened this issue May 31, 2024 · 7 comments
Assignees
Labels
enhancement New feature or request

Comments

@ahoppen
Copy link
Member

ahoppen commented May 31, 2024

Sometimes, when debugging a non-deterministic test, it is useful if you can run it multiple times in a row until it fails. It would be great if the Swift Extension offered that ability.

@ahoppen ahoppen added the enhancement New feature or request label May 31, 2024
@adam-fowler
Copy link
Contributor

Using swift-testing you could do that.

@Test(arguments: 0..<100)
func testMultiple(attempt: Int){
    ...
}

@ahoppen
Copy link
Member Author

ahoppen commented Jun 1, 2024

I was thinking XCTest ;-)

@adam-fowler
Copy link
Contributor

This seems quite niche. Is there any reason you can't wrap the test in a for loop that runs it multiple times.

@ahoppen
Copy link
Member Author

ahoppen commented Jun 3, 2024

I could. I just found the Xcode feature to run a test repeatedly useful quite regularly because it also gives you a few things out of the box like logging which iteration it’s currently on.

@plemarquand
Copy link
Contributor

I've used this a few times in Xcode to diagnose flakey XCTests that have setUp/tearDown

@adam-fowler
Copy link
Contributor

I didn't know this was an option in Xcode.

@plemarquand plemarquand self-assigned this Jun 5, 2024
plemarquand added a commit to plemarquand/vscode-swift that referenced this issue Aug 12, 2024
Adds two new items to the context menu when you right click a test in
the test explorer:

- Run Multiple Times
- Run Until Failure

Selecting either of these promts with a text input where the user can
input how many times they want to run the test(s).

If the user selected Run Until Failure the tests will be run a maximum
number of times, stopping at the first iteration that produces a
failure.

A current limitation in VS Code is if you have multiple tests selected
in the Test Explorer (by shift or ctrl clicking them) only the first is
passed to the command.

Issue: swiftlang#832
plemarquand added a commit to plemarquand/vscode-swift that referenced this issue Aug 12, 2024
Adds two new items to the context menu when you right click a test in
the test explorer:

- Run Multiple Times
- Run Until Failure

Selecting either of these promts with a text input where the user can
input how many times they want to run the test(s).

If the user selected Run Until Failure the tests will be run a maximum
number of times, stopping at the first iteration that produces a
failure.

A current limitation in VS Code is if you have multiple tests selected
in the Test Explorer (by shift or ctrl clicking them) only the first is
passed to the command.

Issue: swiftlang#832
plemarquand added a commit to plemarquand/vscode-swift that referenced this issue Aug 15, 2024
Adds two new items to the context menu when you right click a test in
the test explorer:

- Run Multiple Times
- Run Until Failure

Selecting either of these promts with a text input where the user can
input how many times they want to run the test(s).

If the user selected Run Until Failure the tests will be run a maximum
number of times, stopping at the first iteration that produces a
failure.

A current limitation in VS Code is if you have multiple tests selected
in the Test Explorer (by shift or ctrl clicking them) only the first is
passed to the command.

Issue: swiftlang#832
plemarquand added a commit that referenced this issue Aug 15, 2024
Adds two new items to the context menu when you right click a test in
the test explorer:

- Run Multiple Times
- Run Until Failure

Selecting either of these promts with a text input where the user can
input how many times they want to run the test(s).

If the user selected Run Until Failure the tests will be run a maximum
number of times, stopping at the first iteration that produces a
failure.

A current limitation in VS Code is if you have multiple tests selected
in the Test Explorer (by shift or ctrl clicking them) only the first is
passed to the command.

Issue: #832
@plemarquand plemarquand added the needs verification Issue has been fixed, but requires verification before closing label Aug 15, 2024
@award999 award999 assigned award999 and unassigned plemarquand Aug 27, 2024
@award999
Copy link
Contributor

Verified

@plemarquand plemarquand removed the needs verification Issue has been fixed, but requires verification before closing label Aug 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

4 participants