PYTHON-5075 Convert test.test_csot to async #2088
Conversation
Co-authored-by: Noah Stapp <[email protected]>
|
The async CSOT tests are continuing to be extremely flakey, even on consecutive runs of the same variant and platform. Some failures are due to the test runner never picking up |
|
|
||
| if "csot" in self.id().lower(): | ||
| # Retry CSOT tests up to 2 times to deal with flakey tests. | ||
| attempts = 3 | ||
| for i in range(attempts): | ||
| try: | ||
| return await self._run_scenario(spec, uri) | ||
| except AssertionError: | ||
| except (AssertionError, OperationFailure) as exc: |
There was a problem hiding this comment.
Do we want to do this for all CSOT tests, or only the asynchronous ones?
There was a problem hiding this comment.
OHHH good point, probably just the async ones right? I don't think I've seen flakey sync tests failing with OperationFailure
There was a problem hiding this comment.
Agreed, no need to change the synchronous behavior if it's not seeing the same issue.
There was a problem hiding this comment.
not sure if it was the best way to only retry in async, but i just changed the logic of the if statement inside the except
test/asynchronous/unified_format.py
Outdated
| if not ( | ||
| isinstance(exc, OperationFailure) | ||
| and not _IS_SYNC | ||
| and "failpoint" in exc._message |
There was a problem hiding this comment.
Try something like this instead:
if isinstance(exc, OperationFailure) and (_IS_SYNC or "failpoint" not in exc._message):
raise
That way it's clear that we raise any OperationFailures that are either thrown by a sync test or don't have the failpoint message.
There was a problem hiding this comment.
ah yeah, that's cleaner
Convert test.test_csot to async