PYTHON-5089 Convert test.test_mongos_load_balancing to async #2107
Conversation
|
Note: I believe the failing evergreen tests are unrelated to the changes I've made |
sleepyStick
requested review from
NoahStapp and
ShaneHarvey
and removed request for
NoahStapp
|
|
||
| @async_client_context.require_connection | ||
| @async_client_context.require_no_load_balancer | ||
| def asyncSetUpModule(): |
There was a problem hiding this comment.
Sadly there isn't an async version of unittest.setUpModule. I'd just add these decorators to an asyncSetUp method for TestMongosLoadBalancing instead since it's the only test class in this file.
There was a problem hiding this comment.
Ah good to know, makes sense
| pass | ||
|
|
||
|
|
||
| if not _IS_SYNC: |
There was a problem hiding this comment.
Let's standardize these checks to if _IS_SYNC for clarity.
There was a problem hiding this comment.
Okay yes, that's what I wanted (and did initially) but for some reason if its if _IS_SYNC first, type-checker assumes both definitions of SimpleOp inherit from threading.Thread and then insist that both implementations adhere to the thread api (in this case join must return a value)
is that preferred? it felt weird to return a value for the sake of it in the async version of SimpleOp simply because its not used at all
There was a problem hiding this comment.
Our typing checks do, or the IDE's own highlighting does? Either way, we should have all checks be for _IS_SYNC whenever possible for clarity.
There was a problem hiding this comment.
both, our typing checks is what caught my attention first actually
|
|
||
| class SimpleOp: | ||
| def __init__(self, client): | ||
| self.task: asyncio.Task |
There was a problem hiding this comment.
| self.task: asyncio.Task | |
| self.task: asyncio.Task = None |
There was a problem hiding this comment.
hmmm I don't love that because it'd mess with the type of self.task (specifically it'd have to be Optional[asyncio.Task]) in the interest of not making it optional, I moved the assignment from start() to the init. Does that work? Is that better?
There was a problem hiding this comment.
Can you show what you mean exactly?
There was a problem hiding this comment.
(I thought I committed and pushed HAHA sorry about that actually pushed now)
|
|
||
|
|
||
| async def do_simple_op(client, nthreads): | ||
| threads = [SimpleOp(client) for _ in range(nthreads)] |
There was a problem hiding this comment.
What do you think about using tasks here instead of threads?
There was a problem hiding this comment.
i like it / wanted to do it but noticed that it'd result in the sync version of the file to be tasks, would I need to modify synchro to change tasks to threads to get this to work?
There was a problem hiding this comment.
We'd need to have it only do so within test files and with a specific token set to not also change things that happen to be called threads but aren't what we want to synchro.
I'd rather have concurrent executors always be called tasks for consistency rather than have them be called threads in the async code.
There was a problem hiding this comment.
makes sense, its tasks now :)
| if not _IS_SYNC: | ||
|
|
||
| class SimpleOp: | ||
| def __init__(self, client): | ||
| self.task: asyncio.Task | ||
| self.task = asyncio.create_task(self.run()) |
There was a problem hiding this comment.
I prefer the earlier version, sorry for the churn.
There was a problem hiding this comment.
nope all good, i offered it as an alternative :)
test/test_mongos_load_balancing.py
Outdated
|
|
||
| def run(self): | ||
| self.client.db.command("ping") | ||
| self.passed = True # No exception raised. |
There was a problem hiding this comment.
Can we settle on a common pattern for the threading.Thread -> Task conversions? I made some suggestions in #2103.
There was a problem hiding this comment.
Thoughts on the approach in #2094?
There was a problem hiding this comment.
Yeah, I believe if I implement noah's suggestion above we'd be using the same pattern :)
There was a problem hiding this comment.
(i think my tab wasn't fully refreshed earlier so i'm just seeing Noah's comment right now) but I actually really like that approach!
|
Sorry, I found the same |
test/asynchronous/helpers.py
Outdated
| @@ -395,7 +395,7 @@ def __init__(self, **kwargs): | |||
| async def start(self): | |||
| self.task = create_task(self.run(), name=self.name) | |||
|
|
|||
| async def join(self, timeout: float | None = 0): # type: ignore[override] | |||
There was a problem hiding this comment.
Great find on this bug. It's not at all obvious and causes a lot of confusing behavior that's hard to spot.
There was a problem hiding this comment.
yup 😭 it took about 20 print debug statements and a good nights rest and a whole lot of "this makes NO sense" until i figured it out HAHA but now that I've noticed it, i'm like why didn't i see it sooner!
|
|
||
| @client_context.require_connection | ||
| @client_context.require_no_load_balancer | ||
| def setUpModule(): |
There was a problem hiding this comment.
We can keep these to improve performance when skipping this test suite.
There was a problem hiding this comment.
I moved it to the class's setUp because on the async side it'd require setUpModule to be awaited and i couldn't find an easy way to achieve that. I figured it was the only class in this module so it didn't make a difference to just move it into the class. But if you know how i could await this in async, I have no hesitations to bring it back!
There was a problem hiding this comment.
Oh whoops my bad, forgot how the wrappers interact with async. You're right!
There was a problem hiding this comment.
All good, wasn't sure if I was missing / forgetting what the trick was!
| passed.append(True) | ||
|
|
||
| if _IS_SYNC: | ||
| threads = [threading.Thread(target=f) for _ in range(nthreads)] |
There was a problem hiding this comment.
Can we use ConcurrentRunner for these as well?
No description provided.