-
Notifications
You must be signed in to change notification settings - Fork 43
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
asyncioreactor: make sure task isn't deleted midway #274
Conversation
cassandra/io/asyncioreactor.py
Outdated
@@ -176,7 +176,10 @@ def push(self, data): | |||
) | |||
else: | |||
# avoid races/hangs by just scheduling this, not using threadsafe | |||
self._loop.create_task(self._push_msg(chunks)) | |||
background_tasks = set() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This needs to be a member, accessed by self.background_tasks, no? Otherwise it can be garbage collected as well (unless PYthon's garbage collector works in some non-obvious way)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
python GC works with ref counting, and since this is referenced in the now in the task callback it won't be evicted
saving it on the instance might be problematic, since this can be called multiple times, and cause deletion of all of the tasks added
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
so Python GC can't deal with cyclic references?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it can handle cyclic references, but I was referring to what you suggested, of saving the list of task on the instance:
def __init__(self):
background_tasks = set()
def push(self):
....
# avoid races/hangs by just scheduling this, not using threadsafe
task = self._loop.create_task(self._push_msg(chunks))
self.background_tasks.add(task)
task.add_done_callback(self.background_tasks.discard)
this can end up exact the same issue this PR is trying to elevate, of task object getting cleared midway, if we might have multiple calls to push, there would be multiple tasks in the set, and all of them would be discarded at the same time, when the first task would end.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm... I don't follow why it would delete all items - set.discard removed specified item, and callback receives finished task as argument, so will remove only this task from set.
Even in python docs example background_tasks is a global variable
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
o.k., I've was confused, and thought .clear was used
changed it
af8d36e
to
e78ff33
Compare
in push function, self._loop.create_task is called and it's return value is ignored. While the tests may pass now, this code is not correct and this example is called out in docs as a source of bugs, as python docs suggests. Ref: https://docs.python.org/3/library/asyncio-task.html#asyncio.create_task
e78ff33
to
b80960f
Compare
in push function, self._loop.create_task is called and it's return value is ignored. While the tests may pass now, this code is not correct and this example is called out in docs as a source of bugs, as python docs suggests.
Ref: https://docs.python.org/3/library/asyncio-task.html#asyncio.create_task