Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make library thread safe using single eventloop, avoid multithreading #534

Open
sacOO7 opened this issue Sep 28, 2023 · 2 comments
Open

Comments

@sacOO7
Copy link
Collaborator

sacOO7 commented Sep 28, 2023

  • Create custom eventloop for asyncio couroutines/tasks/futures rather than using common/default asyncio event loop.
  • Since eventloop uses single thread, if the external app is using default asyncio loop for it's own operations, it can block internal eventloop. Need to create separate eventloop that doesn't use the default asyncio eventloop thread.
  • This is also needed since public APIs can be called asynchronously from different threads or custom eventloop. This will disrupt internal workflow where external eventloops will influence order of the code execution. With current implementation, it will work fine if it's called from a single thread or default asyncio eventloop
  • By having a single custom eventloop for whole internal workflow, we make sure all internal operations are thread-safe. So, thread safety-related issues can be avoided ~~> https://github.com/ably/ably~~python/issues/531
  • Since ably-clients do heavy network IO operations. use of a single-threaded event loop model should be preferred https://www.pythontutorial.net/python-concurrency/python-event-loop/.
  • This is similar to the js single-threaded event loop, which is specifically efficient when file IO or network IO operations are performed.
  • This also means the use of multithreading/threadpool (threading package in python) should be avoided at all costs since it won't be as efficient and there will be an extra overhead to manage shared resources using a thread synchronization means. Also. asyncio objects are not thread safe ( typically not an issue ) and a combination of multithreading and asyncio will introduce complexity to use asyncio threadsafe apis ~~> https://docs.python.org/3/library/asyncio~~dev.html#concurrency-and-multithreading
  • The current ably-js client works without the use of multithreading, so most likely we won't need it either. APIs provided as a part of python native asyncio package should be enough to satisfy spec implementation.

┆Issue is synchronized with this Jira Task by Unito

@sync-by-unito
Copy link

sync-by-unito bot commented Sep 28, 2023

➤ Automation for Jira commented:

The link to the corresponding Jira issue is https://ably.atlassian.net/browse/SDK-3882

@sacOO7 sacOO7 changed the title Create separate eventloop for asyncio Prefer use of separate eventloop, avoid multithreading Sep 28, 2023
@sacOO7 sacOO7 changed the title Prefer use of separate eventloop, avoid multithreading Prefer use of separate global eventloop, avoid multithreading Sep 28, 2023
@sacOO7 sacOO7 changed the title Prefer use of separate global eventloop, avoid multithreading Prefer use of custom global eventloop, avoid multithreading Sep 28, 2023
@sacOO7 sacOO7 changed the title Prefer use of custom global eventloop, avoid multithreading Prefer use of separate global eventloop, avoid multithreading Sep 28, 2023
@sacOO7 sacOO7 changed the title Prefer use of separate global eventloop, avoid multithreading Make library thread safe using single eventloop, avoid multithreading Oct 1, 2023
@sacOO7
Copy link
Collaborator Author

sacOO7 commented Oct 1, 2023

Remove loop provided via clientOptions, as per guidance and spec requirement, external loops should be avoided

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

1 participant