You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Continuation from some of things touched on in #84
What is wrong
Using py-trie with an asyncio codebase is problematic. Trie operations can be heavy and in an asyncio environment you typically don't want that stuff blocking the main event loop.
How can it be fixed.
I don't really know....
There are two things that I think this can try to address.
The computationally heavy components (like all the trie node hashing)
The database components, actually writing to and reading from the base database.
The database part feels easy enough. An AsyncTrie class where get/set/delete are awaitables, or extend the existing class to have coro_get/coro_set/coro_delete methods that are awaitable. I lean towards the AsyncTrie approach but I could be convinced either way.
The heavy computation offloading feels harder. I think this is because there isn't a one true way to offload computationally heavy stuff because the right approach varies in different contexts. That leads me to believe that we might need to build an API that would enable users of the library to hook into the right places where offloading should/could occur and leave it up to the developer to do the actual offloading. We could provide some built-in mechanisms that operate using this API for things like using the asyncio.Executor patterns.
The text was updated successfully, but these errors were encountered:
Continuation from some of things touched on in #84
What is wrong
Using
py-trie
with an asyncio codebase is problematic. Trie operations can be heavy and in an asyncio environment you typically don't want that stuff blocking the main event loop.How can it be fixed.
I don't really know....
There are two things that I think this can try to address.
The database part feels easy enough. An
AsyncTrie
class whereget/set/delete
are awaitables, or extend the existing class to havecoro_get/coro_set/coro_delete
methods that are awaitable. I lean towards theAsyncTrie
approach but I could be convinced either way.The heavy computation offloading feels harder. I think this is because there isn't a one true way to offload computationally heavy stuff because the right approach varies in different contexts. That leads me to believe that we might need to build an API that would enable users of the library to hook into the right places where offloading should/could occur and leave it up to the developer to do the actual offloading. We could provide some built-in mechanisms that operate using this API for things like using the
asyncio.Executor
patterns.The text was updated successfully, but these errors were encountered: