You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Dramatiq operations on builtin brokers are thread-safe
however the broker in this repo is not built-in and I can't see any mention of thread safety nor any locking.
As far as I can see when a dramatiq task is enqueued then the following generic call chain occurs: Actor.send > Actor.send_with_options > Broker.enqueue
Looking at SQSBroker.enqueue in this repo (which doesn't do any locking):
it does queue.send_message where queue comes from self.queues[queue_name]
the queue object originally comes from self.sqs.create_queue(...) where self.sqs is a boto3 "Resource"
Given the above, is this broker potentially not thread safe, e.g. for simultaneously enqueuing from multiple threads of a web server process (such as gunicorn) to the same queue?
The text was updated successfully, but these errors were encountered:
Looking at your notes, I think you're probably right. I had missed that detail about boto resources. If anyone wants to fix this, I'd be happy to accept a PR.
I noticed in https://github.com/Bogdanp/dramatiq/blob/master/docs/source/troubleshooting.rst it says:
however the broker in this repo is not built-in and I can't see any mention of thread safety nor any locking.
As far as I can see when a dramatiq task is enqueued then the following generic call chain occurs:
Actor.send
>Actor.send_with_options
>Broker.enqueue
Looking at
SQSBroker.enqueue
in this repo (which doesn't do any locking):queue.send_message
wherequeue
comes fromself.queues[queue_name]
self.sqs.create_queue(...)
whereself.sqs
is a boto3 "Resource"sqs.create_queue
returns aQueue
resourceGiven the above, is this broker potentially not thread safe, e.g. for simultaneously enqueuing from multiple threads of a web server process (such as gunicorn) to the same queue?
The text was updated successfully, but these errors were encountered: