Skip to content

Thread Safety

Jump to bottom
Peter Goldstein edited this page Jan 22, 2023 · 1 revision

TLDR

Asimov::Client should NOT be considered thread-safe.

That is, do not share instances of Asimov::Client between different threads. While there's nothing explicitly non-thread safe in the Asimov::Client at this time, this recommendation is being made to preserve the option to introduce implementations that are not thread-safe going forward.

Options for using Asimov::Client with Multiple Threads

There are essentially two options for using Asimov::Client in a multi-threaded (or multi-fibered) context. These are:

  1. Create an Asimov::Client per thread/fiber
  2. Use a thread-safe pool to manage your Asimov::Client instances

Instances are very cheap to create, so option 1 can be used without any significant impact on application performance.

If you wish to throttle the number of concurrent connections to OpenAI, or you wish to limit the number of Asimov::Client instances for any other reason, there are excellent gems for pooling. We recommend connection_pool as a good option.

Application-wide Configuration

Generally speaking, Asimov should be configured at application startup, and the application-wide configuration should be treated as immutable thereafter. Changing the configuration after one has started creating Asimov::Client instances may lead to inconsistent behavior.

Clone this wiki locally