All asynchronous composition in Blaze runs its *-async steps on the default executor of CompletableFuture, which is ForkJoinPool.commonPool(). That pool is constructed with asyncMode=false, so a worker takes the newest task from its own deque first. That order is right for divide-and-conquer work, where the newest task is the most cache-local one, but not for the independent continuations of concurrent requests, where taking the newest first delays the oldest and widens the tail latency. Work stealing runs in the other direction and mitigates that, but only while other workers are idle, which is exactly not the case under load.
This came up while reviewing the futures waiting for a t to be indexed (#4055). A transaction releases all its waiters at once and each continuation is submitted to the pool, so their relative order is decided by the pool alone. The question isn't specific to that path though. It applies to every then-apply-async and do-async in Blaze.
We should measure whether a dedicated executor gives a better tail latency: either a ForkJoinPool with asyncMode=true or a fixed size ThreadPoolExecutor with a FIFO queue, used as the default executor of the blaze.async.comp functions. A separate pool would also isolate Blaze from the other users of the common pool inside the JVM, which is the kind of interference that caused #2584, and would give those threads a name of their own in profiles and thread dumps. Against that, it's another pool to size, and continuations block on key-value store reads without a ManagedBlocker, so sizing it wrongly is worse than what we have today.
The outcome should be a decision backed by a measurement, either the change itself or a documented reason why the common pool stays. If it changes, the docstrings promising the common ForkJoinPool in blaze.db.api, blaze.handler.fhir.util and blaze.job.util have to be updated as well.
All asynchronous composition in Blaze runs its
*-asyncsteps on the default executor ofCompletableFuture, which isForkJoinPool.commonPool(). That pool is constructed withasyncMode=false, so a worker takes the newest task from its own deque first. That order is right for divide-and-conquer work, where the newest task is the most cache-local one, but not for the independent continuations of concurrent requests, where taking the newest first delays the oldest and widens the tail latency. Work stealing runs in the other direction and mitigates that, but only while other workers are idle, which is exactly not the case under load.This came up while reviewing the futures waiting for a
tto be indexed (#4055). A transaction releases all its waiters at once and each continuation is submitted to the pool, so their relative order is decided by the pool alone. The question isn't specific to that path though. It applies to everythen-apply-asyncanddo-asyncin Blaze.We should measure whether a dedicated executor gives a better tail latency: either a
ForkJoinPoolwithasyncMode=trueor a fixed sizeThreadPoolExecutorwith a FIFO queue, used as the default executor of theblaze.async.compfunctions. A separate pool would also isolate Blaze from the other users of the common pool inside the JVM, which is the kind of interference that caused #2584, and would give those threads a name of their own in profiles and thread dumps. Against that, it's another pool to size, and continuations block on key-value store reads without aManagedBlocker, so sizing it wrongly is worse than what we have today.The outcome should be a decision backed by a measurement, either the change itself or a documented reason why the common pool stays. If it changes, the docstrings promising the common ForkJoinPool in
blaze.db.api,blaze.handler.fhir.utilandblaze.job.utilhave to be updated as well.