Ten off-the-shelf servers with fine-grained accelerator offload added. These
are for the case the transparent runtime cannot help with: an event-loop or
thread-pool server, where there is no per-connection thread to turn into a
fiber, so the overlap has to be arranged by a small edit instead of by
LD_PRELOAD.
The edit is the same shape every time. A synchronous offload on the loop thread blocks every other client; move it to a worker, park the request, and reply on completion. What changes between servers is only how that server lets you park a request — a module, an extension, a UDF, a native addon, a filter protocol.
| Server | Version | Architecture | How it hooks in | Lines added | Upstream lines modified | Speedup |
|---|---|---|---|---|---|---|
| redis | 6.0.16 | single event loop | loadable module | 83 | 0 | 4.8× |
| nginx | 1.18.0 | event loop + thread pool | addon module | 112 | 0 | 5.2× |
| memcached | 1.6.18 | event loop (libevent) | source patch, 3 files | 70 | 1 | 3.8× |
| nodejs | 22.11.0 | single event loop (libuv) | native addon (N-API) | 34 | 0 | 24.8× |
| python | 3.10 asyncio | single event loop | ctypes, no build step | 22 | 0 | 13.8× |
| apache | 2.4.52 | thread pool (mpm_worker) | module via apxs | 27 | 0 | 59× scaling |
| go | 1.18 net/http | goroutine runtime | cgo, no async code | 28 | 0 | 59× scaling |
| postgresql | 14.23 | process per connection | C extension | 42 | 0 | 141× intra-query |
| mariadb | 10.6.23 | thread per connection | UDF | 34 | 0 | 212× intra-query |
| haproxy | 2.4.30 | event-loop proxy | SPOE + agent | 21 | 0 | 13.3× |
java/ is a JVM case study without a full integration.
Nine of the ten need zero upstream source modification: they attach through an extension mechanism the server already offers. memcached is the exception, and its change is one line plus a preserved patch.
Measurements, methodology and the per-server details are in bench/results/minimal-edit.md. Speedup figures come from different workloads per server — some are request throughput, some are intra-query latency — so read that file before comparing numbers across rows.
redis/accel_module.c is the clearest example. It
implements the same offload twice: accel.sync runs it on the event-loop
thread and blocks the whole server, accel.async hands it to a worker pool and
uses RedisModule_BlockClient to reply on completion. The diff between those
two commands is the technique, in one file, with nothing else going on.
python/server.py is the smallest — 22 lines, no build
step, run_in_executor and nothing more. Good for seeing the shape before
reading a C implementation.
go/server.go is the interesting one: it contains no async code at all. Go's runtime schedules other goroutines onto other OS threads while one blocks in the cgo offload, so a plain blocking handler already overlaps. Some runtimes give you this for free.
Each server has its own build story, since each hooks in differently. The common part is the accelerator: build the emulated backend from the repository root and point the integration at it.
make # builds build/lib/libaccel.so
export ACCEL_LIB=$PWD/build/lib/libaccel.so
export LD_LIBRARY_PATH=$PWD/build/libThen follow the server's own mechanism — apxs -c for Apache, node-gyp for
Node.js, make with PGXS for PostgreSQL, --add-module for nginx, and so
on. Where a build needs the upstream source tree, that tree is not vendored
here; fetch the version in the table above from upstream.
To swap in a real accelerator, point ACCEL_LIB at a different backend — the
CUDA one, or your own. See
docs/writing-a-backend.md.
Every integration calls the same offload entry point:
void accel_run(unsigned char *buf, int n); /* or accel_encrypt, the historical alias */The integrations still call accel_encrypt, which remains exported by every
backend and interposed by the runtime. New code should prefer accel_run.