Skip to content

quickstart.md

Latest commit

 

History

History
165 lines (124 loc) · 5.86 KB

quickstart.md

File metadata and controls

165 lines (124 loc) · 5.86 KB
Raw

Quickstart

Ten minutes from a clone to the effect reproducing on your machine, then to trying it against your own server.

1. Build

git clone https://github.com/19PINE-AI/transparent-offload
cd transparent-offload
make

You need a C compiler, GNU make, OpenSSL headers and zlib. On Debian or Ubuntu:

sudo apt install build-essential libssl-dev zlib1g-dev

No GPU, no root, no kernel tuning. make produces:

build/lib/libtransparent.so   the runtime
build/lib/libaccel.so         the emulated accelerator (real AES, latency knob)
build/bin/conn_server         an ordinary blocking thread-per-connection server
build/bin/client              a load generator that verifies every response

2. See it work

make check    # correctness: 13 tests, no GPU, no root
make demo     # the headline A/B

make demo runs one unmodified server binary twice — once plain, once with the runtime preloaded — across a range of offload latencies:

  offload    |  stock rps |  toffl rps | speedup | toffl p50 |   bad
  -----------+------------+------------+---------+----------+------
  20us       |     123444 |     119607 |   0.97x |     257us |     0
  50us       |     110651 |     115068 |   1.04x |     280us |     0
  100us      |      90857 |     114585 |   1.26x |     274us |     0
  200us      |      70951 |     104461 |   1.47x |     295us |     0

(A real run on a 32-thread Core i9-13900KS, unprivileged, nothing tuned. Your absolute numbers will differ; the shape should not.)

Three things to read out of that table.

The runtime column is nearly flat — 120K down to 104K while the device gets ten times slower. That is the offload latency being hidden. The stock column falls away, 123K to 71K, because that is time a core spends doing nothing.

The 20 µs row is 0.97x: slightly slower. That is the honest bottom edge of the useful regime. At 20 µs the offload is barely longer than the work of parking and resuming a fiber, so there is nothing to win. Overlap starts paying around 50 µs here and grows from there. If your offload is a few microseconds, this runtime is not the tool.

bad is the number of responses that failed verification against a CPU-computed AES reference. It must be 0.

The paper reports larger speedups at the same latencies (1.43x at 20 µs) because those runs pinned the carrier to an isolated core under SCHED_FIFO, which needs privileges. make demo deliberately does not, so that what you see is what an ordinary user gets on an ordinary machine. See reproducing.md for the paper's exact setup.

Why the demo pins cores

make demo gives both sides exactly one core for the application and one for the device. This matters enough to be worth a paragraph.

The runtime buys per-core efficiency: it gets more work out of one core while the accelerator is busy. A stock thread-per-connection server on a large machine hides the same latency a different way — it spreads 64 OS threads over 64 cores and lets the hardware absorb the stall. Run the comparison unpinned on a big box and the two converge; you have measured the size of the machine rather than the effect.

So if you compare by hand, equalise the core budget:

taskset -c 4,6 ./build/bin/conn_server 20443                      # stock
taskset -c 4,6 env LD_PRELOAD=$PWD/build/lib/libtransparent.so \
    ./build/bin/conn_server 20443                                  # runtime

This is also the honest statement of when the runtime pays off in production: when cores are the scarce resource, not when you have spare ones.

3. Run it by hand

Terminal one:

export LD_LIBRARY_PATH=$PWD/build/lib
ACCEL_LAT_US=200 TOFFLOAD_STATS=stderr:1 \
  LD_PRELOAD=$PWD/build/lib/libtransparent.so \
  taskset -c 4,6 ./build/bin/conn_server 20443

Terminal two:

./build/bin/client 64 3000 20443

The server prints a counter line every second:

[toffload] fibers_created=64 fibers_live=64 switches=2817174 offloads=708339 \
           io_yields=1416678 det_faults=0 det_conflicts=0 serialized=0

fibers_created is the check that matters. If it is 0 the runtime loaded but never engaged — it is passing your application through untouched, which is safe but pointless. docs/is-my-server-supported.md explains why that happens and what to do.

4. Try it on your own server

The runtime interposes an offload call by name. For your own program that means routing the expensive call through the backend API:

#include <toffload/accel.h>

void handle_request(unsigned char *buf, int len) {
    read_request(buf, len);
    accel_run(buf, len);      /* the runtime interposes this */
    write_response(buf, len);
}

Build against the emulated backend to start with, then swap in a real one:

cc -O2 myserver.c -Ipath/to/include -Lpath/to/build/lib -laccel -lpthread -o myserver
LD_PRELOAD=path/to/build/lib/libtransparent.so ./myserver

If your accelerator is not AES on a CPU thread — a GPU, an FPGA, a remote signer — write a backend. It is four functions and about fifty lines; see docs/writing-a-backend.md.

For an unmodified third-party server, there is nothing to edit, but whether you get overlap depends on the server's concurrency architecture. Read docs/is-my-server-supported.md first — it will save you an afternoon.

5. Where to go next

You want to Read
Know whether your server can benefit is-my-server-supported.md
Understand the mechanism architecture.md
Support your own accelerator writing-a-backend.md
Know when it is unsafe safety.md
Tune it configuration.md
Reproduce the paper reproducing.md
Fix something that broke troubleshooting.md