Skip to content

Performance‐Testing Manual

Jump to bottom
Sergei Kliuikov edited this page May 13, 2025 · 4 revisions

This guide explains exactly how the benchmark above was produced so you can reproduce (or extend) it on any workstation with comparable resources. It is organised as a step-by-step recipe.

1 . Prerequisites

Requirement Version / Notes
Host OS Any recent Linux (tested on Arch Linux 6.8)
CPU / RAM Intel Core i5-10500H @ 2.50 GHz - 6 cores / 12 threads, 64 GB RAM
Docker ≥ 25.0, with BuildKit enabled
docker-compose v2 plugin (ships with Docker 25+)
Rust tool-chain Needed only for oha; install with rustup
oha Commit ≥ v0.5.6 (cargo install oha)

Tip: Disable any host-side CPU governors and run on performance to minimise frequency scaling noise.

2 . Directory Layout

easyrest-bench/
├── docker-compose.yaml        # full stack: Postgres, Valkey, EasyREST, PostgREST
├── init.sql                   # 1 M-row data set + roles
├── bench_config.yaml          # EasyREST runtime config
└── Dockerfile                 # multistage build for easyrest-server

Place all four files in the same directory before continuing.

  • docker-compose.yaml
version: '3.8'

services:
  db:
    image: postgres:16
    container_name: postgres_db
    environment:
      POSTGRES_DB: testdb
      POSTGRES_USER: testuser
      POSTGRES_PASSWORD: testpass
    command: -c 'max_connections=200'
    volumes:
      - ./init.sql:/docker-entrypoint-initdb.d/init.sql
      - postgres_data:/var/lib/postgresql/data
    networks:
      rest_comparison_net:
        ipv4_address: 172.20.0.10
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U testuser -d testdb"]
      interval: 10s
      timeout: 5s
      retries: 5

  cache:
    image: valkey/valkey:7.2
    container_name: valkey_cache
    networks:
      rest_comparison_net:
        ipv4_address: 172.20.0.20

  postgrest:
    image: postgrest/postgrest:v12.2.11
    container_name: postgrest_api
    ports:
      - "3000:3000"
    environment:
      PGRST_DB_URI: postgres://testuser:testpass@db:5432/testdb
      PGRST_DB_ANON_ROLE: anonymous
      PGRST_DB_SCHEMA: public
      PGRST_OPENAPI_SERVER_PROXY_URI: http://localhost:3000
      PGRST_SERVER_TIMING_ENABLED: 'true'
      PGRST_JWT_CACHE_MAX_LIFETIME: '10'
      PGRST_JWT_SECRET: 'SMMDJHas78mV74GT749po1U1j97EkbXS'
    networks:
      rest_comparison_net:
        ipv4_address: 172.20.0.30
    depends_on:
      db:
        condition: service_healthy

  easyrest:
    image: ghcr.io/onegreyonewhite/easyrest:latest
    build:
      context: https://github.com/onegreyonewhite/easyrest.git
      dockerfile: Dockerfile
    container_name: easyrest_api
    ports:
      - "8080:8080"
    volumes:
      - ./bench_config.yaml:/app/config.yaml
    command: ["-config", "/app/config.yaml"]
    networks:
      rest_comparison_net:
        ipv4_address: 172.20.0.40
    depends_on:
      db:
        condition: service_healthy
      cache:
        condition: service_started

volumes:
  postgres_data:

networks:
  rest_comparison_net:
    driver: bridge
    ipam:
      config:
        - subnet: 172.20.0.0/16
  • init.sql
-- Create the main table with various data types
CREATE TABLE prtable (
    id SERIAL PRIMARY KEY,
    random_int INTEGER,
    random_text VARCHAR(50),
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    flag BOOLEAN,
    measurement NUMERIC(12, 4),
    description TEXT
);

-- Create a simple view based on the table
CREATE VIEW ptable AS
SELECT id, random_int, random_text, created_at, flag, measurement, description
FROM prtable;

-- Function to generate random strings (for variety)
CREATE OR REPLACE FUNCTION random_string(length INTEGER) RETURNS TEXT AS $$
DECLARE
  chars TEXT[] := '{0,1,2,3,4,5,6,7,8,9,A,B,C,D,E,F,G,H,I,J,K,L,M,N,O,P,Q,R,S,T,U,V,W,X,Y,Z,a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y,z}';
  result TEXT := '';
  i INTEGER := 0;
BEGIN
  IF length < 0 THEN
    RAISE EXCEPTION 'Given length cannot be less than 0';
  END IF;
  FOR i IN 1..length LOOP
    result := result || chars[1 + floor(random() * (array_length(chars, 1) - 1))];
  END LOOP;
  RETURN result;
END;
$$ LANGUAGE plpgsql;

-- Insert 1,000,000 rows with generated data
-- NOTE: This insert statement will take a significant amount of time to execute
-- during the initial startup of the 'db' container.
INSERT INTO prtable (random_int, random_text, flag, measurement, description)
SELECT
    floor(random() * 1000000)::INTEGER,
    random_string( (random() * 20 + 10)::INTEGER ), -- Random length between 10 and 30
    random() > 0.5,
    random() * 10000.0,
    'Description for row ' || s::TEXT || E'\n' || random_string( (random() * 50 + 50)::INTEGER ) -- Longer text
FROM generate_series(1, 1000000) s;

-- Grant necessary permissions (adjust role 'testuser' if needed)
GRANT SELECT, INSERT, UPDATE, DELETE ON prtable TO testuser;
GRANT SELECT ON ptable TO testuser;
CREATE ROLE anonymous NOLOGIN;
CREATE ROLE webuser NOLOGIN;
CREATE ROLE authenticator LOGIN NOINHERIT NOCREATEDB NOCREATEROLE NOSUPERUSER;

GRANT USAGE ON SCHEMA public TO webuser;
GRANT SELECT ON public.prtable TO webuser;
GRANT SELECT ON public.ptable TO webuser;
GRANT SELECT ON public.ptable TO anonymous;
  • bench_config.yaml
port: 8080
plugin_log: true
access_log: false
token_user_search: sub
token_url: https://auth.example.com/token
token_cache_ttl: 10
auth_flow: password
check_scope: false
token_secret: "SMMDJHas78mV74GT749po1U1j97EkbXS"
cors:
  enabled: false
  allow_origin:
    - "*"
  methods:
    - "GET"
    - "POST"
    - "PUT"
    - "DELETE"
    - "OPTIONS"
  headers:
    - "Accept"
    - "Content-Type"
    - "Authorization"
    - "X-Requested-With"
  max_age: 86400
plugins:
  test:
    title: "Test API"
    uri: postgres://testuser:testpass@db:5432/testdb?maxOpenConns=100&maxIdleConns=25&connMaxLifetime=5&connMaxIdleTime=10&timeout=30&bulkThreshold=100&sslmode=disable&search_path=public
    enable_cache: false
    cache_name: cache
    exclude:
      table:
        - easyrest_cache
  cache:
    uri: redis://cache:6379/1
anon_claims:
  role: "anonymous"
  sub: 0
  • Dockerfile
FROM golang:1.24-alpine AS builder

RUN apk add --no-cache make tzdata

ARG GOARCH="amd64"
ENV CGO_ENABLED=0
ENV GOOS=linux

WORKDIR /app

# Use Docker BuildKit's mount=type=cache to cache Go modules
COPY go.mod go.sum ./
RUN --mount=type=cache,target=/go/pkg/mod \
    --mount=type=cache,target=/root/.cache/go-build \
    go mod download

COPY internal ./internal
COPY plugin ./plugin
COPY plugins ./plugins
COPY cmd ./cmd
COPY Makefile ./

RUN --mount=type=cache,target=/go/pkg/mod \
    --mount=type=cache,target=/root/.cache/go-build \
    make clean server


FROM scratch
# Copy the timezone database from Alpine (builder) image
COPY --from=builder /usr/share/zoneinfo /usr/share/zoneinfo

COPY --from=builder /app/bin/easyrest-server /easyrest-server

ENTRYPOINT ["/easyrest-server"]

3 . Build & Launch the Stack

# 1. Build the EasyREST image (≈ 2 min on fast link / SSD)
docker compose build easyrest

# 2. Start everything in the background
docker compose up -d

# 3. Follow the health-checks
docker compose ps

The database container is considered healthy once pg_isready succeeds five times; both API containers wait for this condition automatically.

3.1 Inspect the network

Each container receives a fixed IPv4 (see docker-compose.yaml) on the bridge rest_comparison_net.
Verify:

docker network inspect rest_comparison_net --format '{{json .Containers}}' | jq .

4 . Generate a Benchmark Token

EasyREST and PostgREST are configured with the same HS256 JWT secret:

python -m pip install pyjwt
TOKENBENCH=$(python - <<'PY'
import jwt, time, os, json, sys
secret = "SMMDJHas78mV74GT749po1U1j97EkbXS"
payload = {
    "sub": 1,
    "role": "webuser",
    "iat": int(time.time()),
    "exp": int(time.time()) + 3600
}
print(jwt.encode(payload, secret, algorithm="HS256"))
PY
)

Keep this shell variable for all oha invocations.

5 . Warm-up the APIs (optional but recommended)

curl -H "Authorization: Bearer $TOKENBENCH"      'http://172.20.0.40:8080/api/test/ptable/?limit=1'
curl -H "Authorization: Bearer $TOKENBENCH"      'http://172.20.0.30:3000/ptable?limit=1'

A warm-up ensures caches, connection pools and the JIT inside Postgres are ready.

6 . Running the Load Tests

Both commands are executed from the host (not inside a container).

# EasyREST (port 8080)
oha -z 60s -p 8 -c 100 -q 9000     -H "Content-Type: application/json"     -H "Authorization: Bearer $TOKENBENCH"     'http://172.20.0.40:8080/api/test/ptable/?limit=10'

# PostgREST v12 (port 3000)
oha -z 60s -p 8 -c 100 -q 9000     -H "Content-Type: application/json"     -H "Authorization: Bearer $TOKENBENCH"     'http://172.20.0.30:3000/ptable?limit=10'
Parameter meaning Value
-z 60s Test duration: 60 seconds
-c 100 Concurrency: 100 TCP connections
-p 8 HTTP/1.1 pipeline length
-q 9000 Target max RPS (acts as a soft upper bound)

Why pipelining? Both servers are fronted by the Docker bridge; pipelining amortises RTT overhead and reveals pure request processing latency.

7 . Raw Results

(identical to the user-supplied run, repeated here for completeness)

Metric (60 s run) EasyREST 0.8.x PostgREST 12.2.11
Requests/second 5663.7 3794.0
Throughput MiB/s 15.08 9.58
Avg latency (ms) 17.6 26.4
95-th pct (ms) 26.5 45.3
99-th pct (ms) 31.6 62.3
Slowest request 117 ms 381 ms
Fastest request 2.2 ms 0.8 ms
Success rate 100 % (deadline-aborted) 100 % (deadline-aborted)

8 . Interpreting the Numbers

  • EasyREST sustained ~1.49× higher request rate and ~1.57× higher data throughput than PostgREST on the same hardware.
  • Tail latency (99-th percentile) was ≈ 2 × lower for EasyREST, indicating fewer outliers under load.
  • The extremely low fastest figure on PostgREST (0.8 ms) shows that its minimal overhead per request is comparable; the difference lies in variability and peak throughput.

9 . Cleaning Up

docker compose down --volumes

This removes containers and the 1 M-row database volume, keeping your workstation clean between runs.

10 . Extending the Test Suite

Ideas you can add without changing the core stack:

  1. JSON output: oha --json easyrest.json … then feed into jq or Grafana for time-series charts.
  2. TLS-offload: Place an Nginx side-car with HTTPS to measure certificate cost.
  3. Cache layer: Set enable_cache: true in bench_config.yaml and compare cold vs warm runs.
  4. Read vs Write mix: Insert/Update tests to evaluate transaction contention.
  5. Connection scaling: Sweep -c from 10 to 500 and plot latency curves.

11 . Troubleshooting Checklist

Symptom Likely Cause Fix
connection refused Wrong IP or container not healthy docker compose ps and inspect logs
oha stalls at 0 RPS Forgot to export $TOKENBENCH Re-generate token in § 4
High error rate DB still initialising (1 M rows) Wait until CPU drops after data load

12 . Reproducibility Tips

  • Pin Docker image versions explicitly (e.g., postgres:16.2) to avoid future tag drift.
  • Commit the generated JWT to a .bench.env file if you automate CI runs.
  • Tag the EasyREST server Git commit in Dockerfile’s context URL to make builds deterministic.

13 . Conclusion

Following this manual you can reproduce the benchmark that shows EasyREST outperforming a stock PostgREST v12 by a comfortable margin on commodity hardware. The same workflow doubles as a template for stress-testing any EasyREST plug-in or schema—just swap the SQL and update bench_config.yaml.

Clone this wiki locally