OpenEverest Plugin Bench runs PostgreSQL (pgbench), MySQL (Sysbench), and MongoDB (go-ycsb) benchmarks as temporary Kubernetes Jobs from the database cluster tab or backend API. All three drivers are enabled by default in the Helm chart.
MVP complete: The PostgreSQL pgbench workflow supports configuring, launching, and inspecting benchmark runs.
The plugin backend coordinates a benchmark run:
- Resolve authorized connection details for the selected instance through the OpenEverest API.
- Pick the database: the one entered in the form, otherwise the instance's
default database (the provider's
databaseconnection key, or the database in its connection URI). The go-ycsb driver supports only theycsbdatabase. - Select the requested driver, validate its options and connection details, and create a temporary Secret containing its password or connection URI.
- Create a Kubernetes Job using the configured driver image and prepared settings.
- Wait for the Job and collect the runner logs and optional structured metrics.
- Delete the Job and temporary Secret.
The benchmark tab is built with MUI and themed from the host through
@openeverest/plugin-theme.
The backend uses a shared driver interface with pgbench, Sysbench, and go-ycsb implementations.
Additional benchmark tools can supply their own option validation and workload
preparation while sharing the Kubernetes execution lifecycle.
New clients submit driver and a driver-specific options object to
POST /api/runs. Target, database, resources, and node affinity remain shared
request fields. The driver applies defaults, validates options, and records the
effective configuration and configured runner image for later inspection.
GET /api/runs/{id} returns status, effective settings, bounded log output, and
optional numeric metrics. Access requires authorization for the stored target.
Missing or invalid metrics do not turn a successful benchmark into a failed run.
The frontend submits the explicit driver/options envelope and displays these
effective settings and metrics.
GET /api/runs?k8sCluster=…&namespace=…&instance=… lists one instance's runs,
newest first. limit is optional (default 20, maximum 100). The caller's access
to the instance is verified through OpenEverest before history is read. Entries
contain status, timestamps, effective settings, metrics, and errors but omit
runner output; fetch a single run for its logs. Both run responses include the
target's databaseType as reported by OpenEverest.
GET /api/status returns generic backend readiness with status, runnerReady,
storage (ephemeral for in-memory history, postgresql when persistent),
message, and a sorted drivers array of enabled driver IDs, such as
["go-ycsb", "pgbench", "sysbench"]. Disabled drivers are excluded. The former PostgreSQL-only
database field has been removed. This endpoint does not test database
connectivity or runner-image availability.
See the request and runner result contracts for payloads, defaults, compatibility, and metrics transport details.
Sysbench is enabled by default in Helm. Submit an explicit driver: "sysbench"
with the OpenEverest target and Sysbench options to POST /api/runs:
{
"target": {"k8sCluster": "main", "namespace": "default", "instance": "my-mysql"},
"database": "sbtest",
"driver": "sysbench",
"options": {"durationSeconds": 5, "threads": 2, "tables": 1, "tableSize": 1000, "initialize": true}
}The target must report database type mysql; its database and account must
already exist. Poll GET /api/runs/{id} with the accepted ID and caller
authorization. See the request contract
for defaults and connection rules, and the runner README
for standalone usage and table cleanup.
go-ycsb is enabled by default in Helm and uses the runner image from the matching
release. Tilt supplies a locally built image for development.
Submit this body to POST /api/runs with caller authorization:
{
"target": {"k8sCluster": "main", "namespace": "default", "instance": "my-mongodb"},
"database": "ycsb",
"driver": "go-ycsb",
"options": {"collection": "benchmark_records", "recordCount": 1000, "operationCount": 2000, "threads": 2, "initialize": true}
}The target must report type mongodb and supply a mongodb:// URI. Only the
ycsb database is supported by the pinned upstream tool: omit database or
set it to ycsb; other names are rejected. The backend preserves authentication,
including provider URIs using /admin, while targeting ycsb. The supplied
account needs read/write access to ycsb. For TLS instances, a provider CA
published as ca.crt in the connection details is passed to the runner to
verify the server certificate.
Workload A uses equal reads and updates. Set initialize=true to load a fresh,
disposable collection; reloading existing keys fails. To reuse it, set
initialize=false and keep the collection and record count unchanged. Job cleanup
leaves the data in MongoDB; drop the collection separately when finished.
Poll GET /api/runs/{id} for logs and optional ops_per_second (ops/s) and
latency_avg (ms). See the runner reference
for defaults.
Open Performance Benchmark in the sidebar to list accessible instances in the host's current Kubernetes cluster. Only providers supported by an enabled driver appear. Select an instance or Benchmark to open its benchmark tab; use Retry if discovery fails. Discovery uses your OpenEverest permissions and reads instance metadata without requesting database credentials.
Open a supported instance's Performance Benchmark tab. Its provider determines which driver is offered:
| Instance provider | Driver |
|---|---|
provider-cloudnative-pg, provider-percona-postgresql |
pgbench |
percona-server-mysql, percona-xtradb-cluster |
Sysbench |
percona-server-mongodb |
go-ycsb |
The tab checks /api/status and only offers drivers enabled on the backend.
If none are enabled for the provider, run lookup remains available.
Database, Job resources, and Job placement are shared settings. For SQL
drivers, a blank database uses the instance default when available; otherwise
enter an existing database name. Use a disposable benchmark database.
For both percona-server-mysql and PXC (percona-xtradb-cluster), a database name
is required because neither provider supplies a default. Create the database
before starting the benchmark; table preparation does not create it.
pgbench starts with 30 seconds, one client/thread, scale 1, and initialization enabled. Initialization drops and recreates pgbench tables. When reusing tables, scale is disabled and displayed as From existing tables in the run result.
Sysbench starts with 30 seconds, one thread, one table, 10,000 rows per table,
and preparation on. It runs oltp_read_write. Prepare Sysbench tables creates
a fresh dataset; existing table names cause preparation to fail. To reuse
tables, turn preparation off and match Tables and Rows per table to the
existing dataset. Tables remain after Job cleanup. Sysbench must be enabled with
an accessible runner image; see Driver configuration.
go-ycsb fixes Database to the read-only value ycsb. It starts with collection
usertable, 10,000 records, 20,000 operations, one thread, and Load benchmark data
checked. Workload A performs equal reads and updates. Use a fresh disposable
collection for loading; to reuse it, turn loading off and keep Collection and
Records matched to the dataset. Operations controls work performed, subject to
the overall execution timeout. The account needs read/write access to ycsb;
MongoDB can create the database/collection during loading, and Job cleanup leaves
the data intact.
Each accepted run has its own card and polls for status every two seconds until completion. Completed cards show the driver, configured runner image, effective options, and a metrics table. Raw logs expands the captured output; a Truncated badge identifies output that reached the backend limit. Missing or invalid metrics show Structured metrics unavailable while preserving the run's status and logs. Failed runs show their error alongside any available results.
History lists the instance's 20 most recent runs, newest first, with the tool, status, duration, and metrics (or the error of a failed run). It reloads when you start a run and when that run finishes; Refresh reloads it on demand. View opens a run as a card with its full results and logs; runs already shown are marked Shown. When history is kept only in memory, the section says so. See run history storage.
Use Look up a benchmark run to retrieve a stored run by ID. Looking up an already displayed ID does not add a duplicate card. Records remain subject to the configured run history storage.
Administrators can pin every runner Job to specific nodes, such as a tainted,
dedicated benchmark node pool, with runner.nodeSelector and runner.tolerations
in Helm:
runner:
nodeSelector:
node-role/benchmark: "true"
tolerations:
- key: dedicated
operator: Equal
value: benchmark
effect: NoScheduleInvalid values stop the backend at startup. Users can additionally require node
labels for a single run under Job placement in the UI. Those rules are sent as
nodeAffinity and apply on top of the deployment's node selector, so a node must
satisfy both.
While waiting for a Job, the coordinator checks the runner Pod's scheduling
condition. If the same Pod is continuously observed as PodScheduled=False with
reason Unschedulable for runner.schedulingGracePeriod (default 2m, or
BENCHMARK_SCHEDULING_GRACE_PERIOD), the run fails and its Job and temporary
credential Secret are cleaned up. Users see a generic "could not be scheduled"
message; the scheduler's explanation, which reveals node counts and taints, is
written to the backend log.
The grace period starts at the first observation of scheduler rejection, not at
submission. Scheduling recovery, Pod replacement, or a gap in scheduling
observations resets it. It does not limit benchmark runtime or image downloading.
Choose a period that allows enough time for autoscalers to provision nodes, or
set "0" to disable early failure and skip the extra Pod lookup on each Job poll.
BENCHMARK_EXECUTION_TIMEOUT (default 5m) bounds the whole run, including
scheduling, image startup, connection setup, and table preparation. The duration
option limits only the measured benchmark phase.
For local development, install:
- Docker
- kubectl
- Helm
- k3d
- Tilt
- Go
- Node.js and npm
An OpenEverest installation and a provider for the selected database type are
required when running the plugin in a cluster. Sysbench matches database type
mysql rather than provider name. A live benchmark against a PXC
(percona-xtradb-cluster) instance has been verified. The API also accepts
provider-mariadb targets reporting mysql, but MariaDB compatibility has not
been verified. The Sysbench runner has been verified against MySQL 8.4 on Linux/ARM64.
Create the development cluster and start the Tilt environment:
make dev-upIf the cluster already exists, run:
tilt up -f dev/TiltfileTilt builds the frontend, backend image, and pgbench, Sysbench, and go-ycsb runner
images. It injects each runner's repository and tag
through drivers.<id>.image.repository and drivers.<id>.image.tag. Runner images
are used when the coordinator starts benchmark Jobs; no published runner image
is needed for this flow.
Set BENCH_STORAGE=postgresql in dev/.env to deploy a disposable PostgreSQL
and store run history there, which lets you exercise persistence and restart
recovery locally. Its data is lost when that Pod is replaced.
By default, run history lives in backend memory: it is lost on restart and completed runs expire after one hour. To keep history, point the plugin at a PostgreSQL database, for example one provisioned with OpenEverest:
-
Create a database for the plugin, such as
benchmarks, and an account that can create tables in it. -
Create a Secret in the plugin's release namespace that holds the connection URI. Kubernetes reads Secret references only from the Pod's own namespace, so a connection Secret in another namespace must be copied:
kubectl -n everest-system create secret generic plugin-bench-history \ --from-literal=uri='postgresql://bench:<password>@<host>:5432/benchmarks?sslmode=require' -
Enable PostgreSQL storage:
storage: type: postgresql postgresql: existingSecret: plugin-bench-history uriKey: uri
On startup, the backend applies its own schema migrations (the
benchmark_runs, benchmark_run_metrics, plugin_bench_installation, and
plugin_bench_schema_migrations tables) under an advisory lock and refuses to
start against a schema created by a newer release. A database belongs to one
plugin installation, identified by its Helm release name: a second installation
pointed at the same database refuses to start before changing its schema, so
give each installation its own database or schema (for example, with
search_path=<schema> in the URI). An
unreachable database or an invalid URI stops startup rather than falling back
to memory. Each run records its target, database type (for example
postgresql or mysql), driver, effective options, resources, status,
timestamps, and redacted logs; it never contains target database credentials.
Metrics are stored one row per name and unit in benchmark_run_metrics, so any
benchmark tool can report its own metrics and results can be compared across
runs with SQL. PostgreSQL history is kept until
you delete it; there is no automatic retention yet.
Avoid benchmarking the database that stores history: the benchmark load skews its own results and competes with history writes.
A run cannot be resumed after the backend restarts, because redacting its logs
requires target credentials that only existed in the previous process. Before
accepting requests, the backend therefore marks runs left running as failed
("interrupted because the benchmark service restarted") and deletes the runner
Jobs and credential Secrets in the workload namespace labeled
app.kubernetes.io/managed-by=plugin-bench and
app.kubernetes.io/instance=<release name>. Other installations' resources are
untouched. In memory mode, the records are already gone, but leftover resources
are still removed. If cleanup fails, for example because of a transient
Kubernetes API error, startup continues and cleanup is retried in the
background with backoff until it succeeds. Each backend process also labels its
runner resources with benchmark.openeverest.io/backend-process, and cleanup
skips the current process's resources, so retries never touch runs started
after the restart.
This cleanup assumes one backend per installation. The chart uses the
Recreate Deployment strategy, so the old backend stops before the new one
starts, and rejects replicaCount values other than 1.
Configure enabled benchmark drivers and their runner images through Helm:
drivers:
pgbench:
enabled: true
image:
repository: ""
tag: ""
pullPolicy: ""
sysbench:
enabled: true
image:
repository: ""
tag: ""
pullPolicy: ""
go-ycsb:
enabled: true
image:
repository: ""
tag: ""
pullPolicy: ""Empty image fields use the defaults below after any legacy pgbench overrides.
With no overrides, pgbench uses
ghcr.io/openeverest/plugin-bench-pgbench:<appVersion> and IfNotPresent.
appVersion comes from the chart being installed, including metadata set by
helm package --app-version during a release.
All three drivers are enabled by default and use their matching release images. To use a custom Sysbench image, override its repository or tag:
drivers:
sysbench:
enabled: true
image:
repository: registry.example.com/team/plugin-bench-sysbench
tag: tested-version
pullPolicy: IfNotPresentThe example registry and tag are placeholders for an image you supply. Empty
Sysbench image fields resolve to
ghcr.io/openeverest/plugin-bench-sysbench:<appVersion> and IfNotPresent;
the release workflow publishes the matching image alongside the chart.
| Image field | First choice | Deprecated pgbench fallback | Default |
|---|---|---|---|
| Repository | drivers.<id>.image.repository |
runner.image.repository |
ghcr.io/openeverest/plugin-bench-<id> |
| Tag | drivers.<id>.image.tag |
runner.image.tag |
Chart appVersion |
| Pull policy | drivers.<id>.image.pullPolicy |
runner.image.pullPolicy |
IfNotPresent |
Precedence applies independently to each field. For example, this pins just the pgbench runner version while inheriting its repository and pull policy:
drivers:
pgbench:
image:
tag: "0.1.0"Legacy runner.image overrides apply only to pgbench. Migrate them to
drivers.pgbench.image when updating your values. Shared resources, timeouts,
node selectors, and tolerations remain under runner; the backend Deployment's
own image remains under the top-level image settings.
Helm serializes the resolved settings into one backend environment variable.
For a chart with appVersion: 0.2.0:
{"go-ycsb":{"enabled":true,"image":"ghcr.io/openeverest/plugin-bench-go-ycsb:0.2.0","imagePullPolicy":"IfNotPresent"},"pgbench":{"enabled":true,"image":"ghcr.io/openeverest/plugin-bench-pgbench:0.2.0","imagePullPolicy":"IfNotPresent"},"sysbench":{"enabled":true,"image":"ghcr.io/openeverest/plugin-bench-sysbench:0.2.0","imagePullPolicy":"IfNotPresent"}}The variable is named BENCHMARK_DRIVERS. Each entry requires an explicit
enabled boolean. Enabled entries require a nonempty image reference, and pull
policy must be Always, IfNotPresent, or Never (omitted/empty defaults to
IfNotPresent). Only drivers compiled into the backend can be enabled; adding
a Helm entry does not install an implementation. The compiled drivers are
pgbench, Sysbench, and go-ycsb. At least one must remain enabled with a usable image.
For deployments setting environment variables directly, RUNNER_IMAGE and
RUNNER_IMAGE_PULL_POLICY remain deprecated pgbench fallbacks only when
BENCHMARK_DRIVERS is absent. In that legacy mode, RUNNER_IMAGE is required
and pull policy defaults to IfNotPresent. A present payload is authoritative:
empty/malformed JSON, unknown fields, unknown enabled drivers, or no enabled
drivers stop startup instead of falling back to legacy variables. Explicit
disablement is never reversed by a legacy image setting.
The pgbench runner receives its PostgreSQL connection through these variables:
PGHOST
PGDATABASE
PGUSER
PGPASSWORD or PGPASSFILE
Benchmark options are configured with:
BENCH_INITIALIZE default false
BENCH_DURATION default 60 seconds
BENCH_CLIENTS default 1
BENCH_THREADS default 1
BENCH_SCALE default 1
BENCH_INITIALIZE=true drops and recreates the pgbench tables. Use it only
with a disposable database. See runners/pgbench/README.md
for detailed runner instructions.
The Sysbench runner receives MYSQL_HOST, MYSQL_PORT, MYSQL_DATABASE,
MYSQL_USER, and MYSQL_PASSWORD, plus BENCH_DURATION, BENCH_THREADS,
BENCH_TABLES, BENCH_TABLE_SIZE, and BENCH_INITIALIZE. The backend supplies
the password through a Kubernetes Secret reference; ordinary connection and
benchmark settings use environment values. See the
Sysbench configuration reference
for defaults and credential-file encoding rules.
The go-ycsb runner receives MONGODB_URI and, when the provider publishes one,
MONGODB_TLS_CA through Secret references and ordinary
BENCH_* options through environment values. See its
configuration reference.
npm ci
npm run test:form
npm run test:components
npm run test:submission
npm run test:status
npm run test:polling
npm run test:result
npm run build
cd backend && go test ./...
cd backend && go vet ./...
# PostgreSQL storage integration tests (skipped unless the URI is set):
cd backend && PLUGIN_BENCH_TEST_POSTGRES_URI='postgres://user:pass@localhost:5432/db?sslmode=disable' go test ./internal/pgstore/ .
helm lint charts/plugin-bench/
docker build -t plugin-bench-pgbench:dev runners/pgbench
docker build -t plugin-bench-sysbench:dev runners/sysbench
docker build -t plugin-bench-go-ycsb:dev runners/go-ycsbBoth PR CI and release builds run these frontend test suites before building the bundle. Component tests include the complete submission/polling/results and run lookup workflows with mocked API responses, including concurrent cards and cancellation on unmount or target changes.
For a live check after starting Tilt, use a disposable supported database:
- Start a short run and confirm the POST contains
driverand nestedoptions. - Confirm polling reaches a terminal status and stops. Check effective options, metrics, and expandable logs; look up the same ID and confirm no duplicate card.
- Start two runs and confirm both cards retain their own status and results.
- Leave the tab during a run and confirm status requests stop. A missing run ID should pause status checks with a clear error rather than resubmit a benchmark.
For Sysbench, prepare a small fresh dataset, then repeat with preparation off and matching table settings. Preparing the same tables again should fail clearly. For go-ycsb, load a fresh collection, then repeat with loading off and the same collection and record count.
Push a vX.Y.Z tag, or vX.Y.Z-<pre-release> for a pre-release, to run the
release workflow. It publishes:
ghcr.io/openeverest/plugin-bench,ghcr.io/openeverest/plugin-bench-pgbench,ghcr.io/openeverest/plugin-bench-sysbench,ghcr.io/openeverest/plugin-bench-go-ycsb, andghcr.io/openeverest/plugin-bench-hammerdbimages forlinux/amd64andlinux/arm64- the
oci://ghcr.io/openeverest/charts/plugin-benchHelm chart - a GitHub release with the chart archive and release notes
The workflow tests all four runners from their built AMD64 archives before publishing. HammerDB is currently a standalone runner; API/UI integration is a separate task. See its runner reference for direct usage. Release notes include each image reference and digest.
GHCR creates new packages as private, so the first release fails at the
"Verify public access" step. Make each new package, including
plugin-bench-sysbench, plugin-bench-go-ycsb and plugin-bench-hammerdb, public in its GHCR package
settings, then re-run the failed job. Publishing is idempotent, so re-runs are safe.
src/ React frontend
backend/ Go backend
backend/internal/everest/ OpenEverest API client
backend/internal/driver/ Driver contracts, registry, workload preparation
backend/internal/coordinator Kubernetes Job and Secret orchestration
backend/internal/runnerresult Generic runner metrics decoder
backend/internal/api/storetest Shared run store conformance tests
backend/internal/pgstore/ PostgreSQL run history storage and migrations
runners/pgbench/ pgbench runner image
runners/sysbench/ MySQL Sysbench runner image
runners/go-ycsb/ MongoDB go-ycsb runner image
runners/hammerdb/ Standalone PostgreSQL/MySQL HammerDB runner image
charts/plugin-bench/ Helm chart and Plugin resource
dev/ Tilt and k3d configuration
spec.md MVP requirements
The UI supports the providers listed above; MySQL requires an enabled Sysbench
driver and a usable image. MongoDB is limited to the ycsb database and Workload A;
additional workloads remain future work.
With the default in-memory storage, run records, including output and metrics,
are lost on restart, and completed records become unavailable one hour after
completion. Configure PostgreSQL storage to keep
history. Runs in progress during a restart are marked failed rather than
resumed. One backend replica is supported. The UI lists recent runs per
instance; paging beyond them, charts, and comparisons remain future work.
Licensed under the Apache License 2.0. See LICENSE.