Add fixed-capacity API workload isolation pools

[Alexey Soldatchenko (soldatchenko)]

This adds an opt-in workload-isolation capability for Kubernetes data planes whose ingress or gateway can route traffic to the dedicated Services.

For existing deployments, upgrading to a chart release that includes this change does not enable workload isolation or change the existing API routing. When api.workloadIsolation.enabled: false, the chart renders no additional API pools and preserves the default API Deployment and Service behavior.

To enable isolation, deploy the chart with the feature enabled, verify the dedicated workloads and Services are healthy, then configure the ingress or gateway to route the paths below. The chart does not configure public routing. Enablement creates two API Deployments and Services, and changes Brainstore's internal scoring/function target to the background Service. Plan for a rolling update of the API and Brainstore workloads.

This feature uses fixed replica counts; it does not configure HPA or other autoscaling behavior.

Route contract

The ingress or gateway remains the route owner. It must keep braintrust-api as the fallback backend and route this path set to the dedicated Services:

Pool	Paths
braintrust-api (default)	All requests not matched by an explicit ingest or background route
braintrust-api-ingest	/logs3, /otel/v1/traces, /attachment, /attachment/status
braintrust-api-background	/v1/eval, /v1/eval/*, /function/eval, /function/sandbox, /function/use, /function/invoke-async-batch, /function/insert-functions, /automation/logs/trigger, /v1/proxy/chat/completions, /v1/proxy/responses

GKE routing is path-based rather than method-aware, so matching paths are isolated for all HTTP methods.

Rollback

First route the isolated public paths back to the default API Service and verify it is serving them. Only then disable workload isolation in the chart. The chart cannot update an external ingress or gateway on its own.

[Alexey Soldatchenko (soldatchenko)]

Validation completed

• ./test.sh passed: 282 Helm unit tests, Azure/GCP/AWS/minimal renders, and strict lint.
• Deployed in a real GKE sandbox and verified distinct healthy default, ingest, and background backends with the expected path ownership.
• During a controlled background-pool outage, the background route became unavailable as expected while the default health route remained available. All 30 concurrent ingestion writes were acknowledged and subsequently read back.
• Restored the background pool and confirmed the workloads and backends recovered healthy.

[Alexey Soldatchenko (soldatchenko)]

This PR intentionally ports the workload-isolation model introduced in AWS PR #287.

The k8s implementation uses Deployments, Services, and GKE Ingress rather than ECS services and ALB target groups, but preserves the same traffic contract: ingestion is isolated from scoring/evals/proxy background work, while all other API traffic remains on the default pool.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1304354ef5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "Codex (@codex) review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "Codex (@codex) address that feedback".