CLO-216: balancerd: enforce a max connection limit

[jubrad]

Issue

balancerd has no connection ceiling. It accepts connections until the container hits its memory limit and is OOMKilled, which drops every established session instead of shedding only the load it cannot serve. A self-managed deployment at a 256Mi limit lost both replicas at ~400-500 concurrent connections; clients saw the connection is lost and SSL error: unexpected eof, nothing pointing at the proxy.

Linear: https://linear.app/materializeinc/issue/CLO-216/balancerd-should-enforce-a-max-connection-limit-instead-of-being

Solution

New balancerd_max_connections dyncfg (default 5000, 0 disables), so it is tunable through LaunchDarkly like the other balancerd_* flags and settable at startup with --default-config=balancerd_max_connections=N. A single ConnectionLimiter is shared by the pgwire and HTTPS listeners, since memory is shared; the internal HTTP port stays unlimited so health and metrics keep working while the limit is in effect.

Over the limit, pgwire gets a fatal ErrorResponse with SQLSTATE 53300 and HTTPS gets a raw 503. Two new metrics, mz_balancer_connection_rejected_total and mz_balancer_connection_limit, pair with the existing mz_balancer_connection_active to make the state diagnosable and give HPA a utilization signal (DEP-235). The log line fires on entering and leaving the limited state rather than per connection.

The default of 5000 sits above cloud's HPA target of 2600 connections per pod, so cloud behavior is unchanged until the flag is set.

Testing

ConnectionLimiter unit test covers acquire/release, refusal at the limit, lowering the limit below the active count, and 0. A new max_connections mzcompose workflow in test/balancerd/mzcompose.py runs balancerd with a limit of 1 and asserts the second connect fails with the server's message, that the established connection keeps working, and that both metrics are exported.

Gotchas for the reviewer

• The ceiling covers connections that have sent a startup message. Sockets still in TLS or pre-startup are not counted, and server-core has no handshake timeout, so a slowloris storm is still unbounded. That is pre-existing and left for separate hardening.
• Because the check precedes Frontegg auth, unauthenticated clients can occupy permits. That is deliberate: those connections consume memory too, and this limit is about memory, unlike environmentd's post-auth max_connections.
• pgwire cancel requests bypass the limiter, as they are best-effort and short-lived.
• doc/user/data/metrics.yml is regenerated by bin/lint, not hand-edited.

🤖 Generated with Claude Code