One AnyCable process at 100% opening websocket connections

[cjamison]

Thank you for this great library! We have been using AnyCable successfully and I was testing the limits using Apache jMeter. We are using a server with 16 vCPUs. The test simulation signs into our system and then initiates a websocket connection. Doing a simulation of 5,000 users, we noticed that during the Websocket Open Connection phase, one process running ./shared/bundle/2.7.0/bin/anycable stays pegged at 100%. There are other anycable ruby processes but they don't stay pegged at 100% (maybe 5-10%). Once the Websocket Open Connection phase stops, the usage drops back down to "normal".

Here is my question. Is this an unavoidable bottleneck due to the architecture? Or, is there some way to address it? Right now, everything is running on one server and we are currently running anycable (1.0.2) and anycable-rails (1.0.5) on Ruby 2.7.4.

[palkan]

There are other anycable ruby processes but they don't stay pegged at 100% (maybe 5-10%)

One instance of AnyCable RPC uses a single Ruby process (not sure about the underlying C extension server implementation; likely, it's multi-core, and other low CPU processes explain this). This Ruby process could become a bottleneck if the message rate is extremely high (I guess, that's exactly what happens when you open 5k connection simultaneously). I'd recommend to add a random jitter to the script to open connections (which would be more realistic).

Is this an unavoidable bottleneck due to the architecture?

You can scale RPC servers horizontally, see Load balancing docs.

[cjamison]

Thanks!