[Other] Deployment guideline for multiple installations of single-tenant application

[dario-l]

Community Support Policy

• I have read RabbitMQ's Community Support Policy

RabbitMQ version used

4.0.7

How is RabbitMQ deployed?

Debian package

Steps to reproduce the behavior in question

We have monolithic application which now creates and operates on ~1200 queues. Our current library creates one channel per queue. We have two nodes where application is running so that give us ~2400 channels (each application process have one connection for publishing and another connection for consuming).

Our application is single-tenant. We have 20 tenants so... RabbitMQ must handle (for now) ~48000 channels. This leads to unresponsive RabbitMQ Management UI when any of the application is really busy. Of course we want to sell our product and have more clients (tenants).

RabbitMQ is deployed as a cluster on 3 nodes (node = physical server with Ubuntu). We use virtual hosts for tenant separation.

Reading this section of documentation we are a lot of a head than this and we do not know is it really bad?

I ask here how many channels will create MassTransit and it seems it will be the same number.

At this moment applications work on different servers than RabbitMQ Cluster. Is it a good practice to run a one of rabbit nodes on the same machine where applications run?

We do not use cloud/docker/VMs... We have 5 physical servers. 3 of them are for RabbitMQ Cluster (and Reverse Proxy, and small Redis cache). The 2 of them are for applications.

Is our installation bad? If it is how we could do it better?

Almost all guidelines are for cloud deployment, Docker or Kubernetes. We do not need it... but finding useful information for our situation is really hard.

[michaelklishin]

You can work with 1200 queue using any number of channels, not just 1200. Even one or two (for example, one for publishing and one for consuming) would do.

This recommendation is also applicable for this particular question.

[dario-l]

Thanks for the reply.

That's right. Anyone can make application which can run on two connections and a few channels but... this will be enough for small one.

More advanced systems will suffer and performance will be terrible.

According to the advice and opinions found here and on the dotnet RabbitMQ.Client discussions we have implementation where each component has only one channel for all component subscribers.

Channel (as overall) performs quite nice ~79 transactions per sec (~79/s)

but this channel is used by 300 subscribers and it seams that each individual subscriber waits for other because they are run one after another - just divide ~79/s / 300 = ~0.26/s

...this is really bad fo us. 😞

Really few channels for application is enough? Those guidelines are confusing. 😭