Skip to content

Conversation

@b2cc
Copy link

@b2cc b2cc commented Nov 6, 2025

Add ulimits for nofile to multiple services in docker-compose.

In recent distributions, docker sets max open files to "unlimited" (technically billions) as default, causing several issues for the RabbitMQ and taiga-async containers.

RabbitMQ

The ErlangVM allocates memory depending on max available file handles on startup. If ulimit is set to "unlimited", RabbitMQ takes 2GB RAM+ during startup, and settles around 1.6GB. With two RabbitMQs starting, this is a guaranteed OOM situation for any server with <8GB RAM (accounting for mem use of the other containers).
This can easily be investigated in the pod with rabbitmq-diagnostics memory_breakdown in other_system.
With ulimit set to "unlimited":

other_system: 1.623 gb (73.61%)
allocated_unused: 0.5082 gb (23.05%)
...

With ulimit set to "32786":

other_system: 0.0132 gb (8.98%)
allocated_unused: 0.0 gb (0.0%)

taiga-async

There seems to be an issue with a library used in the taiga-async stack that works itself through all available file handles during startup. If max open files to "unlimited", depending on CPU power, this can take hours during which the container is running at constant 100% CPU usage until it has worked through all file handles.


Setting sane ulimit values for these containers resolves those problems, and also helps avoid many hard-to-debug issues people have been running into lately. Together with additional PR like #151 it should make the docker-compose deployment more reliable for self-hosters.

After setting the ulimit correctly, the whole stack can be run on a 2GB VPS as advertised.


See also (among others):

Added ulimits for nofile to multiple services in docker-compose.
@b2cc b2cc changed the title Set ulimits for nofile in RabbitMQ and async services to avoid high CPU and OOM issues Set ulimits for nofile in RabbitMQ and taiga-async services to avoid high CPU and memory OOM issues Nov 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant