Skip to content

Add singularity.resourceLimits config option#7352

Open
bentsherman wants to merge 1 commit into
masterfrom
6797-singularity-cpu-memory-limits
Open

Add singularity.resourceLimits config option#7352
bentsherman wants to merge 1 commit into
masterfrom
6797-singularity-cpu-memory-limits

Conversation

@bentsherman

Copy link
Copy Markdown
Member

Close #6797

This PR adds the ability to enforce resource limits on Singularity/Apptainer containers when using the local executor

When singularity.resourceLimits = true, Singularity containers are launched with --cpus and --memory, which prevents them from exceeding their allocated resources. Similar to what Docker does automatically

This feature needs to be opt-in because it requires cgroups in the host environment. Enabling it automatically could break existing code

Signed-off-by: Ben Sherman <bentshermann@gmail.com>
@bentsherman
bentsherman requested a review from pditommaso July 16, 2026 20:22
@bentsherman
bentsherman requested a review from a team as a code owner July 16, 2026 20:22
@netlify

netlify Bot commented Jul 16, 2026

Copy link
Copy Markdown

Deploy Preview for nextflow-docs ready!

Name Link
🔨 Latest commit 6adf215
🔍 Latest deploy log https://app.netlify.com/projects/nextflow-docs/deploys/6a593d700c02690008a715e7
😎 Deploy Preview https://deploy-preview-7352--nextflow-docs.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@christopher-hakkaart christopher-hakkaart left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Docs look good

@pditommaso

Copy link
Copy Markdown
Member

Naming suggestion: this adds resourceLimits (covering both --cpus and --memory), while Docker uses docker.cpuLimits for essentially the same feature. Since docker.cpuLimits was only just added in #7289 (2026-07-13) and has not shipped in a stable release yet, it should be safe to rename it to docker.resourceLimits here so the option name is unified across all container engines. resourceLimits is also the more accurate name since it now governs memory too.

@bentsherman

Copy link
Copy Markdown
Member Author

Well, there is a difference here:

  • docker.cpuLimits only controls whether to use the legacy CPU limits via --cpus. Otherwise it always uses --cpu-shares. And it always specifies --memory limits regardless of this option

  • For Singularity and Apptainer, resourceLimits controls whether to specify cpu/memory limits at all

So the different names actually reflect the different behavior of each container runtime. Forcing them to be the same for consistency's sake would just blur meaning

Although I agree it would be nice to have consistent naming. I think these settings are largely a historical accident, as each container runtime adds support for proper cgroups-based resource limits over time in different ways

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add memory and cpu limits to processes launched through Apptainer / Singularity with a local executor

3 participants