Skip to content

Memory usage increases rapidly as the number of tests grows #10605

Open

Description

@Lucas-C

Hi !

I am witnessing this memory usage increase on fpdf2, using memunit: py-pdf/fpdf2#641
A contributor also mentioned that it was reported on StackOverflow some years ago:
https://stackoverflow.com/questions/20900380/py-test-excessive-memory-usage-with-large-number-of-tests

In short: for a tests suite with ~1000 unit tests, I have witnessed an increase in memory usage of more than 10MB for 40 unit tests added.

Is this surprising to you, or expected?

And secondly, could there be optimizations to make, in pytest or in my tests suite, in order to lower this memory usage?

I have searched the issues & discussions history about this but found nothing anwering those questions:

Activity

  1. RonnyPfannschmidt commented on Dec 22, 2022

    @RonnyPfannschmidt
    Member

    I wonder how much this changes with the -s flag or disabling the logging plugin

  2. Lucas-C commented on Dec 22, 2022

    @Lucas-C
    ContributorAuthor

    I wonder how much this changes with the -s flag or disabling the logging plugin

    On my computer, measures were similar when using -s or -p no:logging, no big difference.

  3. added
    status: help wanteddevelopers would like help from experts on this topic
    type: performanceperformance or memory problem/improvement
    on Dec 23, 2022
  4. nicoddemus commented on Dec 23, 2022

    @nicoddemus
    Member

    Is this surprising to you, or expected?

    Definitely something is fishy, we routinely have thousands of tests in our suites at work, and we don't see a memory growth like that.

  5. TobiMcNamobi commented on Mar 28, 2025

    @TobiMcNamobi
    Contributor
  6. RonnyPfannschmidt commented on Mar 28, 2025

    @RonnyPfannschmidt
    Member

    Nope

    More than 100k tests is a entirely different league than this

  7. pourushsiddharth commented on Oct 30, 2026

    @pourushsiddharth

    Here is a minimal reproduction for the memory growth observed with large parametrized test suites in this issue:

    Reproduction files

    Example

    python run_memory_repro.py --case param --items 5000
    python run_memory_repro.py --case param --items 5000 --collect-only
    python run_memory_repro.py --case param --items 5000 --no-terminal

    Using Python 3.13 and pytest 9.2, the reproduction showed approximately:

    • 60 MiB RSS increase for a normal run with 5,000 parametrized tests
    • 32 MiB RSS increase with --collect-only
    • 7 MiB RSS increase for 1,000 tests with the terminal plugin disabled

    This does not by itself prove a single leak, but it demonstrates that memory usage grows substantially with the number of collected parametrized items. The results also suggest that collection/parametrization metadata accounts for a significant part of the baseline, while reporting and terminal handling add additional overhead during normal execution.

  8. RonnyPfannschmidt commented on Oct 30, 2026

    @RonnyPfannschmidt
    Member

    🤖 Written by Claude Opus 5.5 via Claude Code for the pytest maintainers; I prompted it, it did the work, I read it.

    Thanks for the repro. I re-measured on current main (Python 3.13), splitting RSS into collection and run phases, with and without tracemalloc:

    N collect run run with -p no:terminal
    1k 2.6 MiB 2.5 MiB 0.4 MiB
    5k 14.0 MiB 13.6 MiB 2.1 MiB
    10k 27.2 MiB 27.3 MiB 5.0 MiB

    Observations:

    • tracemalloc roughly doubles RSS. With it enabled I get the numbers from the repro: 59 MiB at 5k, and 31 MiB with --collect-only. Without it, the totals are 27.6 MiB and 14.3 MiB.
    • Collection costs about 2.8 KB per item, which is the item, callspec, keywords and nodeid. It grows linearly, which is expected.
    • The run phase costs about 2.8 KB per test, and nearly all of it comes from the terminal reporter. It keeps every TestReport in stats: 30,000 reports at 10k tests (setup, call and teardown). Without it, the run phase costs about 0.5 KB per test. Not keeping the passed setup/teardown reports in stats[''] could be a real saving.
    • None of this explains the original report. That one showed about 10 MB per 40 tests, which is about 250 KB per test and roughly 50 times more than this. That suggests something suite-specific is being retained. A measurement on the fpdf2 suite itself that counts objects still alive after each test would be the useful next step.

    Generated by Claude Code

  9. pourushsiddharth commented on Oct 30, 2026

    @pourushsiddharth

    I reproduced this on Python 3.13 with pytest 9.2. The growth is reproducible and roughly linear with the number of parametrized cases.

    In collection-only runs, TestReport and CollectReport counts were zero, while one Function item and one CallSpec were retained per parameter case. The largest allocations came from pytest’s node, parametrization, mark, and fixture metadata.

    This suggests high parametrized collection memory retention rather than an unbounded leak. The test body and retained test object do not appear to be the source. Execution can add report-related memory on top of the collection cost.

  10. AkaashParmar commented on Nov 11, 2026

    @AkaashParmar

    Hi! I'd like to investigate this issue as an external contributor.

    I reviewed the recent memory profiling results, particularly the memory retained by the terminal reporter in its stats collection.

    Would you be open to a focused contribution that reduces unnecessary retention of passed setup/teardown reports, provided the existing reporting behavior and test compatibility are preserved?

    I'd first prepare a benchmark and regression tests to validate the memory improvement.

    Please let me know if this direction would be useful.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    status: help wanteddevelopers would like help from experts on this topictype: performanceperformance or memory problem/improvement

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions