Skip to content

fix: disable libvips operation cache to stop image-conversion memory leak - #29

Merged
lesnik512 merged 1 commit into
mainfrom
fix/libvips-cache-leak
Jul 15, 2026
Merged

fix: disable libvips operation cache to stop image-conversion memory leak#29
lesnik512 merged 1 commit into
mainfrom
fix/libvips-cache-leak

Conversation

@lesnik512

Copy link
Copy Markdown
Member

Problem

Long-running services that convert many uploads (e.g. an upload/S3 gateway) show steady process memory growth correlated with traffic that looks like a leak.

Root cause

FileValidator._convert_image converts every image with libvips via pyvips.Image.new_from_buffer(...).write_to_buffer(...). libvips keeps a process-global operation cache enabled by default (up to ~1000 operations, plus memory/file caps). Each cached operation pins references to its image buffers, so across conversions memory climbs until the caps are hit. Every upload is a unique buffer, so the cache never produces a hit: it is pure retained memory with no benefit for this workload.

This is the standard libvips-in-a-server behavior; pyvips' own examples/soak-test.py prevents it with pyvips.cache_set_max(0).

Measured (pyvips 3.1.1 / libvips 8.17.3)

Running the exact conversion in a loop and reading pyvips.cache_get_size():

  • start: 0
  • after 5 default conversions: 35 (one new_from_buffer decomposes into several cached ops, each pinning buffers)
  • after cache_set_max(0): stays 0

Fix

Disable the libvips operation cache on the conversion path with pyvips.cache_set_max(0), guarded by a pyvips.cache_get_max() check so the setter effectively runs once. The import pyvips stays local to _convert_image. No public API change; conversion output, formats, quality, and error handling are unchanged.

Test

Added a TDD regression test asserting pyvips.cache_get_size() == 0 after a conversion. It only observes cache size (never mutates it), so it is order-independent: red on main (nonzero), green with the fix. Full suite: 36 passing, 100% coverage.

🤖 Generated with Claude Code

…leak

libvips keeps a process-global operation cache enabled by default. Every
conversion caches its load operations, pinning image buffers across calls,
so long-running services that convert many uploads leak memory. The cache
never hits for one-shot conversions of unique buffers, so disable it via
pyvips.cache_set_max(0), guarded by cache_get_max() so it runs once.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@lesnik512
lesnik512 requested a review from vrslev July 15, 2026 13:21
@lesnik512
lesnik512 merged commit 18d3d73 into main Jul 15, 2026
6 checks passed
@lesnik512
lesnik512 deleted the fix/libvips-cache-leak branch July 15, 2026 13:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant