Skip to content

Conversation

pythrick
Copy link

@pythrick pythrick commented Aug 2, 2025

Description

This PR implements dogpile prevention (also known as cache stampede mitigation) to address the issue described in #203.

Problem

When multiple concurrent requests hit an expired or missing cache entry, they all attempt to recompute the value simultaneously, causing unnecessary load on the backend. This is known as the "dogpile" or "cache stampede" problem.

Solution

Implemented an optimistic locking mechanism inspired by aiocache's OptimisticLock that:

  • Allows only the first request to compute the expensive operation
  • Makes subsequent requests wait for the computation to complete
  • Returns the cached result to all waiting requests once available

Key Features

  • Configurable per-endpoint or globally: Can be enabled/disabled and configured at both global and per-endpoint levels
  • Backend-agnostic locking: Works with Redis (using atomic operations) and InMemory backends
  • Graceful timeout handling: Configurable wait times and grace periods to prevent indefinite blocking
  • Zero overhead when disabled: No performance impact when dogpile prevention is not enabled

Implementation Details

  • Added class for coordinating cache computation
  • Added and for backend-specific implementations
  • Extended decorator with dogpile prevention parameters
  • Added comprehensive tests covering concurrent scenarios

Configuration

# Global configuration
FastAPICache.init(
    backend=backend,
    enable_dogpile_prevention=True,
    dogpile_grace_time=60.0,
    dogpile_wait_time=0.1,
    dogpile_max_wait_time=5.0
)

# Per-endpoint configuration
@cache(expire=60, enable_dogpile_prevention=True)
async def expensive_operation():
    ...

Testing

  • Added comprehensive test suite in
  • Tests cover single requests, concurrent requests, timeouts, and different backends
  • All existing tests pass without modification

Closes #203

Signed-off-by: Patrick Rodrigues <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Dogpile race solving / cache key lock & wait, maybe with aiocache?
1 participant