Skip to content

Conversation

@VladimirPress
Copy link

@VladimirPress VladimirPress commented Nov 19, 2025

Description

Adds a new per collection-index configuration option use_indexer_date_filter.
When enabled for a collection, pywb forwards from and to date parameters to the indexer, if present.
This shifts date filtering to the indexer, reducing pywb CPU and memory usage, preventing OOM conditions, and lowering network traffic.

Unfortunately, I could not find an appropriate section in the documentation to add this parameter description.

Motivation and Context

This feature allows heavy date filtering to be offloaded to the indexer, which prevents pywb server overload and high memory usage for large collections.

Screenshots (if appropriate):

Types of changes

  • Replay fix (fixes a replay specific issue)
  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have added or updated tests to cover my changes.
  • All new and existing tests passed.

Adds a new per-collection configuration option use_indexer_date_filter.
When enabled for a collection, pywb forwards from and to date parameters to the indexer, if present.
This shifts date filtering to the indexer, reducing pywb CPU and memory usage, preventing OOM conditions, and lowering network traffic.
@VladimirPress
Copy link
Author

VladimirPress commented Nov 19, 2025

I have tested the functionality of this change on my system.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant