Skip to content

Conversation

@LifeJiggy
Copy link

Summary

This PR adds a PaginationHelper utility class that simplifies handling paginated API responses, enabling developers to easily fetch all items across multiple pages.

Problem

Many Gradient API endpoints return paginated results, requiring developers to manually handle pagination logic. This leads to:

  • Repetitive pagination code across different endpoints
  • Difficulty managing page limits and error handling
  • Manual tracking of pagination state
  • Complex logic for handling different response formats

Solution

Add PaginationHelper class with:

  • Automatic pagination through all pages
  • Configurable page size and maximum pages
  • Support for different response formats (data/items/results)
  • Async pagination support
  • Error handling for pagination end conditions

Key Features

  • Automatic Pagination: Fetches all pages until no more data
  • Configurable Limits: Set page size and maximum pages
  • Format Agnostic: Handles different response structures
  • Async Support: Full async/await compatibility
  • Error Handling: Graceful handling of pagination boundaries
  • Standard Library: No external dependencies

Benefits

  • Eliminates repetitive pagination code
  • Simplifies large dataset handling
  • Improves code maintainability
  • Handles edge cases automatically

Testing

  • Added comprehensive test suite covering:
  • Basic pagination functionality
  • Max pages limits
  • Different response formats
  • Empty responses and edge cases
  • Item extraction from various structures
  • Error handling scenarios

All tests pass with full coverage of pagination functionality.

Usage Examples

from gradient._utils import PaginationHelper

helper = PaginationHelper(page_size=50, max_pages=10)

# Paginate through all models
all_models = helper.paginate(client.models.list)

# Paginate with custom parameters
all_agents = helper.paginate(
    client.agents.list,
    workspace_id="workspace-123"
)

# Async pagination
import asyncio

async def get_all_data():
    all_data = await helper.paginate_async(client.data.list)
    return all_data

# Limit maximum pages to prevent runaway requests
limited_helper = PaginationHelper(page_size=20, max_pages=5)
results = limited_helper.paginate(client.large_dataset.list)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant