Merged
Conversation
…g, deduplication, and rate limiting - Added `InMemoryCacheStore`, `InMemoryDedupeStore`, and `InMemoryRateLimitStore` for in-memory data management. - Implemented `SQLiteCacheStore`, `SQLiteDedupeStore`, and `SQLiteRateLimitStore` for persistent storage using SQLite. - Enhanced the `ComicVine` client to support custom store options for caching and rate limiting. - Created example usage files demonstrating the integration of these stores with the ComicVine client. - Added comprehensive tests for all new store implementations to ensure functionality and reliability.
- Added '**/dist' to .gitignore to prevent build artifacts from being tracked in version control.
- Introduced a new markdown file outlining best practices for TypeScript type safety. - Included guidelines on avoiding `any` type, forceful casting, and preferred approaches for type handling. - Provided examples of both incorrect and correct usage to enhance understanding of TypeScript features.
- Added a metadata section to the TypeScript type safety rules markdown file to ensure consistent application of the rules. - This update enhances the structure of the documentation for better clarity and usability.
- Introduced a new markdown file to enforce the use of pnpm for package management in the project. - Provided guidelines for using pnpm commands instead of npm or yarn, ensuring consistency in package management practices. - Included project context to clarify the environment and structure of the TypeScript library for the Comic Vine API.
- Revised pnpm package manager rules to ensure consistent application and updated the description for clarity. - Enhanced TypeScript type safety rules with additional guidelines on using `unknown` and avoiding type escape hatches, along with improved examples for better understanding.
- Deleted the old `.eslintrc.json` file and migrated to a new ESLint configuration structure using `eslint.config.js`. - Introduced a base ESLint configuration in `packages/eslint-config/base.js` for consistent linting across packages. - Updated package.json files to include new ESLint and TypeScript configurations. - Added shared configurations for `tsup` and `vitest` in their respective packages to streamline build and testing processes. - Enhanced TypeScript type safety by replacing `any` with `unknown` in various store implementations. - Updated pre-commit hooks to use the new linting and formatting commands.
- Updated test files in `sqlite-cache-store` and `sqlite-dedupe-store` to specify promise types for better type safety. - Changed `const promises = [];` to `const promises: Promise<void>[] = [];` in both test files. - Improved clarity and maintainability of concurrent operation tests by explicitly defining promise types.
- Updated various files to use `Array<Type>` syntax instead of `Type[]` for consistency with TypeScript best practices. - Changes made in `comic-vine-stores.test.ts`, `comic-vine.ts`, `url-builder.ts`, `base-resource.ts`, `request.ts`, and multiple store implementations. - Introduced a new test file to verify the ESLint rule for array type syntax. - Enhanced type safety and maintainability of the codebase.
…file - Introduced the '@typescript-eslint/array-type' rule in the ESLint configuration to enforce consistent array type syntax. - Deleted the obsolete test file 'test-array-syntax.ts' as it is no longer needed with the new rule in place.
- Changed all instances of '@comic-vine' to '@repo' in ESLint, tsup, and vitest configurations across multiple packages. - Updated package.json files to reflect the new package names for consistent dependency management. - Ensured that ESLint configurations in various packages now import from the updated repository structure. - Enhanced maintainability and clarity by standardizing package references throughout the codebase.
- Introduced ResourceInterface to standardize the retrieve and list methods across resources. - Updated ComicVine class to utilize the new interface, eliminating unnecessary type casting. - Improved type safety and clarity in method implementations for resource handling.
…r-middleware-logic-49e2 Refactor HTTP client for middleware logic
- Included the .notes file in the .gitignore to prevent it from being tracked by Git. - This change helps maintain a cleaner repository by ignoring temporary or personal files.
- Adjusted the import statements in both `comic-vine-stores.test.ts` and `http-client.ts` for better organization and clarity. - Ensured that the necessary stores are imported correctly in the test file, improving code readability.
- Enhanced documentation for the HttpClientOptions interface by adding clearer JSDoc comments for each property. - Improved code readability and maintainability by ensuring that the purpose of each option is well-defined. - Fixed a minor regex pattern in the extractResourceType method for consistency.
- Updated the regex pattern in the extractResourceType method of HttpClient to use consistent syntax for matching URLs. - This change improves code clarity and ensures proper extraction of resource types from API URLs.
….test.ts - Updated the import statements to include HttpClientStores and HttpClientOptions for improved type safety. - Refactored axios instance definition to ensure proper typing. - Cleaned up comments and organized mock setup for better clarity and maintainability in the test file.
- Introduced a new RateLimitConfig interface for per-resource rate limiting configuration. - Updated import statements across in-memory and SQLite stores to reference the new RateLimitConfig from the client package. - Cleaned up redundant RateLimitConfig definitions in store implementations to enhance type safety and maintainability.
- Added DEFAULT_RATE_LIMIT constant to standardize rate limit configuration across stores. - Updated in-memory and SQLite rate limit store constructors to use DEFAULT_RATE_LIMIT instead of hardcoded values. - Enhanced maintainability by centralizing rate limit settings in the rate-limit-config module.
- Deleted the REFACTORING_SUMMARY.md file to streamline documentation and reduce redundancy. - This change reflects the completion of the HTTP client refactoring and consolidates relevant information into more appropriate locations.
- Removed "vitest/globals" from the types array in both tsconfig.base.json and tsconfig.dev.json to streamline type definitions. - This change enhances clarity and ensures that unnecessary type references are eliminated from the configuration files.
- Updated test files for both in-memory and SQLite rate limit stores to specify the type of promises as Array<Promise<void>> for better type safety. - Simplified the instantiation of SQLiteRateLimitStore by removing unnecessary parameters related to cleanup configuration, enhancing code clarity and maintainability.
AllyMurray
changed the title
- Upgraded better-sqlite3 dependency from version 11.10.0 to 12.2.0 across multiple packages for improved performance and features. - Removed unnecessary comments and improved code clarity in the ComicVine and SQLite stores. - Enhanced type safety by ensuring the BetterSQLite3Database type includes a close method. - Streamlined cache and dedupe store implementations by simplifying job expiration checks and cleanup logic.
… and cleanup logic - Removed the close method type assertion from SQLiteCacheStore and SQLiteDedupeStore, ensuring better type safety with BetterSQLite3Database. - Simplified the destroy method in all SQLite store classes by delegating to the close method, improving code clarity and maintainability. - Updated cleanup logic to check for the existence of the close method before invoking it, preventing potential runtime errors.
…eStore functionality - Introduced a new example file demonstrating the setup and usage of a memory-managed cache with the InMemoryCacheStore. - Enhanced InMemoryCacheStore to support memory limits, item eviction based on LRU strategy, and improved memory usage calculations. - Added tests for memory management features, including item eviction and memory utilization tracking. - Updated the index file to export InMemoryCacheStoreOptions for better configuration handling.
- Updated the job count retrieval logic in SQLiteDedupeStore to remove unnecessary type assertions, enhancing code clarity and maintainability. - This change ensures that the counts default to zero when no results are found, improving robustness.
- Updated CacheStore and DedupeStore interfaces to use generic types, improving type safety across implementations. - Modified InMemoryCacheStore, InMemoryDedupeStore, SQLiteCacheStore, and SQLiteDedupeStore to support the new generic type definitions, ensuring consistent handling of cached values and request results. - Improved type handling in related methods to prevent potential runtime errors and enhance code clarity.
- Updated the URL builder to ensure proper type conversion for fieldList values, enhancing type safety. - Refactored SQLiteCacheStore to store the Database instance separately, improving code clarity and maintainability. - Simplified database size retrieval logic by using the sqlite instance directly, ensuring more accurate and readable code.
…he stores - Added fast-safe-stringify dependency to handle circular references and ensure deterministic string lengths for memory calculations. - Updated InMemoryCacheStore to use fast-safe-stringify instead of JSON.stringify for accurate size estimation of cached items. - Enhanced InMemoryDedupeStore with a deferred promise helper to improve type handling and avoid non-null assertions.
…Lite stores - Updated SQLiteCacheStore to ensure safe parsing of database size values, defaulting to zero if parsing fails. - Modified SQLiteDedupeStore to handle both '__UNDEFINED__' and '__NULL__' results consistently, improving type safety and clarity in result handling. - Adjusted test expectations to reflect changes in result handling, ensuring accurate assertions in SQLiteDedupeStore tests.
- Added a maximum entry size limit to SQLiteCacheStore to prevent caching of values that exceed specified byte size, enhancing stability and performance. - Introduced tests to verify behavior for values exceeding and within the max entry size, ensuring correct functionality and expectations.
…equivalently - Updated the hashRequest function to normalize numeric and boolean values to their string representation before sorting, ensuring consistent hashing for parameters like `10` and `'10'`. - Added a new test case to verify that the hashing function treats numeric and string representations of the same value as equivalent, improving cache consistency.
- Added a wait function that supports cancellation via AbortSignal, allowing for graceful handling of rate-limit waits. - Updated the HttpClient's get method to accept an AbortSignal, enabling users to cancel requests and associated waits. - Enhanced tests to verify that requests are aborted correctly when the signal is triggered, improving user experience and control over HTTP requests.
- Included fast-safe-stringify version 2.1.1 in the lock file to ensure consistent serialization and size estimation across cache stores. - Updated package resolutions to reflect the new dependency, enhancing overall project stability.
- Introduced maxItems and maxMemoryBytes options to limit cache size and prevent excessive memory usage. - Implemented O(1) memory usage tracking with a totalSize counter, improving performance during cache operations. - Updated eviction strategy to remove items based on LRU when memory limits are exceeded. - Enhanced README documentation to reflect new features and usage examples.
- Added unref method to the cleanup interval timer, enabling the Node.js process to exit naturally if this timer is the only event loop item, improving resource management.
…LiteDedupeStore - Implemented unref for the cleanup interval timers in both SQLiteCacheStore and SQLiteDedupeStore, allowing the Node.js process to exit gracefully when these timers are the only active event loop items, enhancing resource management.
- Changed the type of the params object in HttpClient from Record<string, unknown> to Record<string, string> to enforce consistent string handling for URL parameters, improving type safety and reducing potential errors in request processing.
- Introduced SQLiteCacheStoreOptions, SQLiteDedupeStoreOptions, and SQLiteRateLimitStoreOptions interfaces to allow flexible database connection handling. - Updated constructors in SQLiteCacheStore, SQLiteDedupeStore, and SQLiteRateLimitStore to manage SQLite connections based on provided options, improving resource management. - Ensured that the underlying database connection is only closed if the store instance created it, enhancing stability and preventing potential errors.
…arity - Renamed the repository from "Comic Vine Monorepo" to "Comic Vine API Client" for better alignment with the project's purpose. - Expanded the README to include detailed features, installation instructions, and usage examples for the Comic Vine API client. - Improved documentation for in-memory store implementations, highlighting key features and configuration options. - Added TypeScript support details and examples for both in-memory and SQLite stores, enhancing usability for developers. - Updated the API reference sections to provide clearer guidance on available methods and configurations.
…ions - Changed the constructor to accept a single options object, consolidating parameters for API key, base URL, stores, and client options. - Updated README and examples to reflect the new constructor syntax, enhancing clarity for users. - Improved type safety and consistency across the codebase by ensuring all stores and client options are passed through the new object structure. - Adjusted tests to validate the new constructor behavior and ensure compatibility with existing functionality.
- Introduced a new CLAUDE.md file to provide comprehensive guidance for developers working with the Comic Vine API client. - Included sections on development commands, architecture overview, TypeScript conventions, testing strategy, and build system details. - Enhanced documentation to facilitate better understanding of the project's structure and development workflow.
- Fix CommonJS module compatibility by renaming .js test files
to .cjs
- Correct store method expectations in build tests:
- CacheStore: get, set, delete, clear
- DedupeStore: waitFor, register, complete, fail,
isInProgress
- RateLimitStore: canProceed, record, getStatus, reset,
getWaitTime
- Standardize package.json path fields with consistent ./
prefixes
- Enhance SQLite store test mocks with proper raw().all()
method chains
- Add missing defaultConfig for SQLiteRateLimitStore test
instances
- Update functionality tests to use correct store APIs instead
of generic methods
…rt details - Enhanced the README to provide a detailed overview of the Comic Vine API client, including installation instructions, configuration options, and advanced usage examples. - Added sections on TypeScript support, highlighting type safety, intelligent field selection, and auto-completion features. - Included comprehensive documentation for pagination, filtering, and error handling, improving clarity for developers. - Updated examples to demonstrate both manual and automatic pagination, showcasing the library's capabilities.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
This branch introduces a comprehensive request optimization system for the @comic-vine/client SDK, implementing caching, deduplication, and rate limiting through a pluggable store architecture.
🏗️ Architecture Changes
Core Client Integration
packages/client/src/comic-vine.ts- Integrated store functionality directly into ComicVine client with object-based constructor optionspackages/client/src/index.ts- Updated exports to include store interfaces and typespackages/client/src/comic-vine-stores.test.ts- Tests for integrated store functionalityStore Interface System
packages/client/src/stores/cache-store.ts- Cache interface with TTL supportpackages/client/src/stores/dedupe-store.ts- Request deduplication interfacepackages/client/src/stores/rate-limit-store.ts- Rate limiting interfacepackages/client/src/stores/rate-limit-config.ts- Rate limiting configuration typespackages/client/src/stores/request-hasher.ts- SHA-256 request hashing utility with numeric parameter normalizationpackages/client/src/stores/index.ts- Consolidated store exportsHTTP Client Enhancements
packages/client/src/http-client/http-client.ts- Added abortable rate limiting with graceful request queuingpackages/client/src/http-client/http-client-factory.ts- Updated factory for store integration📦 New Packages
@comic-vine/in-memory-store
Complete package with Map-based implementations:
InMemoryCacheStore- TTL-based caching with LRU eviction, memory management, and graceful cleanupInMemoryDedupeStore- Promise-based request deduplicationInMemoryRateLimitStore- Sliding window rate limiting with per-resource configuration@comic-vine/sqlite-store
Complete package with persistent implementations:
SQLiteCacheStore- Database-backed caching with size guards and fast-safe-stringify serializationSQLiteDedupeStore- Cross-process deduplication with automatic cleanup and unref supportSQLiteRateLimitStore- Persistent rate limiting with sliding windows🧪 Test Coverage
Comprehensive Test Suites Added:
packages/client/src/stores/request-hasher.test.ts- Request hashing and parameter normalization testspackages/in-memory-store/src/in-memory-cache-store.test.ts- Memory management and LRU eviction testspackages/in-memory-store/src/in-memory-dedupe-store.test.ts- Request deduplication testspackages/in-memory-store/src/in-memory-rate-limit-store.test.ts- Rate limiting algorithm testspackages/sqlite-store/src/schema.test.ts- Database schema validation testspackages/sqlite-store/src/sqlite-cache-store.test.ts- Persistent caching tests with size managementpackages/sqlite-store/src/sqlite-dedupe-store.test.ts- Cross-process deduplication testspackages/sqlite-store/src/sqlite-rate-limit-store.test.ts- Persistent rate limiting tests📚 Documentation & Examples
examples/rate-limited-client-usage.ts- Comprehensive rate limiting examples with per-resource configuration and error handlingexamples/memory-managed-cache-usage.ts- Memory-managed caching examples with LRU eviction and statistics monitoringpackages/in-memory-store/README.md- In-memory store documentationpackages/sqlite-store/README.md- SQLite store documentation with deployment guidesCLAUDE.md- Development guidance and architecture overview✨ Key Features
Enhanced Constructor Pattern
Advanced Request Flow
Production Enhancements
This implementation provides a complete, production-ready solution for optimizing API requests with advanced memory management, cross-process coordination, and robust error handling while maintaining full backward compatibility.