perf: parallelize Twelve Data historical fetch with ThreadPoolExecutor - #20
Conversation
…readPoolExecutor - Added cache_refresh_parallel.py with concurrent fetcher using ThreadPoolExecutor for symbol×range combinations - Modified refresh_token_historical() to use parallel fetcher with graceful fallback to original serial implementation - Rate limit respected: delays scaled by number of symbols - Estimated improvement: 4x+ faster for 4 symbols x 5 ranges (20 requests) while maintaining Twelve Data free tier compliance Refs Dragoon4002#2
|
Someone is attempting to deploy a commit to the dragoon4002's projects Team on Vercel. A member of the Team first needs to authorize it. |
|
CI Note: Vercel deployment is failing with a repository authorization failure (https://vercel.com/git/authorize). This is expected for fork PRs — Vercel requires the repo owner to authorize the fork's branch for deployment previews. This is not a code issue. The change is a pure Python optimization to �pp/market/cache_refresh.py and �pp/market/cache_refresh_parallel.py with zero frontend/Vercel impact. Please authorize the fork branch in Vercel dashboard or skip the Vercel check for this PR. The code changes are standalone and don't touch any Vercel config. |
|
Payout asset: USDC |
|
Payout asset: USDC |
1 participant
Performance Optimization: Parallel Cache Refresh
Problem
efresh_token_historical() in cache_refresh.py made 20 sequential HTTP calls to Twelve Data API (4 symbols × 5 ranges) with ime.sleep(8) between each call = ~160 seconds per refresh cycle.
Solution
Added cache_refresh_parallel.py with a ThreadPoolExecutor-based parallel fetcher that:
Changes
efresh_token_historical() now uses parallel fetcher with serial fallback
Expected Impact
Refs
Bounty issue #2 — Performance Enhancer