fix(core): fail over immediately on rate limits in FallbackModel chains - #117
Conversation
A 429 with a large Retry-After made _retry_on_error sleep for the full provider cooldown (unbounded, up to minutes) before exhausting retries and handing off — turning "fallback in <1s" into multi-minute stalls. - Cap retry sleeps at MAX_RETRY_DELAY (30s) even when Retry-After is larger - New fail_fast_rate_limit: raise on 429/RateLimitError without retrying; FallbackModel sets it for every entry that still has a fallback behind it - Last entry in the chain (and single models) retry in place as before
PR SummaryMedium Risk Overview
Tests cover huge Reviewed by Cursor Bugbot for commit 88142ea. Bugbot is set up for automated code reviews on this repo. Configure here. |
A 429 with a large Retry-After made _retry_on_error sleep for the full provider cooldown (unbounded, up to minutes) before exhausting retries and handing off — turning "fallback in <1s" into multi-minute stalls.