-
Notifications
You must be signed in to change notification settings - Fork 232
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
SDK doesn't appear to use Retry-After on 429s #978
Comments
This seems to be caused by the fact that the Because calling Graph from client-side apps is a CORS request, clients are only allowed to access a predefined set of safe headers, plus additional headers defined by the server as safe using the To fix this issue, we'd need to add the |
Quick update on this issue. We are working with the AGS team to add this header to the |
@sebastienlevert I'm not seeing this header in the response to graph API requests. I should expect to see it, right? (I assume you closing the topic means it was exposed at some point) I do see |
Some APIs might not send a "Retry-After" and the SDK will automatically backoff exponentially. |
@sebastienlevert are you sure about the exponential backoff? Here's what I'm seeing: Note the timestamps. Retries seem to be happening every few seconds without any backoff. |
Based on this, it should https://github.com/microsoftgraph/msgraph-sdk-javascript/blob/dev/src/middleware/RetryHandler.ts#L136. What is the version of the SDK you are using? |
I'm on @microsoft/microsoft-graph-client v3.0.7 (latest). I realized what is going on: I am trying to fix this in my code by having a much higher FWIW I think it would make sense to have a MUCH higher default |
Even that doesn't work because That's not a long enough wait for the OneNote API throttling which is per hour. I need to be able to specify a number of retries that will cause the graph API SDK to wait >= 1 hour. |
Ideally, you shouldn't need to guess the retry value. That said, based on your observation @altano, @sebastienlevert I wonder if we should increase the retry number to 12 to accommodate the reset period of 1 hour for OneNote APIs. |
Some of the graph APIs do not provide the `retry-after` header in HTTP 429 throttling responses. The default behavior of this SDK is to retry 3 times, with exponential backoff. The problem with this approach is that the max retry of 3 attempts (with a delay of ~8 seconds) just isn't enough for some APIs. For example, OneNote's API will potentially return 429s for 1 hour. To fix this, let us create some new defaults: | Thing | Old Value | New Value | Reasoning | | ------------------- | ----------- | -------------- | ------------------------------------------------------------------------------------------------------------------------- | | DEFAULT_MAX_RETRIES | 3 | 12 | 2^12 seconds is just over 1 hour, which should be enough for OneNote | | MAX_DELAY | 180 seconds | 3_600 (1 hour) | This makes it so that retry 12+ only wait 1 hour between retries instead of continuing to backoff the delay exponentially | | MAX_MAX_RETRIES | 10 | 64 | This needs to be higher than DEFAULT_MAX_RETRIES but the choice of 64 is arbitrary | See [this GitHub issue](microsoftgraph#978) for more discussion.
Bug Report
Prerequisites
Description
During testing of a tool to simulate 429s on the network layer (i.e. outside of the JavaScript library) it was observed that a when a Retry-After or 5s was returned to the client application the SDK issued a retry request before the 5s window had elapsed
Console Errors: N/A
Screenshots:
Steps to Reproduce
Expected behavior: [What you expected to happen]
SDK should not be retrying until after the number of seconds provided in the
Retry-After
headerActual behavior: [What actually happened]
SDK retries the request at 3s and eventually succeeds due to the exponential backoff behavior
Additional Context
Add any other context about the problem here..
Chaos Proxy is here: https://github.com/microsoftgraph/msgraph-chaos-proxy (Microsoft private repo at present)
Usage Information
Request ID - N/A
SDK Version - script link to https://cdn.jsdelivr.net/npm/@microsoft/microsoft-graph-client/lib/graph-js-sdk.js so I assume latest
The text was updated successfully, but these errors were encountered: