Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support retry when batch write items #195

Open
yoshidan opened this issue Feb 21, 2024 · 1 comment
Open

Support retry when batch write items #195

yoshidan opened this issue Feb 21, 2024 · 1 comment
Labels
enhancement New feature or request

Comments

@yoshidan
Copy link
Contributor

yoshidan commented Feb 21, 2024

An error occurred when executing dy import.

275 items processed (98.58 items/sec)Error: 
BatchWriteError(Service(ProvisionedThroughputExceeded("The level of configured provisioned throughput for the table was exceeded. Consider increasing your provisioning level with the UpdateTable API.")))

AWS guidelines recommend retries with Exponential Backoff.
https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Programming.Errors.html#Programming.Errors.BatchOperations
Therefore, in my project, I have forked this repository and added Exponential Backoff by backon.
Do you plan to add retry handling in the future?

@yoshidan
Copy link
Contributor Author

I met other exceptions.

ThrottlingException

com.amazon.coral.availability#ThrottlingException\",\"message\":\"Throughput exceeds the current capacity of your table or index. DynamoDB is automatically scaling your table or index so please try again shortly. If exceptions persist, check if you have a hot key: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/bp-partition-key-design.html\"}", headers: {"server": "Server", "date": "Mon, 26 Feb 2024 06:54:54 GMT", "content-type": "application/x-amz-json-1.0", "content-length": "367", "connection": "keep-alive", "x-amzn-requestid": ...

InternalServerError

[2024-02-26 18:27:57] Error: BatchWriteError(Service(InternalServerError("Internal server error")))
[2024-02-26 18:27:58] exit status 1

2024e41
51694aa
also allows retries for ThrottilingException and InternalServerError.
Ideally, it would be better to consider retry for all of the following retrievable exceptions.

https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Programming.Errors.html#Programming.Errors.MessagesAndCodes

@StoneDot StoneDot added the enhancement New feature or request label Mar 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants