Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

kenesisStreamReady produces LimitExceededException when calling DescribeStream #42

Open
StevenMcVicker opened this issue Sep 16, 2019 · 2 comments
Open

kenesisStreamReady produces LimitExceededException when calling DescribeStream #42

StevenMcVicker opened this issue Sep 16, 2019 · 2 comments

Comments

@StevenMcVicker
Copy link
Contributor

We are experiencing very frequent LimitExceededExceptions when starting our kinsumer applications. This is a result of the hard 10 TPS rate limit on DescribeStream calls in AWS.

Right now we are avoiding this by creating a kinsumer with the NewWithInterfaces constructor and passing in a custom kinesisiface.KinesisAPI with a workaround implementation of DescribeStream that calls ListShards instead.

It would be great if there was a config toggle to use an alternate method to check kenesis stream readiness so that we can avoid this issue.
@StevenMcVicker StevenMcVicker mentioned this issue Sep 16, 2019
Merged
@garethlewin
Copy link
Contributor

Hi, @StevenMcVicker I'll take a look soon, but are you using the latest? I thought that #21 addressed this (if it doesn't I'm just going by the description of the issue and it might not be related)

@garethlewin
Copy link
Contributor

Oh, I see #21 didn't replace all uses of DescribeStream.

@garethlewin garethlewin mentioned this issue Apr 6, 2020
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@garethlewin @StevenMcVicker