Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stopping the peer does not interrupt retry loops #6

Open
sundbry opened this issue May 4, 2019 · 0 comments
Open

Stopping the peer does not interrupt retry loops #6

sundbry opened this issue May 4, 2019 · 0 comments

Comments

@sundbry
Copy link

sundbry commented May 4, 2019

This plugin can hang in situations where the write-batch is stuck in a long loop, especially if you have a long retry timeout period. When the instance is (stop)ped, however, it doesn't interrupt this loop, and it can stick around still running in the background. This has some negative consequences

  1. It prevents jobs from checkpointing their output state and you notice the error when you try to resume
  2. it causes issues with background threads when reloading a lot during development

We could mitigate this by setting a smaller retry timeout, but I like to have a long one (especially in production) to cover any down time windows in external services to give them time to recover.

sundbry added a commit to arctype-co/onyx-http that referenced this issue May 4, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant