Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Redis does not reconnect as expected. #3505

Open
abulvenz opened this issue Jun 17, 2024 · 4 comments
Open

Redis does not reconnect as expected. #3505

abulvenz opened this issue Jun 17, 2024 · 4 comments
Labels
bug Something isn't working

Comments

@abulvenz
Copy link
Contributor

abulvenz commented Jun 17, 2024

Describe the bug
Reflex does not reconnect to redis when connection was lost.

To Reproduce
Steps to reproduce the behavior:

  • Start app using redis
  • Stop redis
  • Immediately restart redis
  • When doing some state changes you only see redis.exceptions.ConnectionError: Connection closed by server.
  • The frontend is dead by now.
  • Restart app, everything works again.

Expected behavior
Reflex should automatically try to reconnect on connection errors and log a warning.

Are there any configuration options that I missed?

@abulvenz abulvenz added the bug Something isn't working label Jun 17, 2024
@ElijahAhianyo
Copy link
Collaborator

ElijahAhianyo commented Jun 17, 2024

seems like the connection is created when the StateManager is created which is done once. We probably need a polling mechanism that times out after x period before connectionError is thrown

@abulvenz
Copy link
Contributor Author

abulvenz commented Jun 17, 2024

@ElijahAhianyo Thanks for responding so fast!
Maybe we can also try to solve this via configuration?
The Redis-constructor seems to accept some retry-arguments.

@abulvenz
Copy link
Contributor Author

@ElijahAhianyo I'm sorry, just checked, that retrying is already the default for the client, but not for individual calls on the client.

So we would need to have something in redis that catches the errors and either retries the state change/loading or skips it.
Implementation idea of @benedikt-bartscher: Create a @redis_retry-decorator that wraps the relevant functions.

At least the backend should not be blocked permanently by those errors. What do you think?

@benedikt-bartscher
Copy link
Contributor

Related: #3506

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants