Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Question for HealthChecks.Redis behavior #2288

Open
cwhsu1984 opened this issue Aug 20, 2024 · 1 comment
Open

Question for HealthChecks.Redis behavior #2288

cwhsu1984 opened this issue Aug 20, 2024 · 1 comment

Comments

@cwhsu1984
Copy link

What would you like to be added:

The checking for redis requires all nodes to be availble for it to return healthy.
We recenlt get caught in an issue due to we use the healthcheck for readiness and liveness probe for container, During our redis cluster maintenance, we need to upgrade node by node. We need to take down a node which cause the healthcheck to return unhealthy in the response, and hence, our application is down.

Why is this needed:

The reason that I feel this is unexpected is because redis cluster is still working if just one slave node down. It may be unstable, but still working.
Is this checking behavior is expected?
Is there any risk if we simply connect to one node and make sure cluster_state:ok?

@idkburkes
Copy link

I am facing a similar problem statement where a certain number N nodes up still indicates my system is healthy due to only needing a quorum of N Redis nodes to do work. I am exploring a custom health check to implement this logic. However, since the use case is seemingly common for Redis it would be nice to have built-in aggregated healthcheck for multiple nodes with requirement that N nodes up is still healthy.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants