Very frequent new SSLClusterConnection instance

[alseambusher]

Hi!

I see this log statement very frequently although I am using a single RedisCluster object:

Createing new SSLClusterConnection instance

Everything works fine functionally but it is loading the redis cluster with too many connection requests. Is it possible to control how often it creates a new instance?

Here is how I am creating the object:

redis = RedisCluster(host=host,
             port=port,
             password=password,
             skip_full_coverage_check=True,
             ssl=True,
             socket_timeout=2)

[Grokzen]

@alseambusher So this is not super strange as each cluster instance has a connection pool where it can contain many many different connections into each node in your cluster. If memory serves me right, it can be up to 32 connections per node in your cluster. This is expected to be seen from a normal cluster, but yes the connection logging can be a bit verbose when you run at a high load with your client

[umang92]

@Grokzen The reason we created this discussion item is the following.
We have an AWS Elasticache Redis cluster serving our production traffic running 12 shards. It is being connected from multiple python and java services. Java services use the lettuce client and the python services use redis-py-cluster client.

There was a sudden surge in the number of connections on only one shard out of 12. This surge was from 20-30 to 200-210. The surge was followed by CPU utilization spike from 10% to 100% and due to that a lot of requests going to that shard delayed or failed. This whole thing was observed from the AWS provided elasticache dashboards. It happened just for a 1-2 minutes and then the system was back to normal. The shard itself was not killed and the same shard came back to normal state. There was no increase in load nor there was any surge in number of containers connecting to the service. Also is that was the case, the spike would be seen in all the shards.

With the above context, we were investigating whether some logic in the client libraries we use(lettuce or redis-py-cluster) could have led to this surge. And while investigating we found these connection logs which seemed too many. Please let us know if you may have some thoughts on the above.