[Scalability] Enable Cluster Autoscaler (CA) for Alcor deployment

[xieus]

Feature Requirements

The K8s Cluster Autoscaler is a component that automatically adjusts the size of a Kubernetes Cluster so that all pods have a place to run and there are no unneeded nodes. This naturally fits into Alcor's performance requirement to scale up the number of nodes that hosts Alcor control plane when the load increases, otherwise cut the deployment cost of Alcor control plane.

Detailed Requirements

• Define ClusterAutoscaler policy
• Tune scaling policies including ScaleDown and ScaleUp policies. Make recommended default behavior for Alcor deployment.
• Stress testing a few key microservices (PM for example) to trigger CA conditions
  * Scale up: There are pods that failed to run in the cluster due to insufficient resources.
  * Scale down: there are nodes in the cluster that have been underutilized for an extended period of time and their pods can be placed on other existing nodes.

Reference

K8s CA: https://github.com/kubernetes/autoscaler

[xieus]

@kevin-zhonghao When you config HPA (Issue #635), if you can take CA as well, that would be great.

[kevin-zhonghao]

@kevin-zhonghao When you config HPA (Issue #635), if you can take CA as well, that would be great.

NP, I'll try to configure both of them