[kubernetes] Support `disk_size` and `disk_tier`

[clayrosenthal]

Right now sky doesn't support disk_size or disk_tier for kubernetes. I think a good way to support disk_size in kubernetes would be to have a config field to map disk_tiers to kubernetes StorageClass.

ie:

kubernetes:
  disk_tiers:
    low:
      storage_class_name: 'my-slow-sc'
      access_mode: ReadWriteMany  # optional
    medium:
      storage_class_name: 'a-better-sc'
    high:
      storage_class_name: 'way-better-sc'
    ultra:
      storage_class_name: 'super-fast-sc'
      access_mode: ReadWriteOnce

Correct me if I'm wrong, but I assume best just picks the best of the above tiers.

I imagine you could also configure this per context for heterogeneous clusters.

[clayrosenthal]

could also help with #6107, where if storage classes are defined, you can use StatefulSets

[andylizf]

@clayrosenthal We do have some supports in k8s based HA Controller. The spec of that controller pod will use the specified storage_class and disk_size.

Once we have k8s level disk_size and disk_tier, we should thinking about the relationship with those HA configs.