Operator trust the certificate rotated by certmanager in tenant

[cniackz]

Is your feature request related to a problem? Please describe.

Yes, the problem is simple. When certmanager changes the certificate of the tenant, certificate isn't rotated for operator unless manual steps are performed, which is not ideal. We are requesting for an automated way to achieve this.

Describe the solution you'd like

First of all, we should support mutiple tenants, one per secret and secondly, we should keep operator's secrets up to date with respect to the tenant certmanager secret.

related prs:

• Multitenancy Support for External Certs #1971
• Renew external certificates via the added multi-tenancy support #1973

Do they actually works? I have seen them failing in OpenShift, can we test them truly? and the document?

Describe alternatives you've considered

manual process is the alternative for now

Additional context

I am going to describe the steps for you to see this issue:

1. Have a cluster
2. Install operator
3. install certmanager
4. deploy tenant as in our example
5. perform manual steps as in https://github.com/minio/operator/blob/master/docs/cert-manager.md#create-operator-ca-tls-secret
6. then delete certmanager secret in the tenant's namespace, a new one is created by certmanager
7. notice how operator secret isn't rotated hence hitting this common issue below:

I0325 20:03:26.927950       1 monitoring.go:122]
'tenant-certmanager/myminio' Failed to get cluster health:
Get "https://minio.tenant-certmanager.svc.cluster.local/minio/health/cluster": 
tls: failed to verify certificate: x509: certificate signed by unknown authority 
(possibly because of "x509: invalid signature: parent certificate cannot sign this kind of certificate" while trying to verify candidate authority certificate "serial:31299030680238480824367599823199567087")

If rotated, above signature wouldn't happen otherwise.

[sebhoss]

I think this is basically the same as #1986