Wait for CRD ready state

[rpuserh]

I have CRD which I want to wrap to helm(template).

The question is how do I know if CRD resource is ready?
In other words, I want cli to wait until CRD becomes ready when using --wait

[gjenkins8]

From the docs:

It might take a few seconds for the endpoint to be created. You can watch the Established condition of your CustomResourceDefinition to be true or watch the discovery information of the API server for your resource to show up.

IMHO it would be good if helm would wait for the CRD resource to be available (Established) here. I think a PR would be welcome.

[joejulian]

The wait flag already waits for CRD readiness:

helm/pkg/kube/ready.go

Lines 153 to 163 in 76157c6

	case *apiextv1beta1.CustomResourceDefinition:
	if err := v.Get(); err != nil {
	return false, err
	}
	crd := &apiextv1beta1.CustomResourceDefinition{}
	if err := scheme.Scheme.Convert(v.Object, crd, nil); err != nil {
	return false, err
	}
	if !c.crdBetaReady(*crd) {
	return false, nil
	}

[rpuserh]

I'm trying to use terraform k8s operator (but it's not waiting even when using --wait flag with helm )

also reached terraform k8s operator team here is ticket: hashicorp/terraform-k8s#155

[joejulian]

That's not terraform, that's FluxCD. I see you say it takes 15 minutes. The helm default is 5 minutes. Perhaps you need to extend the timeout. For FluxCD, see their documentation: https://fluxcd.io/flux/components/helm/helmreleases/

For support with FluxCD, see https://fluxcd.io/support/#getting-help-from-the-flux-community

[rpuserh]

HELM status changes to ready in a few seconds.
However, helm created resource is not ready (it becomes ready ~15 min).

I want helm to wait for created resources to become ready.

[joejulian]

Ah, so you don't mean the CRD, you mean the custom resource. There's no way to determine when a custom resource is ready - or even if a custom resource can be ready. There's no standard. This would need to be tackled in kubernetes before helm could do anything.

[aweis89]

@joejulian since there's no official standard, nor an indication of whether there will be one or whether it's useful for other use-cases, why not just create a simple interface and check if it's implemented so we have a way to achieve this?

Could be as simple as:

type Readiness interface {
    IsReady() bool
}

Then we can check if the interface is implemented in the IsReady check.

Not supporting any way to incorporate readiness for the CR, can cause all sorts of issues with deployment pipelines and even just generally using helm for templating CRs.

[joejulian]

Maybe you can do that with hooks: set up a pod that checks the custom resource for your desired status. It can't be done as part of the helm binary though. It's not up to helm to create kubernetes standards. iirc, that's under the authority of sig-apimachinery.

[zmberg]

@joejulian Can helm support CR like this?

[drewwells]

Can we update the title to CR and re-open this? Helm does not support ready status of CRs: https://github.com/helm/helm/blob/main/pkg/kube/ready.go#L88

[drewwells]

Just checking how kubectl works, CRDs define additional printer columns that kubectl will use.

https://github.com/kubernetes/kubernetes/blob/55f2bc10435160619d1ece8de49a1c0f8fcdf276/staging/src/k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/types.go#L72

A very standard one is type=Ready, see cert-manager
https://github.com/cert-manager/cert-manager/blob/58151d2998ad85f14e1ea04bed4406df16b6ac7a/deploy/crds/crd-certificaterequests.yaml#L40-L42

Helm could attempt to divine this and return True if it doesn't match