Bookkeeper operator fails to install when used as a Helm dependency

[junglie85]

Description

When specified as a dependency of a Helm chart, the Bookkeeper Operator fails to install due to CRD's not being installed.

apiVersion: v2
name: my-name
description: My description
type: application
version: 0.1.0
appVersion: 0.1.0
dependencies:
  - name: bookkeeper
    version: "0.7.1"
    repository: "https://charts.pravega.io"
  - name: bookkeeper-operator
    version: "0.1.3"
    repository: "https://charts.pravega.io"
...

Helm cannot find the relevant CRD's because they have not yet been installed, so fails with the following error:

Error: unable to build kubernetes objects from release manifest: [unable to recognize "": no matches for kind "BookkeeperCluster" in version "bookkeeper.pravega.io/v1alpha1", unable to recognize "": no matches for kind "PravegaCluster" in version "pravega.pravega.io/v1beta1", unable to recognize "": no matches for kind "ZookeeperCluster" in version "zookeeper.pravega.io/v1beta1"]

The CRD template file has the conditional statement which prevents kubectl apply -f https://raw.githubusercontent.com/pravega/bookkeeper-operator/master/charts/bookkeeper-operator/templates/bookkeeper.pravega.io_bookkeeperclusters_crd.yaml:

error: error parsing https://raw.githubusercontent.com/pravega/bookkeeper-operator/master/charts/bookkeeper-operator/templates/bookkeeper.pravega.io_bookkeeperclusters_crd.yaml: json: line 0: invalid character '{' looking for beginning of object key string

This is equally applicable to the Pravega and Zookeeper CRD's - I'll raise separate issues for those.

Importance

This is a must have for packaging Pravega as part of a larger application.

Location

charts/bookkeeper-operator/templates/bookkeeper.pravega.io_bookkeeperclusters_crd.yaml

Suggestions for an improvement

CRD's can be included in a crd directory and Helm will apply these first. The CRD's could also be provided in a separate Helm chart but I wouldn't find this particularly useful due to not being able to specify a dependency application order. https://helm.sh/docs/chart_best_practices/custom_resource_definitions/

Alternatively, the CRD could be provided as a release artefact that can be installed before running Helm with crd.create=false:

kubectl apply -f https://github.com/pravega/bookkeeper-operator/releases/download/0.1.3/bookkeeper-operator.crds.yaml

[jdmaguire]

The CRD template file has the conditional statement which prevents kubectl apply -f https://raw.githubusercontent.com/pravega/bookkeeper-operator/master/charts/bookkeeper-operator/templates/bookkeeper.pravega.io_bookkeeperclusters_crd.yaml:

As a workaround right now, note that https://github.com/pravega/bookkeeper-operator/blob/master/deploy/crds/bookkeeper.pravega.io_bookkeeperclusters_crd.yaml doesn't have that

This chart was made during the Helm 2 days. Didn't know about the new CRD support in Helm 3. Quite useful, thanks.

[SrishT]

@ambye85 having the crds folder was not required in our use-case since the crd installation is done in the bookkeeper-operator charts and required in the bookkeeper charts. Hence if you can ensure that the dependencies are installed in a particular order, i.e. the bookkeeper-operator installation happens before attempting to install the bookkeeper charts, that should fix the issue you are facing.

[junglie85]

@jdmaguire thanks, I hadn't spotted the CRD in the deploy folder.

@SrishT I initially attempted to enforce an order but soft dependencies between charts is not supported by Helm:

Soft dependency: A chart may simply not function without another chart being installed in a cluster. Helm does not provide tooling for this case. In this case, dependencies may be managed separately.

There is a post from 2018 on Stackoverflow which suggests a possible workaround - if it works for Helm 3 the charts would need updating to allow custom annotations to be added. I think a CRD's folder would be a neater solution.