-
Notifications
You must be signed in to change notification settings - Fork 180
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
feat: hip-0019 split manifest proposal
Signed-off-by: Louis Cabrol <[email protected]>
- Loading branch information
Showing
1 changed file
with
65 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,65 @@ | ||
| --- | ||
| hip: "0019" | ||
| title: "Secrets storage driver support for split manifests" | ||
| authors: [ "Louis Cabrol" ] | ||
| created: "2023-02-06" | ||
| type: "feature" | ||
| status: "draft" | ||
| --- | ||
|
|
||
| ## Abstract | ||
|
|
||
| Helm, by default in version 3, stores its Release manifest in a Secret object in the target Kubernetes namespace, after a double base64 encoding + GZIP compression. | ||
|
|
||
| This object type, [by Kubernetes design](https://kubernetes.io/docs/concepts/configuration/secret/#restriction-data-size) has a maximum size allowed for data restricted to 1Mib. | ||
|
|
||
| This imposes a limit on the maximum number of templates a Helm Release can manage, as well as limits the overall complexity a Helm Chart can reach before exceeding this limit. | ||
|
|
||
| ## Motivation | ||
|
|
||
| This has been reported a number of times ([here](https://github.com/helm/helm/issues/10986), [here](https://github.com/helm/helm/issues/8281), and many others within those two links). As Charts keep growing in complexity through modular, optional components, and Releases keep growing in the number of Kubernetes objects they manage, more and more people will keep hitting this limit. | ||
|
|
||
| Deploying a Chart that exceeds this preset size currently yields a Kubernetes error, without first warning the user that the Release Manifest size exceeds the maximum, which isn't a good user experience. | ||
|
|
||
| My specific motivation for this fix is a complex, modular Helm Chart that deploys Grafana dashboards. As such, it contains (many) dashboard templates as JSON that are templated through Helm into ConfigMap objects. Those template files cannot be `helmignored`, and their cumulated size alone is close to the Secret size limit, before encoding and compression. As such, I'm starting to run into issues when deploying to complex environments that require a lot of those dashboard definitions, which could be solved by splitting the chart among different sub-charts and/or sub-releases, but this would be working around the problem, and not adressing it. | ||
|
|
||
| ## Specification | ||
|
|
||
| The proposal is to create a new "type" of Secret, `helm.sh/partial.v1`, that would enable the Release Manifest to be split across multiple secrets as needed, named after the following format `sh.helm.partial.v1.<release name>.v<release version>-<partial chunk number>`. The presence of further "partial" secrets would then be indicated through an optional `continuedIn` label. Changes should only be made within the Secrets storage driver, so as to retain the current driver interface. | ||
|
|
||
| ## Backward compatibility | ||
|
|
||
| Containing changes to the Secrets storage driver means there's no compatibility issue with other storage types. | ||
|
|
||
| For Secrets, marking the Secrets with `helm.sh/partial.v1` for partial elements means identifying releases still only relies on the presence of secrets that are named after this format `sh.helm.release.v1.<release name>.v<version>` with the `helm.sh/release.v1` type. | ||
|
|
||
| Older versions of Helm would not be able to interact with the new split manifests in most cases. | ||
|
|
||
| ## Rationale | ||
|
|
||
| The only storage type that contains this hardcoded limit is Secrets. ConfigMap size can be increased at the `etcd` level, and the SQL driver doesn't have such limitations by design. | ||
|
|
||
| This expansion to the Secrets spec allows users not to maintain a separate PSQL database to store Helm release manifests, instead allowing them to make use of the internal components Kubernetes provides, while maintaining backwards compatibility with current releases. | ||
|
|
||
| ## How to teach this | ||
|
|
||
| This size limit currently isn't well documented within Helm, and only found through the aforementioned issues I linked to, so there isn't much to teach. There would not be any user-facing changes, barring the ability to deploy larger charts. | ||
|
|
||
| ## Reference implementation | ||
|
|
||
| [Helm PR](https://github.com/helm/helm/pull/11791) | ||
|
|
||
| ## Rejected ideas | ||
|
|
||
| None as far as I know | ||
|
|
||
| ## Open issues | ||
|
|
||
| None as far as I know | ||
|
|
||
| ## References | ||
|
|
||
| - [Kubernetes Secrets data size restrition](https://kubernetes.io/docs/concepts/configuration/secret/#restriction-data-size) | ||
| - Issues: | ||
| - [An issue discussing this problem](https://github.com/helm/helm/issues/10986) | ||
| - [Another issue discussing this problem](https://github.com/helm/helm/issues/8281) |