Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Replace webhook validations with CEL validation #475

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

erikgb
Copy link
Contributor

@erikgb erikgb commented Nov 16, 2024

I want to see if it's possible to remove the trust-manager validating webhook, as it will eliminate the requirement to pre-install cert-manager. But it doesn't seem possible, at least not with the current Bundle API, to fully replace the webhook validations with CEL api-server validation. We might get a bit further after redesigning the target structure ref. #242, but the label selectors require a webhook to be validated AFAIK.

I personally think the CEL validations are more compact and fairly easy to read, at least the common union validations. The additional formats validation CEL expression is probably a bit over the line, so I am open to reverting it. But I wanted to see how far I could get with CEL, and I even got assistance from Jordan Liggit himself! 🦄

Jordan suggested optional fields to reduce the complexity, but I decided to avoid them for now - since that will require Kubernetes version >= 1.29 to work. We can simplify later on.

Please let me know what you think!

/cc @juliocamarero @SgtCoDFish @inteon

@cert-manager-prow cert-manager-prow bot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. dco-signoff: yes Indicates that all commits in the pull request have the valid DCO sign-off message. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Nov 16, 2024
@erikgb erikgb force-pushed the more-api-validation branch 2 times, most recently from 3749249 to c7c15cb Compare November 16, 2024 10:25
@cert-manager-prow cert-manager-prow bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Nov 16, 2024
@erikgb erikgb force-pushed the more-api-validation branch 5 times, most recently from 78f63d5 to 3741f38 Compare November 16, 2024 13:33
@cert-manager-prow cert-manager-prow bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Nov 16, 2024
@erikgb erikgb force-pushed the more-api-validation branch 3 times, most recently from d9853ff to 17a72ed Compare November 17, 2024 09:39
@cert-manager-prow cert-manager-prow bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Nov 22, 2024
@cert-manager-prow cert-manager-prow bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Nov 22, 2024
@erikgb erikgb changed the title WIP: Make more use of api-server validation Replace webhook validations with CEL validation Nov 22, 2024
@cert-manager-prow cert-manager-prow bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Nov 22, 2024
Copy link
Contributor

@juliocamarero juliocamarero left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I wasn't aware of the CEL validations, but they seem very expressive and easy to use, I like this approach 👍

@cert-manager-prow
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: juliocamarero
Once this PR has been reviewed and has the lgtm label, please ask for approval from erikgb. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Copy link
Member

@SgtCoDFish SgtCoDFish left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seeing the amount of code this removes is super cool 🚀 I've got a few questions to help me understand better!

docs/api/api.md Outdated Show resolved Hide resolved
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question: Did you run these tests against main, before switching to CEL validation? I think that would be valuable if we've not done it yet!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point! I think some of them actually might fail, and that is because the webhook validation code is complex and error-prone. Let me pull the tests into a new PR to see.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just tried this, and it is not working - for "technical reasons". 😢 Our integration tests are poorly configured IMO, and improving this will require some changes that at least @inteon would reject. 😉 The main issue in this case, is that the validating webhook configuration is only available from the Helm chart. And to bootstrap an integration test with webhook enabled, we need the webhook configuration in a format that is machine-readable.

If we adopted more of the "best practices" established by kubebuilder, this would be an improvement - at least to me.

Relates to cert-manager/makefile-modules#209.

@@ -100,6 +102,9 @@ type BundleSource struct {

// BundleTarget is the target resource that the Bundle will sync all source
// data to.
// +kubebuilder:validation:XValidation:rule="[has(self.configMap), has(self.secret)].exists(x,x)", message="must define at least one target configMap/secret"
// +kubebuilder:validation:XValidation:rule="!has(self.additionalFormats) || ![has(self.additionalFormats.jks), has(self.additionalFormats.pkcs12)].all(x,x) || self.additionalFormats.jks.key != self.additionalFormats.pkcs12.key", message="additional format keys must be unique"
// +kubebuilder:validation:XValidation:rule="!has(self.additionalFormats) || !((has(self.configMap) ? [self.configMap.key] : []) + (has(self.secret) ? [self.secret.key] : [])).exists(k,(has(self.additionalFormats.jks) && self.additionalFormats.jks.key == k) || (has(self.additionalFormats.pkcs12) && self.additionalFormats.pkcs12.key == k))", message="additional format keys must be different from configMap/secret keys"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question: Is a +kubebuilder comment the only way of specifying these validation rules? (I'm unfamiliar generally with this CEL validation stuff)

This CEL code is super important to our UX but it's really difficult to read, understand and test it when it's embedded in a comment like this - seems like a sharp edge.

This isn't a blocker or anything, but it seems a shame if this is the only way today!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, it should be possible to hack the expression directly into the CRDs, but that is not something I would do. 😆 The expressions are mainly complex because the API is suboptimal. After #486, I think the expressions could become really simple.

Thanks for reviewing @SgtCoDFish (and @juliocamarero)! But I think the best option here is to put this PR back into draft-mode and wait for a conclusion on #486.

@erikgb erikgb marked this pull request as draft November 25, 2024 16:54
@cert-manager-prow cert-manager-prow bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Nov 25, 2024
@erikgb erikgb changed the title Replace webhook validations with CEL validation WIP: Replace webhook validations with CEL validation Nov 25, 2024
@cert-manager-prow cert-manager-prow bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Nov 27, 2024
@cert-manager-prow cert-manager-prow bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Nov 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dco-signoff: yes Indicates that all commits in the pull request have the valid DCO sign-off message. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants