Implement cel validation `derive(Validated)` macro for generated CRDs #1649

Danil-Grigorev · 2024-11-24T17:48:32Z

Motivation

Related to #1367

CRDs allow to declare server-side validation rules using CEL. This functionality is supported via #[schemars(schema_with = "<schemagen-wrapper>")], but requires defining a method with handling logic, which may be error-prone.

Since kube owns CRD generation code, the idea is to simplify this process for added validation rules and achieve more declarative approach, similar to the kubebuilder library. This approach will be compatible with kopium generation based on existing CRD structures, already using CEL expressions.

Solution

Allow for a more native handling of CEL validation rules on the CRDs via a field macro.

Current approach only generate methods for the set of provided rules. User still have to specify schemars(schema_with attribute manually. Serde default also works as usual.

#[derive(…, CELValidate, JsonSchema)]
#[kube(
…
)]
pub struct FooSpec {
    // Field with CEL validation
    #[serde(default)]
    #[cel_validate(
         method = cel_validated,
         rule = Rule{rule: "self != 'illegal'".into(), message: Some(Message::Expression("'string cannot be illegal'".into())), reason: Some(Reason::FieldValueForbidden), ..Default::default()},
         rule = Rule{rule: "self != 'not legal'".into(), reason: Some(Reason::FieldValueInvalid), ..Default::default()}
     )]
     #[schemars(schema_with = "cel_validated")]
    cel_validated: Option<String>,

A simple version involving cel_validate proc macro is also available:

#[cel_validate]
#[derive(…, JsonSchema)]
#[kube(
…
)]
pub struct FooSpec {
    // Field with CEL validation
    #[serde(default)]
    #[cel_validate(
         method = cel_validated,
         rule = Rule{rule: "self != 'illegal'".into(), message: Some(Message::Expression("'string cannot be illegal'".into())), reason: Some(Reason::FieldValueForbidden), ..Default::default()},
         rule = Rule{rule: "self != 'not legal'".into(), reason: Some(Reason::FieldValueInvalid), ..Default::default()}
     )]
    cel_validated: Option<String>,

This PR is a followup on #1621 which addresses some of the concerns.

No rewrite of the structure. Method is generated based on user input or automatically from the name of the field.
Based on Implement CEL validation proc macro for generated CRDs #1621 (comment) - using methods from the kube::core and invoking from kube::derive.
Generally simplified the code to make it easier to maintain (Implement CEL validation proc macro for generated CRDs #1621 (comment))

Other things tried (TLDR)

Visitor trait:

It is possible to generate a new Visitor implementation per each validation rule. But the problem with this approach is that the generation happens for Validator derive on the structure, while the CustomResource derive is responsible for populating additional visitors for crd(). There is no one specific method which can collect all visitors under one chain, invoked from schemars. This likely requires every individual field in each struct to implement the Validated trait, involving creation of a shemars/serde type of logic.

Then the schemars Schema has no indicators for the source structure in the schema within Visitor, so there is no way (without generating schemars(title = “FooSpec”) as a metadata) to match the added visitor on the processed object param to make modifications. It is possible to add preserve_order feature to schemars and “search” for the property of the structure, as long as the source struct name is mapped to the Schema content.

Generating schemars attributes

While it is a viable option, such thing is not possible with derive macro, and has to use proc_macro instead, This approach is additionally hiding the updates of derive attributes under the hood, which feels unintuitive, as it performs updates to the macro markers, meant to generate code. Explored in #1621

Manually generating a `JsonSchema` implementation.

This approach is the most powerful, but ends up being really hacky, involving copying the structure in a separate mod and using derived JsonSchema as a “starting template”.

codecov · 2024-11-24T18:08:10Z

Codecov Report

Attention: Patch coverage is 20.00000% with 36 lines in your changes missing coverage. Please review.

Project coverage is 75.2%. Comparing base (3ee4ae5) to head (ee96ec4).

Files with missing lines	Patch %	Lines
kube-derive/src/custom_resource.rs	28.2%	23 Missing ⚠️
kube-core/src/validation.rs	0.0%	11 Missing ⚠️
kube-derive/src/lib.rs	0.0%	2 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##            main   #1649     +/-   ##
=======================================
- Coverage   75.6%   75.2%   -0.3%     
=======================================
  Files         82      83      +1     
  Lines       7405    7450     +45     
=======================================
+ Hits        5591    5600      +9     
- Misses      1814    1850     +36

Files with missing lines	Coverage Δ
kube/src/lib.rs	`88.5% <ø> (ø)`
kube-derive/src/lib.rs	`0.0% <0.0%> (ø)`
kube-core/src/validation.rs	`0.0% <0.0%> (ø)`
kube-derive/src/custom_resource.rs	`75.2% <28.2%> (-7.3%)`	⬇️

---- 🚨 Try these New Features:

Flaky Tests Detection - Detect and resolve failed and flaky tests

- Extend with supported values from docs - https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/#validation-rules - Implement as Validated derive macro - Use the raw Rule for the validated attribute Signed-off-by: Danil-Grigorev <[email protected]>

Danil-Grigorev · 2024-11-24T23:17:24Z

kube-derive/src/custom_resource.rs

+struct CELValidation {
+    #[darling(default)]
+    crates: Crates,
+    data: ast::Data<util::Ignored, Rule>,


Note: may also be useful to provide naming template for common suffix/prefix per each attribute, in case when method is not specified. By default generated method inherits the name of the field for simplicity.

clux

some comments and questions. i think this is a pretty cool approach.

clux · 2024-11-27T07:38:06Z

kube-core/src/validation.rs

+
+/// Reason is a machine-readable value providing more detail about why a field failed the validation.
+///
+/// More in [docs](https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/#field-reason)


Interestingly, I see these ones in the generated docs under https://github.com/kube-rs/k8s-pb/blob/ce4261fb52266f05cd7a06dbb8f4c0fcaa41c06a/k8s-pb/src/apiextensions_apiserver/pkg/apis/apiextensions/v1/mod.rs#L736 but because of bad go enum usage it's just a doc comment :(

Would not pretend I saw it being generated, but yeah, missing enum is better to have :). Maybe worth adding From/Into conversion for ensuring compatibility.

clux · 2024-11-27T07:38:33Z

kube-core/src/validation.rs

@@ -0,0 +1,118 @@
+//! CEL validation for CRDs


should probably call the module cel.rs

clux · 2024-11-27T07:39:50Z

kube-core/src/validation.rs

+    /// Example:
+    /// "must be a URL with the host matching spec.host"
+    Message(String),
+    /// Expression declares a CEL expression that evaluates to the validation failure message that is returned when this rule fails.


are these doc strings taken from anywhere?

clux · 2024-11-27T07:43:50Z

kube/src/lib.rs

@@ -168,6 +168,10 @@ pub use kube_derive::CustomResource;
 #[cfg_attr(docsrs, doc(cfg(feature = "derive")))]
 pub use kube_derive::Resource;

+#[cfg(feature = "derive")]
+#[cfg_attr(docsrs, doc(cfg(feature = "derive")))]
+pub use kube_derive::Validated;


naming wise, possibly this introduces an confusion possiblities with garde::Validate - which we do advocate for

clux · 2024-11-27T07:47:12Z

kube-derive/src/lib.rs

+/// #[kube(group = "kube.rs", version = "v1", kind = "Struct")]
+/// struct MyStruct {
+///     #[serde(default = "default")]
+///     #[validated(rule = Rule{rule: "self != ''".into(), message: Some("failure message".into()), ..Default::default()})]


pretty cool that you can do a full struct here like this. does it support shorthands as well if we were to create builders on Rule? could lead to:

#[validated(rule = Rule::rule("self != ''").message("failure message")]

possibly

it is possible, including something simple such as #[validated(rule = “self != ‘’”.into())].

clux · 2024-11-27T07:48:23Z

kube-core/src/validation.rs

+/// Rule is a CEL validation rule for the CRD field
+#[derive(Default, Serialize, Deserialize, Clone)]
+#[serde(rename_all = "camelCase")]
+pub struct Rule {


note to self; this may perhaps be churned through a cel crate against an object to perform validation client side. should investigate this later.

Signed-off-by: Danil-Grigorev <[email protected]>

Danil-Grigorev force-pushed the cel-expression-on-crds branch 2 times, most recently from 8a8cef3 to f286169 Compare November 24, 2024 17:58

Danil-Grigorev force-pushed the cel-expression-on-crds branch 7 times, most recently from 047b626 to 06ea3e9 Compare November 24, 2024 20:51

Danil-Grigorev force-pushed the cel-expression-on-crds branch from 06ea3e9 to ee96ec4 Compare November 24, 2024 20:58

Danil-Grigorev changed the title ~~[WIP] Implement cel validation proc macro for generated CRDs~~ Implement cel validation proc macro for generated CRDs Nov 24, 2024

Danil-Grigorev changed the title ~~Implement cel validation proc macro for generated CRDs~~ Implement cel validation derive(Validated) macro for generated CRDs Nov 24, 2024

Danil-Grigorev changed the title ~~Implement cel validation derive(Validated) macro for generated CRDs~~ Implement cel validation derive(Validated) macro for generated CRDs Nov 24, 2024

Danil-Grigorev commented Nov 24, 2024

View reviewed changes

clux reviewed Nov 27, 2024

View reviewed changes

Add cel_validate proc macro for completion, rename

abe840c

Signed-off-by: Danil-Grigorev <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement cel validation `derive(Validated)` macro for generated CRDs #1649

Implement cel validation `derive(Validated)` macro for generated CRDs #1649

Danil-Grigorev commented Nov 24, 2024 •

edited

Loading

codecov bot commented Nov 24, 2024 •

edited

Loading

Danil-Grigorev Nov 24, 2024

clux left a comment

clux Nov 27, 2024

Danil-Grigorev Nov 27, 2024

clux Nov 27, 2024

clux Nov 27, 2024

clux Nov 27, 2024

clux Nov 27, 2024

Danil-Grigorev Nov 27, 2024

clux Nov 27, 2024

Implement cel validation derive(Validated) macro for generated CRDs #1649

Are you sure you want to change the base?

Implement cel validation derive(Validated) macro for generated CRDs #1649

Conversation

Danil-Grigorev commented Nov 24, 2024 • edited Loading

Motivation

Solution

Other things tried (TLDR)

Visitor trait:

Generating schemars attributes

Manually generating a JsonSchema implementation.

codecov bot commented Nov 24, 2024 • edited Loading

Codecov Report

Choose a reason for hiding this comment

clux left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Implement cel validation `derive(Validated)` macro for generated CRDs #1649

Implement cel validation `derive(Validated)` macro for generated CRDs #1649

Danil-Grigorev commented Nov 24, 2024 •

edited

Loading

Manually generating a `JsonSchema` implementation.

codecov bot commented Nov 24, 2024 •

edited

Loading