Implement derive(CELSchema) macro for generating cel validation on CRDs

[Danil-Grigorev]

Motivation

Related to #1367

CRDs allow to declare server-side validation rules using CEL. This functionality is supported via #[schemars(schema_with = "<schemagen-wrapper>")], but requires defining a method with handling logic, which may be error-prone.

Since kube owns CRD generation code, the idea is to simplify this process for added validation rules and achieve more declarative approach, similar to the kubebuilder library. This approach will be compatible with kopium generation based on existing CRD structures, already using CEL expressions.

Solution

Allow for a more native handling of CEL validation rules on the CRDs via a field macro.

#[derive(…, CELSchema)]
#[kube(
  rule = Rule::new("self.metadata.name == 'singleton'"),
…
)]
pub struct FooSpec {
    // Field with CEL validation
    #[serde(default)]
    #[cel_validate(
         rule = Rule::new("self != 'illegal'").message("string cannot be illegal").reason(Reason::FieldValueForbidden),
         rule = Rule::new("self != 'not legal'").reason(Reason::FieldValueInvalid),
     )]
    cel_validated: Option<String>,

This PR is a followup on #1621 which addresses some of the concerns.

1. Implemented by generating a JsonSchema, which allows further additions to the schema later, struct level validation rules and is not affected by schemars version.
2. Based on Implement CEL validation proc macro for generated CRDs #1621 (comment) - using methods from the kube::core and invoking from kube::derive.
3. Generally simplified the code to make it easier to maintain (Implement CEL validation proc macro for generated CRDs #1621 (comment))

Other things tried (TLDR)

Visitor trait:

It is possible to generate a new Visitor implementation per each validation rule. But the problem with this approach is that the generation happens for Validator derive on the structure, while the CustomResource derive is responsible for populating additional visitors for crd(). There is no one specific method which can collect all visitors under one chain, invoked from schemars. This likely requires every individual field in each struct to implement the Validated trait, involving creation of a shemars/serde type of logic.

Then the schemars Schema has no indicators for the source structure in the schema within Visitor, so there is no way (without generating schemars(title = “FooSpec”) as a metadata) to match the added visitor on the processed object param to make modifications. It is possible to add preserve_order feature to schemars and “search” for the property of the structure, as long as the source struct name is mapped to the Schema content.

With generating JsonSchema visitor extensions for more complex scenarios are possible to explore in the future.

Generating schemars attributes

While it is a viable option, such thing is not possible with derive macro, and has to use proc_macro instead, This approach is additionally hiding the updates of derive attributes under the hood, which feels unintuitive, as it performs updates to the macro markers, meant to generate code. Explored in #1621

[codecov]

Codecov Report

Attention: Patch coverage is 78.64078% with 22 lines in your changes missing coverage. Please review.

Project coverage is 75.9%. Comparing base (0424cb4) to head (060ad64).
Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
kube-core/src/cel.rs	56.8%	16 Missing ⚠️
kube-derive/src/cel_schema.rs	92.6%	4 Missing ⚠️
kube-derive/src/lib.rs	0.0%	2 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##            main   #1649     +/-   ##
=======================================
+ Coverage   75.8%   75.9%   +0.1%     
=======================================
  Files         82      84      +2     
  Lines       7513    7612     +99     
=======================================
+ Hits        5693    5771     +78     
- Misses      1820    1841     +21     

Files with missing lines	Coverage Δ
kube-derive/src/custom_resource.rs	83.5% <100.0%> (+1.0%)	⬆️
kube-derive/tests/crd_schema_test.rs	96.9% <ø> (ø)
kube/src/lib.rs	88.5% <ø> (ø)
kube-derive/src/lib.rs	0.0% <0.0%> (ø)
kube-derive/src/cel_schema.rs	92.6% <92.6%> (ø)
kube-core/src/cel.rs	56.8% <56.8%> (ø)

[clux]

some comments and questions. i think this is a pretty cool approach.

[clux]

Interestingly, I see these ones in the generated docs under https://github.com/kube-rs/k8s-pb/blob/ce4261fb52266f05cd7a06dbb8f4c0fcaa41c06a/k8s-pb/src/apiextensions_apiserver/pkg/apis/apiextensions/v1/mod.rs#L736 but because of bad go enum usage it's just a doc comment :(

[Danil-Grigorev]

Would not pretend I saw it being generated, but yeah, missing enum is better to have :). Maybe worth adding From/Into conversion for ensuring compatibility.

[clux]

I think this approach makes sense and is a nice way to opt into injecting validations. Have added some comments for code organisation (file is getting big) and for naming, but ultimately am happy with this!

[clux]

Thanks a lot for this! It's a lot more readable and understandable now as it's factored out. Very minor set of comments, and a few questions for my own sanity. Only want one doc field added.

[clux]

Thanks a lot for the long exploration here. This is a very good checkpoint for CEL support. Closing the original issue with this PR and writing one follow-up. (Obviously,fFeel free to add follow-ups as you see fit as well.)