From 7cf6eedd631911140c1596bc0e353557a18d73d9 Mon Sep 17 00:00:00 2001 From: Alexander Zielenski <351783+alexzielenski@users.noreply.github.com> Date: Mon, 24 Jul 2023 18:28:45 -0700 Subject: [PATCH 1/3] add note about CRDValidationRatcheting feature gate --- .../command-line-tools-reference/feature-gates.md | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates.md b/content/en/docs/reference/command-line-tools-reference/feature-gates.md index 88e2cae5d5..18a74a2481 100644 --- a/content/en/docs/reference/command-line-tools-reference/feature-gates.md +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates.md @@ -89,6 +89,7 @@ For a reference to old feature gates that are removed, please refer to | `ComponentSLIs` | `true` | Beta | 1.27 | | | `ContainerCheckpoint` | `false` | Alpha | 1.25 | | | `ContextualLogging` | `false` | Alpha | 1.24 | | +| CRDValidationRatcheting | false | Alpha | 1.28 | | `CrossNamespaceVolumeDataSource` | `false` | Alpha| 1.26 | | | `CustomCPUCFSQuotaPeriod` | `false` | Alpha | 1.12 | | | `CustomResourceValidationExpressions` | `false` | Alpha | 1.23 | 1.24 | @@ -457,6 +458,18 @@ Each feature gate is designed for enabling/disabling a specific feature: - `ContextualLogging`: When you enable this feature gate, Kubernetes components that support contextual logging add extra detail to log output. - `CronJobTimeZone`: Allow the use of the `timeZone` optional field in [CronJobs](/docs/concepts/workloads/controllers/cron-jobs/) +- `CRDValidationRatcheting`: Enable updates to custom resources to contain + violations of their OpenAPI schema if the offending portions of the resource + update did not change. Allows users to update to stricter versions of a CRD + schema without bumping the version of the object or breaking workflows. + While most are supported, does not support ratcheting updates across all + schema changes. The following OpenAPIV3 schema validations are not supported + by ratcheting under the current implementation and if violated will continue + to throw an error as normally: + - `allOf`/`oneOf`/`anyOf`/`not` nested validations + - `x-kubernetes-validations` + - `x-kubernetes-list-type` + - `required` fields - `CrossNamespaceVolumeDataSource`: Enable the usage of cross namespace volume data source to allow you to specify a source namespace in the `dataSourceRef` field of a PersistentVolumeClaim. From e894028da7b101c3e4cef0ebafd8ff2dabddce77 Mon Sep 17 00:00:00 2001 From: Alexander Zielenski <351783+alexzielenski@users.noreply.github.com> Date: Mon, 7 Aug 2023 22:10:04 -0700 Subject: [PATCH 2/3] move to other pages, add links --- .../api-extension/custom-resources.md | 2 +- .../feature-gates.md | 11 +---- .../custom-resource-definitions.md | 44 +++++++++++++++++++ 3 files changed, 46 insertions(+), 11 deletions(-) diff --git a/content/en/docs/concepts/extend-kubernetes/api-extension/custom-resources.md b/content/en/docs/concepts/extend-kubernetes/api-extension/custom-resources.md index 2907d2f0fd..aafb04fd83 100644 --- a/content/en/docs/concepts/extend-kubernetes/api-extension/custom-resources.md +++ b/content/en/docs/concepts/extend-kubernetes/api-extension/custom-resources.md @@ -209,7 +209,7 @@ Aggregated APIs offer more advanced API features and customization of other feat | Feature | Description | CRDs | Aggregated API | | ------- | ----------- | ---- | -------------- | -| Validation | Help users prevent errors and allow you to evolve your API independently of your clients. These features are most useful when there are many clients who can't all update at the same time. | Yes. Most validation can be specified in the CRD using [OpenAPI v3.0 validation](/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/#validation). Any other validations supported by addition of a [Validating Webhook](/docs/reference/access-authn-authz/admission-controllers/#validatingadmissionwebhook-alpha-in-1-8-beta-in-1-9). | Yes, arbitrary validation checks | +| Validation | Help users prevent errors and allow you to evolve your API independently of your clients. These features are most useful when there are many clients who can't all update at the same time. | Yes. Most validation can be specified in the CRD using [OpenAPI v3.0 validation](/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/#validation). [CRDValidationRatcheting](/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/#validation-ratcheting) feature gate allows failing validations specified using OpenAPI also can be ignored if the failing part of the resource was unchanged. Any other validations supported by addition of a [Validating Webhook](/docs/reference/access-authn-authz/admission-controllers/#validatingadmissionwebhook-alpha-in-1-8-beta-in-1-9). | Yes, arbitrary validation checks | | Defaulting | See above | Yes, either via [OpenAPI v3.0 validation](/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/#defaulting) `default` keyword (GA in 1.17), or via a [Mutating Webhook](/docs/reference/access-authn-authz/admission-controllers/#mutatingadmissionwebhook) (though this will not be run when reading from etcd for old objects). | Yes | | Multi-versioning | Allows serving the same object through two API versions. Can help ease API changes like renaming fields. Less important if you control your client versions. | [Yes](/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definition-versioning) | Yes | | Custom Storage | If you need storage with a different performance mode (for example, a time-series database instead of key-value store) or isolation for security (for example, encryption of sensitive information, etc.) | No | Yes | diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates.md b/content/en/docs/reference/command-line-tools-reference/feature-gates.md index 18a74a2481..9734760236 100644 --- a/content/en/docs/reference/command-line-tools-reference/feature-gates.md +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates.md @@ -460,16 +460,7 @@ Each feature gate is designed for enabling/disabling a specific feature: - `CronJobTimeZone`: Allow the use of the `timeZone` optional field in [CronJobs](/docs/concepts/workloads/controllers/cron-jobs/) - `CRDValidationRatcheting`: Enable updates to custom resources to contain violations of their OpenAPI schema if the offending portions of the resource - update did not change. Allows users to update to stricter versions of a CRD - schema without bumping the version of the object or breaking workflows. - While most are supported, does not support ratcheting updates across all - schema changes. The following OpenAPIV3 schema validations are not supported - by ratcheting under the current implementation and if violated will continue - to throw an error as normally: - - `allOf`/`oneOf`/`anyOf`/`not` nested validations - - `x-kubernetes-validations` - - `x-kubernetes-list-type` - - `required` fields + update did not change. See [Validation Ratcheting](/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/#validation-ratcheting) for more details. - `CrossNamespaceVolumeDataSource`: Enable the usage of cross namespace volume data source to allow you to specify a source namespace in the `dataSourceRef` field of a PersistentVolumeClaim. diff --git a/content/en/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions.md b/content/en/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions.md index edade8f825..b3278af65c 100644 --- a/content/en/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions.md +++ b/content/en/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions.md @@ -717,6 +717,50 @@ And create it: kubectl apply -f my-crontab.yaml crontab "my-new-cron-object" created ``` +### Validation Ratcheting + +Validation Ratcheting is alpha since 1.28. Enable the `CRDValidationRatcheting` +[feature gate](/docs/reference/command-line-tools-reference/feature-gates/) to +use the feature. + +Validation Ratcheting refers to the ability of the apiserver to accept updates +to resources which fail validation, if the failing part of the resource was +unchanged by the update operation. + +This feature allows authors of CRDs to confidently add new validations to the +OpenAPIV3 schema under certain conditions. Users can update to the new schema +safely without bumping the version of the object or breaking workflows. + +While most validations placed in the OpenAPIV3 schema of a CRD are support +ratcheting, there are a few exceptions. The following OpenAPIV3 schema +validations are not supported by ratcheting under the current implementation +and if violated will continue to throw an error as normally: + +- Quantors + - `allOf` + - `oneOf` + - `anyOf` + - `not` + - any validations in a descendent of one of these fields +- `x-kubernetes-validations` + CRD Validation Rules are currently ignored by ratcheting. This may be subject + to change. +- `x-kubernetes-list-type` + Errors arising from changing the list type of a subschema will not be + ratcheted. For example adding `set` onto a list with duplicates will always + result in an error. +- x-kubernetes-map-keys + Errors arising from changing the map keys of a list schema will not be + ratcheted. +- `required` + Required fields may not be safely added +- `properties` + Properties may not be safely removed, but changes to validations in their + schemas and subschemas may be ratcheted +- `additionalProperties` + To remove a previously specified `additionalProperties` validation will not be + ratcheted. + ## Validation rules From 75ec81802226dc4a8300146f22f11e6e0230b933 Mon Sep 17 00:00:00 2001 From: Alexander Zielenski <351783+alexzielenski@users.noreply.github.com> Date: Tue, 8 Aug 2023 08:53:42 -0700 Subject: [PATCH 3/3] feedback --- .../custom-resource-definitions.md | 35 +++++++++++-------- 1 file changed, 21 insertions(+), 14 deletions(-) diff --git a/content/en/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions.md b/content/en/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions.md index b3278af65c..9ca480618b 100644 --- a/content/en/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions.md +++ b/content/en/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions.md @@ -717,15 +717,21 @@ And create it: kubectl apply -f my-crontab.yaml crontab "my-new-cron-object" created ``` -### Validation Ratcheting +### Validation ratcheting -Validation Ratcheting is alpha since 1.28. Enable the `CRDValidationRatcheting` +{{< feature-state state="alpha" for_k8s_version="v1.28" >}} + +You need to enable the `CRDValidationRatcheting` [feature gate](/docs/reference/command-line-tools-reference/feature-gates/) to -use the feature. +use this behavior, which then applies to all CustomResourceDefinitions in your +cluster. -Validation Ratcheting refers to the ability of the apiserver to accept updates -to resources which fail validation, if the failing part of the resource was -unchanged by the update operation. +Provided you enabled the feature gate, Kubernetes implements _validation racheting_ +for CustomResourceDefinitions. The API server is willing accept updates to resources that +are not valid after the update, provided that each part of the resource that failed to validate +was not changed by the update operation. In other words, any invalid part of the resource +that remains invalid must have already been wrong. +You cannot use this mechanism to update a valid resource so that it becomes invalid. This feature allows authors of CRDs to confidently add new validations to the OpenAPIV3 schema under certain conditions. Users can update to the new schema @@ -733,8 +739,8 @@ safely without bumping the version of the object or breaking workflows. While most validations placed in the OpenAPIV3 schema of a CRD are support ratcheting, there are a few exceptions. The following OpenAPIV3 schema -validations are not supported by ratcheting under the current implementation -and if violated will continue to throw an error as normally: +validations are not supported by ratcheting under the implementation in Kubernetes +{{< skew currentVersion >}} and if violated will continue to throw an error as normally: - Quantors - `allOf` @@ -743,20 +749,21 @@ and if violated will continue to throw an error as normally: - `not` - any validations in a descendent of one of these fields - `x-kubernetes-validations` - CRD Validation Rules are currently ignored by ratcheting. This may be subject - to change. + For Kubernetes {{< skew currentVersion >}}, CRD validation rules](#validation-rules) are ignored by + ratcheting. This may change in later Kubernetes releases. - `x-kubernetes-list-type` Errors arising from changing the list type of a subschema will not be ratcheted. For example adding `set` onto a list with duplicates will always result in an error. -- x-kubernetes-map-keys +- `x-kubernetes-map-keys` Errors arising from changing the map keys of a list schema will not be ratcheted. - `required` - Required fields may not be safely added + Errors arising from changing the list of required fields will not be ratcheted. - `properties` - Properties may not be safely removed, but changes to validations in their - schemas and subschemas may be ratcheted + Adding/removing/modifying the names of properties is not ratcheted, but + changes to validations in each properties' schemas and subschemas may be ratcheted + if the name of the property stays the same. - `additionalProperties` To remove a previously specified `additionalProperties` validation will not be ratcheted.