--- title: Extend the Kubernetes API with CustomResourceDefinitions approvers: - deads2k - enisoc --- {% capture overview %} This page shows how to install a [custom resource](/docs/concepts/api-extension/custom-resources/) into the Kubernetes API by creating a [CustomResourceDefinition](/docs/api-reference/{{page.version}}/#customresourcedefinition-v1beta1-apiextensions). {% endcapture %} {% capture prerequisites %} {% include task-tutorial-prereqs.md %} * Make sure your Kubernetes cluster has a master version of 1.7.0 or higher. * Read about [custom resources](/docs/concepts/api-extension/custom-resources/). {% endcapture %} {% capture steps %} ## Create a CustomResourceDefinition When you create a new CustomResourceDefinition (CRD), the Kubernetes API Server reacts by creating a new RESTful resource path, either namespaced or cluster-scoped, as specified in the CRD's `scope` field. As with existing built-in objects, deleting a namespace deletes all custom objects in that namespace. CustomResourceDefinitions themselves are non-namespaced and are available to all namespaces. For example, if you save the following CustomResourceDefinition to `resourcedefinition.yaml`: ```yaml apiVersion: apiextensions.k8s.io/v1beta1 kind: CustomResourceDefinition metadata: # name must match the spec fields below, and be in the form: . name: crontabs.stable.example.com spec: # group name to use for REST API: /apis// group: stable.example.com # version name to use for REST API: /apis// version: v1 # either Namespaced or Cluster scope: Namespaced names: # plural name to be used in the URL: /apis/// plural: crontabs # singular name to be used as an alias on the CLI and for display singular: crontab # kind is normally the CamelCased singular type. Your resource manifests use this. kind: CronTab # shortNames allow shorter string to match your resource on the CLI shortNames: - ct ``` And create it: ```shell kubectl create -f resourcedefinition.yaml ``` Then a new namespaced RESTful API endpoint is created at: ``` /apis/stable.example.com/v1/namespaces/*/crontabs/... ``` This endpoint URL can then be used to create and manage custom objects. The `kind` of these objects will be `CronTab` from the spec of the CustomResourceDefinition object you created above. ## Create custom objects After the CustomResourceDefinition object has been created, you can create custom objects. Custom objects can contain custom fields. These fields can contain arbitrary JSON. In the following example, the `cronSpec` and `image` custom fields are set in a custom object of kind `CronTab`. The kind `CronTab` comes from the spec of the CustomResourceDefinition object you created above. If you save the following YAML to `my-crontab.yaml`: ```yaml apiVersion: "stable.example.com/v1" kind: CronTab metadata: name: my-new-cron-object spec: cronSpec: "* * * * */5" image: my-awesome-cron-image ``` and create it: ```shell kubectl create -f my-crontab.yaml ``` You can then manage your CronTab objects using kubectl. For example: ```shell kubectl get crontab ``` Should print a list like this: ```console NAME KIND my-new-cron-object CronTab.v1.stable.example.com ``` Note that resource names are not case-sensitive when using kubectl, and you can use either the singular or plural forms defined in the CRD, as well as any short names. You can also view the raw YAML data: ```shell kubectl get ct -o yaml ``` You should see that it contains the custom `cronSpec` and `image` fields from the yaml you used to create it: ```console apiVersion: v1 items: - apiVersion: stable.example.com/v1 kind: CronTab metadata: clusterName: "" creationTimestamp: 2017-05-31T12:56:35Z deletionGracePeriodSeconds: null deletionTimestamp: null name: my-new-cron-object namespace: default resourceVersion: "285" selfLink: /apis/stable.example.com/v1/namespaces/default/crontabs/my-new-cron-object uid: 9423255b-4600-11e7-af6a-28d2447dc82b spec: cronSpec: '* * * * */5' image: my-awesome-cron-image kind: List metadata: resourceVersion: "" selfLink: "" ``` ## Delete a CustomResourceDefinition When you delete a CustomResourceDefinition, the server will uninstall the RESTful API endpoint and **delete all custom objects stored in it**. ```shell kubectl delete -f resourcedefinition.yaml kubectl get crontabs ``` ```console Error from server (NotFound): Unable to list "crontabs": the server could not find the requested resource (get crontabs.stable.example.com) ``` If you later recreate the same CustomResourceDefinition, it will start out empty. {% endcapture %} {% capture discussion %} ## Advanced topics ### Finalizers *Finalizers* allow controllers to implement asynchronous pre-delete hooks. Custom objects support finalizers just like built-in objects. You can add a finalizer to a custom object like this: ```yaml apiVersion: "stable.example.com/v1" kind: CronTab metadata: finalizers: - finalizer.stable.example.com ``` The first delete request on an object with finalizers merely sets a value for the `metadata.deletionTimestamp` field instead of deleting it. This triggers controllers watching the object to execute any finalizers they handle. Each controller then removes its finalizer from the list and issues the delete request again. This request only deletes the object if the list of finalizers is now empty, meaning all finalizers are done. ### Validation Validation of custom objects is possible via [OpenAPI v3 schema](https://github.com/OAI/OpenAPI-Specification/blob/master/versions/3.0.0.md#schemaObject). Additionally, the following restrictions are applied to the schema: - The fields `default`, `nullable`, `discriminator`, `readOnly`, `writeOnly`, `xml` and `deprecated` cannot be set. - The field `uniqueItems` cannot be set to true. - The field `additionalProperties` cannot be set to false. This feature is __alpha__ in v1.8 and may change in backward incompatible ways. Enable this feature using the `CustomResourceValidation` feature gate on the [kube-apiserver](/docs/admin/kube-apiserver): ``` --feature-gates=CustomResourceValidation=true ``` The schema is defined in the CustomResourceDefinition. In the following example, the CustomResourceDefinition applies the following validations on the custom object: - `spec.cronSpec` must be a string and must be of the form described by the regular expression. - `spec.replicas` must be an integer and must have a minimum value of 1 and a maximum value of 10. Save the CustomResourceDefinition to `resourcedefinition.yaml`: ```yaml apiVersion: apiextensions.k8s.io/v1beta1 kind: CustomResourceDefinition metadata: name: crontabs.stable.example.com spec: group: stable.example.com version: v1 scope: Namespaced names: plural: crontabs singular: crontab kind: CronTab shortNames: - ct validation: # openAPIV3Schema is the schema for validating custom objects. openAPIV3Schema: properties: spec: properties: cronSpec: type: string pattern: '^(\d+|\*)(/\d+)?(\s+(\d+|\*)(/\d+)?){4}$' replicas: type: integer minimum: 1 maximum: 10 ``` And create it: ```shell kubectl create -f resourcedefinition.yaml ``` A request to create a custom object of kind `CronTab` will be rejected if there are invalid values in its fields. In the following example, the custom object contains fields with invalid values: - `spec.cronSpec` does not match the regular expression. - `spec.replicas` is greater than 10. If you save the following YAML to `my-crontab.yaml`: ```yaml apiVersion: "stable.example.com/v1" kind: CronTab metadata: name: my-new-cron-object spec: cronSpec: "* * * *" image: my-awesome-cron-image replicas: 15 ``` and create it: ```shell kubectl create -f my-crontab.yaml ``` you will get an error: ```console The CronTab "my-new-cron-object" is invalid: []: Invalid value: map[string]interface {}{"apiVersion":"stable.example.com/v1", "kind":"CronTab", "metadata":map[string]interface {}{"name":"my-new-cron-object", "namespace":"default", "deletionTimestamp":interface {}(nil), "deletionGracePeriodSeconds":(*int64)(nil), "creationTimestamp":"2017-09-05T05:20:07Z", "uid":"e14d79e7-91f9-11e7-a598-f0761cb232d1", "selfLink":"", "clusterName":""}, "spec":map[string]interface {}{"cronSpec":"* * * *", "image":"my-awesome-cron-image", "replicas":15}}: validation failure list: spec.cronSpec in body should match '^(\d+|\*)(/\d+)?(\s+(\d+|\*)(/\d+)?){4}$' spec.replicas in body should be less than or equal to 10 ``` If the fields contain valid values, the object creation request is accepted. Save the following YAML to `my-crontab.yaml`: ```yaml apiVersion: "stable.example.com/v1" kind: CronTab metadata: name: my-new-cron-object spec: cronSpec: "* * * * */5" image: my-awesome-cron-image replicas: 5 ``` And create it: ```shell kubectl create -f my-crontab.yaml crontab "my-new-cron-object" created ``` {% endcapture %} {% capture whatsnext %} * Learn how to [Migrate a ThirdPartyResource to CustomResourceDefinition](/docs/tasks/access-kubernetes-api/migrate-third-party-resource/). * See [CustomResourceDefinition](/docs/api-reference/{{page.version}}/#customresourcedefinition-v1beta1-apiextensions). {% endcapture %} {% include templates/task.md %}