kubernetes
/
website
mirror of https://github.com/kubernetes/website.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

Make the markdown prettier: storage-version-migration.md

pull/46201/head
Michael 2024-05-05 18:41:59 +08:00
parent 3dffe58925
commit 06113101af
1 changed files with 83 additions and 37 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

120
content/en/docs/tasks/manage-kubernetes-objects/storage-version-migration.md
View File

@ -1,23 +1,22 @@
---
title: Migrate Kubernetes Objects Using Storage Version Migration
reviewers:
- deads2k
- jpbetz
- enj
- nilekhc
- deads2k
- jpbetz
- enj
- nilekhc
content_type: task
min-kubernetes-server-version: v1.30
weight: 60
---
<!-- overview -->
{{< feature-state feature_gate_name="StorageVersionMigrator" >}}
Kubernetes relies on API data being actively re-written, to support some
maintenance activities related to at rest storage. Two prominent examples are
the versioned schema of stored resources (that is, the preferred storage schema
Kubernetes relies on API data being actively re-written, to support some
maintenance activities related to at rest storage. Two prominent examples are
the versioned schema of stored resources (that is, the preferred storage schema
changing from v1 to v2 for a given resource) and encryption at rest
(that is, rewriting stale data based on a change in how the data should be encrypted).
@ -27,12 +26,13 @@ Install [`kubectl`](/docs/tasks/tools/#kubectl).
{{< include "task-tutorial-prereqs.md" >}} {{< version-check >}}
<!-- steps -->
## Re-encrypt Kubernetes secrets using storage version migration
- To begin with, [configure KMS provider](/docs/tasks/administer-cluster/kms-provider/)
- To begin with, [configure KMS provider](/docs/tasks/administer-cluster/kms-provider/)
to encrypt data at rest in etcd using following encryption configuration.
```yaml
kind: EncryptionConfiguration
apiVersion: apiserver.config.k8s.io/v1
@ -45,15 +45,21 @@ Install [`kubectl`](/docs/tasks/tools/#kubectl).
- name: key1
secret: c2VjcmV0IGlzIHNlY3VyZQ==
```
Make sure to enable automatic reload of encryption
configuration file by setting `--encryption-provider-config-automatic-reload` to true.
configuration file by setting `--encryption-provider-config-automatic-reload` to true.
- Create a Secret using kubectl.
```shell
kubectl create secret generic my-secret --from-literal=key1=supersecret
```
- [Verify](/docs/tasks/administer-cluster/kms-provider/#verifying-that-the-data-is-encrypted)
the serialized data for that Secret object is prefixed with `k8s:enc:aescbc:v1:key1`.
- Update the encryption configuration file as follows to rotate the encryption key.
```yaml
kind: EncryptionConfiguration
apiVersion: apiserver.config.k8s.io/v1
@ -70,9 +76,12 @@ configuration file by setting `--encryption-provider-config-automatic-reload` to
- name: key1
secret: c2VjcmV0IGlzIHNlY3VyZQ==
```
- To ensure that previously created secret `my-secert` is re-encrypted
with new key `key2`, you will use _Storage Version Migration_.
with new key `key2`, you will use _Storage Version Migration_.
- Create a StorageVersionMigration manifest named `migrate-secret.yaml` as follows:
```yaml
kind: StorageVersionMigration
apiVersion: storagemigration.k8s.io/v1alpha1
@ -84,27 +93,31 @@ with new key `key2`, you will use _Storage Version Migration_.
version: v1
resource: secrets
```
Create the object using _kubectl_ as follows:
```shell
kubectl apply -f migrate-secret.yaml
```
- Monitor migration of Secrets by checking the `.status` of the StorageVersionMigration.
A successful migration should have its
`Succeeded` condition set to true. Get the StorageVersionMigration object
as follows:
`Succeeded` condition set to true. Get the StorageVersionMigration object as follows:
```shell
kubectl get storageversionmigration.storagemigration.k8s.io/secrets-migration -o yaml
```
The output is similar to:
```yaml
kind: StorageVersionMigration
apiVersion: storagemigration.k8s.io/v1alpha1
metadata:
name: secrets-migration
uid: 628f6922-a9cb-4514-b076-12d3c178967c
resourceVersion: '90'
creationTimestamp: '2024-03-12T20:29:45Z'
resourceVersion: "90"
creationTimestamp: "2024-03-12T20:29:45Z"
spec:
resource:
group: ""
@ -113,19 +126,21 @@ as follows:
status:
conditions:
- type: Running
status: 'False'
lastUpdateTime: '2024-03-12T20:29:46Z'
status: "False"
lastUpdateTime: "2024-03-12T20:29:46Z"
reason: StorageVersionMigrationInProgress
- type: Succeeded
status: 'True'
lastUpdateTime: '2024-03-12T20:29:46Z'
status: "True"
lastUpdateTime: "2024-03-12T20:29:46Z"
reason: StorageVersionMigrationSucceeded
resourceVersion: '84'
resourceVersion: "84"
```
- [Verify](/docs/tasks/administer-cluster/kms-provider/#verifying-that-the-data-is-encrypted)
the stored secret is now prefixed with `k8s:enc:aescbc:v1:key2`.
## Update the preferred storage schema of a CRD
Consider a scenario where a {{< glossary_tooltip term_id="CustomResourceDefinition" text="CustomResourceDefinition" >}}
(CRD) is created to serve custom resources (CRs) and is set as the preferred storage schema. When it's time
to introduce v2 of the CRD, it can be added for serving only with a conversion
@ -136,6 +151,7 @@ version, it's important to ensure that all existing CRs stored as v1 are migrate
This migration can be achieved through _Storage Version Migration_ to migrate all CRs from v1 to v2.
- Create a manifest for the CRD, named `test-crd.yaml`, as follows:
```yaml
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
@ -163,17 +179,21 @@ This migration can be achieved through _Storage Version Migration_ to migrate al
strategy: Webhook
webhook:
clientConfig:
url: https://127.0.0.1:9443/crdconvert
url: "https://127.0.0.1:9443/crdconvert"
caBundle: <CABundle info>
conversionReviewVersions:
- v1
- v2
```
Create CRD using kubectl
Create CRD using kubectl:
```shell
kubectl apply -f test-crd.yaml
```
- Create a manifest for an example testcrd. Name the manifest `cr1.yaml` and use these contents:
```yaml
apiVersion: stable.example.com/v1
kind: SelfieRequest
@ -181,17 +201,24 @@ This migration can be achieved through _Storage Version Migration_ to migrate al
name: cr1
namespace: default
```
Create CR using kubectl
Create CR using kubectl:
```shell
kubectl apply -f cr1.yaml
```
- Verify that CR is written and stored as v1 by getting the object from etcd.
```shell
ETCDCTL_API=3 etcdctl get /kubernetes.io/stable.example.com/testcrds/default/cr1 [...] | hexdump -C
```
where `[...]` contains the additional arguments for connecting to the etcd server.
- Update the CRD `test-crd.yaml` to include v2 version for serving and storage
and v1 as serving only, as follows:
and v1 as serving only, as follows:
```yaml
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
@ -230,17 +257,21 @@ This migration can be achieved through _Storage Version Migration_ to migrate al
strategy: Webhook
webhook:
clientConfig:
url: 'https://127.0.0.1:9443/crdconvert'
url: "https://127.0.0.1:9443/crdconvert"
caBundle: <CABundle info>
conversionReviewVersions:
- v1
- v2
```
Update CRD using kubectl
Update CRD using kubectl:
```shell
kubectl apply -f test-crd.yaml
```
- Create CR resource file with name `cr2.yaml` as follows:
```yaml
apiVersion: stable.example.com/v2
kind: SelfieRequest
@ -248,16 +279,23 @@ This migration can be achieved through _Storage Version Migration_ to migrate al
name: cr2
namespace: default
```
- Create CR using kubectl
- Create CR using kubectl:
```shell
kubectl apply -f cr2.yaml
```
- Verify that CR is written and stored as v2 by getting the object from etcd.
```shell
ETCDCTL_API=3 etcdctl get /kubernetes.io/stable.example.com/testcrds/default/cr2 [...] | hexdump -C
```
where `[...]` contains the additional arguments for connecting to the etcd server.
- Create a StorageVersionMigration manifest named `migrate-crd.yaml`, with the contents as follows:
```yaml
kind: StorageVersionMigration
apiVersion: storagemigration.k8s.io/v1alpha1
@ -269,26 +307,31 @@ This migration can be achieved through _Storage Version Migration_ to migrate al
version: v1
resource: SelfieRequest
```
Create the object using _kubectl_ as follows:
```shell
kubectl apply -f migrate-crd.yaml
```
- Monitor migration of secrets using status. Successful migration should have
`Succeeded` condition set to "True" in the status field. Get the migration resource
as follows:
```shell
kubectl get storageversionmigration.storagemigration.k8s.io/crdsvm -o yaml
```
The output is similar to:
```yaml
kind: StorageVersionMigration
apiVersion: storagemigration.k8s.io/v1alpha1
metadata:
name: crdsvm
uid: 13062fe4-32d7-47cc-9528-5067fa0c6ac8
resourceVersion: '111'
creationTimestamp: '2024-03-12T22:40:01Z'
resourceVersion: "111"
creationTimestamp: "2024-03-12T22:40:01Z"
spec:
resource:
group: stable.example.com
@ -297,17 +340,20 @@ This migration can be achieved through _Storage Version Migration_ to migrate al
status:
conditions:
- type: Running
status: 'False'
lastUpdateTime: '2024-03-12T22:40:03Z'
status: "False"
lastUpdateTime: "2024-03-12T22:40:03Z"
reason: StorageVersionMigrationInProgress
- type: Succeeded
status: 'True'
lastUpdateTime: '2024-03-12T22:40:03Z'
status: "True"
lastUpdateTime: "2024-03-12T22:40:03Z"
reason: StorageVersionMigrationSucceeded
resourceVersion: '106'
resourceVersion: "106"
```
- Verify that previously created cr1 is now written and stored as v2 by getting the object from etcd.
```shell
ETCDCTL_API=3 etcdctl get /kubernetes.io/stable.example.com/testcrds/default/cr1 [...] | hexdump -C
```
where `[...]` contains the additional arguments for connecting to the etcd server.