Address review comments
xing-yang
parent
6d847b0a31
commit
cd03bdddcf
|
|
@ -1,19 +1,19 @@
|
|||
---
|
||||
layout: blog
|
||||
title: "Introducing Volume Group Snapshot"
|
||||
date: 2023-05-08T10:00:00-08:00
|
||||
title: "Kubernetes 1.27: Introducing An API For Volume Group Snapshots"
|
||||
date: 2023-05-08
|
||||
slug: kubernetes-1-27-volume-group-snapshot-alpha
|
||||
---
|
||||
|
||||
**Author:** Xing Yang (VMware)
|
||||
|
||||
Volume group snapshot is introduced as an Alpha feature in Kubernetes v1.27.
|
||||
This feature introduces a Kubernetes API that allows users to take a crash consistent
|
||||
snapshot for multiple volumes together. It uses a label selector to group multiple
|
||||
PersistentVolumeClaims for snapshotting.
|
||||
This new feature is only supported for CSI volume drivers.
|
||||
This feature introduces a Kubernetes API that allows users to take crash consistent
|
||||
snapshots for multiple volumes together. It uses a label selector to group multiple
|
||||
`PersistentVolumeClaims` for snapshotting.
|
||||
This new feature is only supported for [CSI](https://kubernetes-csi.github.io/docs/) volume drivers.
|
||||
|
||||
## What is Volume Group Snapshot
|
||||
## An overview of volume group snapshots
|
||||
|
||||
Some storage systems provide the ability to create a crash consistent snapshot of
|
||||
multiple volumes. A group snapshot represents “copies” from multiple volumes that
|
||||
|
|
@ -21,7 +21,7 @@ are taken at the same point-in-time. A group snapshot can be used either to rehy
|
|||
new volumes (pre-populated with the snapshot data) or to restore existing volumes to
|
||||
a previous state (represented by the snapshots).
|
||||
|
||||
## Why add Volume Group Snapshots to Kubernetes?
|
||||
## Why add volume group snapshots to Kubernetes?
|
||||
|
||||
The Kubernetes volume plugin system already provides a powerful abstraction that
|
||||
automates the provisioning, attaching, mounting, resizing, and snapshotting of block
|
||||
|
|
@ -30,9 +30,9 @@ and file storage.
|
|||
Underpinning all these features is the Kubernetes goal of workload portability:
|
||||
Kubernetes aims to create an abstraction layer between distributed applications and
|
||||
underlying clusters so that applications can be agnostic to the specifics of the
|
||||
cluster they run on and application deployment requires no “cluster specific” knowledge.
|
||||
cluster they run on and application deployment requires no cluster specific knowledge.
|
||||
|
||||
There is already a [VolumeSnapshot API](https://github.com/kubernetes/enhancements/tree/master/keps/sig-storage/177-volume-snapshot)
|
||||
There is already a [VolumeSnapshot](/docs/concepts/storage/volume-snapshots/) API
|
||||
that provides the ability to take a snapshot of a persistent volume to protect against
|
||||
data loss or data corruption. However, there are other snapshotting functionalities
|
||||
not covered by the VolumeSnapshot API.
|
||||
|
|
@ -45,25 +45,28 @@ If snapshots for the data volume and the logs volume are taken at different time
|
|||
the application will not be consistent and will not function properly if it is restored
|
||||
from those snapshots when a disaster strikes.
|
||||
|
||||
It is true that we can quiesce the application first, take an individual snapshot from
|
||||
It is true that you can quiesce the application first, take an individual snapshot from
|
||||
each volume that is part of the application one after the other, and then unquiesce the
|
||||
application after all the individual snapshots are taken. This way we will get application
|
||||
consistent snapshots.
|
||||
However, application quiesce is time consuming. Sometimes it may not be possible to
|
||||
quiesce an application. Taking individual snapshots one after another may also take
|
||||
longer time compared to taking a consistent group snapshot. Some users may not want
|
||||
to do application quiesce very frequently for these reasons. For example, a user may
|
||||
want to run weekly backups with application quiesce and nightly backups without
|
||||
application quiesce but with consistent group support which provides crash consistency
|
||||
across all volumes in the group.
|
||||
application after all the individual snapshots are taken. This way, you would get
|
||||
application consistent snapshots.
|
||||
|
||||
However, sometimes it may not be possible to quiesce an application or the application
|
||||
quiesce can be too expensive so you want to do it less frequently. Taking individual
|
||||
snapshots one after another may also take longer time compared to taking a consistent
|
||||
group snapshot. Some users may not want to do application quiesce very often for these
|
||||
reasons. For example, a user may want to run weekly backups with application quiesce
|
||||
and nightly backups without application quiesce but with consistent group support which
|
||||
provides crash consistency across all volumes in the group.
|
||||
|
||||
## Kubernetes Volume Group Snapshots API
|
||||
|
||||
Kubernetes Volume Group Snapshots introduce [three new API objects](https://github.com/kubernetes-csi/external-snapshotter/blob/master/client/apis/volumegroupsnapshot/v1alpha1/types.go) for managing snapshots:
|
||||
Kubernetes Volume Group Snapshots introduce [three new API
|
||||
objects](https://github.com/kubernetes-csi/external-snapshotter/blob/master/client/apis/volumegroupsnapshot/v1alpha1/types.go)
|
||||
for managing snapshots:
|
||||
|
||||
`VolumeGroupSnapshot`
|
||||
: Created by a Kubernetes user (or perhaps by your own automation) to request
|
||||
creation of a volume group snapshot for multiple volumes.
|
||||
creation of a volume group snapshot for multiple persistent volume claims.
|
||||
It contains information about the volume group snapshot operation such as the
|
||||
timestamp when the volume group snapshot was taken and whether it is ready to use.
|
||||
The creation and deletion of this object represents a desire to create or delete a
|
||||
|
|
@ -81,41 +84,50 @@ was created with a one-to-one mapping.
|
|||
: Created by cluster administrators to describe how volume group snapshots should be
|
||||
created. including the driver information, the deletion policy, etc.
|
||||
|
||||
The Volume Group Snapshot objects are defined as CustomResourceDefinitions (CRDs).
|
||||
These three API kinds are defined as CustomResourceDefinitions (CRDs).
|
||||
These CRDs must be installed in a Kubernetes cluster for a CSI Driver to support
|
||||
volume group snapshots.
|
||||
|
||||
## How do I use Kubernetes Volume Group Snapshots
|
||||
|
||||
Volume Group Snapshot feature is implemented in the
|
||||
Volume group snapshots are implemented in the
|
||||
[external-snapshotter](https://github.com/kubernetes-csi/external-snapshotter) repository. Implementing volume
|
||||
group snapshots meant adding or changing several components:
|
||||
|
||||
* Kubernetes Volume Group Snapshot CRDs
|
||||
* Added new CustomResourceDefinitions for VolumeGroupSnapshot and two supporting APIs.
|
||||
* Volume group snapshot controller logic is added to the common snapshot controller.
|
||||
* Volume group snapshot validation webhook logic is added to the common snapshot validation webhook.
|
||||
* Logic to make CSI calls is added to CSI Snapshotter sidecar controller.
|
||||
* Adding logic to make CSI calls into the snapshotter sidecar controller.
|
||||
|
||||
The volume snapshot controller, CRDs, and validation webhook are deployed once per
|
||||
cluster, while the sidecar is bundled with each CSI driver.
|
||||
|
||||
Therefore, it makes sense to deploy the volume snapshot controller, CRDs, and validation
|
||||
webhook as a cluster addon. It is strongly recommended that Kubernetes distributors
|
||||
webhook as a cluster addon. I strongly recommend that Kubernetes distributors
|
||||
bundle and deploy the volume snapshot controller, CRDs, and validation webhook as part
|
||||
of their Kubernetes cluster management process (independent of any CSI Driver).
|
||||
|
||||
### Creating a new group snapshot with Kubernetes
|
||||
|
||||
Once a VolumeGroupSnapshotClass object is defined and you have volumes you want to
|
||||
snapshot together, you may create a new group snapshot by creating a VolumeGroupSnapshot
|
||||
snapshot together, you may request a new group snapshot by creating a VolumeGroupSnapshot
|
||||
object.
|
||||
|
||||
The source of the group snapshot specifies whether the underlying group snapshot
|
||||
should be dynamically created or if a pre-existing VolumeGroupSnapshotContent
|
||||
should be used. One of the following members in the source must be set.
|
||||
should be used.
|
||||
|
||||
* Selector - Selector is a label query over persistent volume claims that are to be grouped together for snapshotting. This labelSelector will be used to match the label added to a PVC.
|
||||
* VolumeGroupSnapshotContentName - specifies the name of a pre-existing VolumeGroupSnapshotContent object representing an existing volume group snapshot.
|
||||
A pre-existing VolumeGroupSnapshotContent is created by a cluster administrator.
|
||||
It contains the details of the real volume group snapshot on the storage system which
|
||||
is available for use by cluster users.
|
||||
|
||||
One of the following members in the source of the group snapshot must be set.
|
||||
|
||||
* `selector` - a label query over PersistentVolumeClaims that are to be grouped
|
||||
together for snapshotting. This labelSelector will be used to match the label
|
||||
added to a PVC.
|
||||
* `volumeGroupSnapshotContentName` - specifies the name of a pre-existing
|
||||
VolumeGroupSnapshotContent object representing an existing volume group snapshot.
|
||||
|
||||
For dynamic provisioning, a selector must be set so that the snapshot controller can
|
||||
find PVCs with the matching labels to be snapshotted together.
|
||||
|
|
@ -130,7 +142,8 @@ spec:
|
|||
volumeGroupSnapshotClassName: csi-groupSnapclass
|
||||
source:
|
||||
selector:
|
||||
group: myGroup
|
||||
matchLabels:
|
||||
group: myGroup
|
||||
```
|
||||
|
||||
In the VolumeGroupSnapshot spec, a user can specify the VolumeGroupSnapshotClass which
|
||||
|
|
@ -187,7 +200,7 @@ snapshots that are part of a group snapshot.
|
|||
|
||||
## As a storage vendor, how do I add support for group snapshots to my CSI driver?
|
||||
|
||||
To implement the volume group snapshot feature, a CSI driver MUST:
|
||||
To implement the volume group snapshot feature, a CSI driver **must**:
|
||||
|
||||
* Implement a new group controller service.
|
||||
* Implement group controller RPCs: `CreateVolumeGroupSnapshot`, `DeleteVolumeGroupSnapshot`, and `GetVolumeGroupSnapshot`.
|
||||
|
|
@ -197,7 +210,6 @@ See the [CSI spec](https://github.com/container-storage-interface/spec/blob/mast
|
|||
and the [Kubernetes-CSI Driver Developer Guide](https://kubernetes-csi.github.io/docs/)
|
||||
for more details.
|
||||
|
||||
Although Kubernetes poses as little prescriptive on the packaging and deployment of
|
||||
a CSI Volume Driver as possible, it provides a suggested mechanism to deploy a
|
||||
containerized CSI driver to simplify the process.
|
||||
|
||||
|
|
@ -215,8 +227,11 @@ The external-snapshotter watches the Kubernetes API server for the
|
|||
The alpha implementation of volume group snapshots for Kubernetes has the following
|
||||
limitations:
|
||||
|
||||
* Does not support reverting an existing PVC to an earlier state represented by a snapshot that is part of a group snapshot (only supports provisioning a new volume from a snapshot).
|
||||
* No application consistency guarantees beyond any guarantees provided by the storage system (e.g. crash consistency).
|
||||
* Does not support reverting an existing PVC to an earlier state represented by
|
||||
a snapshot (only supports provisioning a new volume from a snapshot).
|
||||
* No application consistency guarantees beyond any guarantees provided by the storage system
|
||||
(e.g. crash consistency). See this [doc](https://github.com/kubernetes/community/blob/master/wg-data-protection/data-protection-workflows-white-paper.md#quiesce-and-unquiesce-hooks)
|
||||
for more discussions on application consistency.
|
||||
|
||||
## What’s next?
|
||||
|
||||
|
|
@ -227,11 +242,11 @@ replication group, volume placement, application quiescing, changed block tracki
|
|||
|
||||
## How can I learn more?
|
||||
|
||||
The design spec for the volume group snapshot feature is [here](https://github.com/kubernetes/enhancements/tree/master/keps/sig-storage/3476-volume-group-snapshot).
|
||||
|
||||
The code repository for volume group snapshot APIs and controller is [here](https://github.com/kubernetes-csi/external-snapshotter).
|
||||
|
||||
Check out additional documentation on the group snapshot feature [here](https://kubernetes-csi.github.io/docs/).
|
||||
- The [design spec](https://github.com/kubernetes/enhancements/tree/master/keps/sig-storage/3476-volume-group-snapshot)
|
||||
for the volume group snapshot feature.
|
||||
- The [code repository](https://github.com/kubernetes-csi/external-snapshotter) for volume group
|
||||
snapshot APIs and controller.
|
||||
- CSI [documentation](https://kubernetes-csi.github.io/docs/) on the group snapshot feature.
|
||||
|
||||
## How do I get involved?
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue