add translation for
Paco Xu
parent
33f4438d6d
commit
c608681964
|
|
@ -1,6 +1,6 @@
|
|||
---
|
||||
layout: blog
|
||||
title: "Kubernetes的取证容器检查点"
|
||||
title: "Kubernetes 的取证容器检查点"
|
||||
date: 2022-12-05
|
||||
slug: forensic-container-checkpointing-alpha
|
||||
---
|
||||
|
|
@ -27,15 +27,14 @@ sandbox environment multiple times without the original container being aware
|
|||
of it. Forensic container checkpointing was introduced as an alpha feature in
|
||||
Kubernetes v1.25.
|
||||
-->
|
||||
取证容器检查点（Forensic container checkpointing）基于 [CRIU][criu]（Checkpoint/Restore
|
||||
In Userspace ，用户空间的检查点/恢复），并允许创建正在运行的容器的有状态副本，而容器不知道它正在被
|
||||
检查。容器的副本，可以在沙箱环境中被多次分析和恢复，而原始容器并不知道。取证容器检查点是作为一个
|
||||
alpha 特性在 Kubernetes v1.25 中引入的。
|
||||
取证容器检查点（Forensic container checkpointing）基于 [CRIU][criu]（Checkpoint/Restore In Userspace ，用户空间的检查点/恢复），
|
||||
并允许创建正在运行的容器的有状态副本，而容器不知道它正在被检查。容器的副本，可以在沙箱环境中被多次分析和恢复，而原始容器并不知道。
|
||||
取证容器检查点是作为一个 alpha 特性在 Kubernetes v1.25 中引入的。
|
||||
|
||||
<!--
|
||||
## How does it work?
|
||||
-->
|
||||
## 它是如何工作的？
|
||||
## 工作原理
|
||||
|
||||
<!--
|
||||
With the help of CRIU it is possible to checkpoint and restore containers.
|
||||
|
|
@ -43,13 +42,13 @@ CRIU is integrated in runc, crun, CRI-O and containerd and forensic container
|
|||
checkpointing as implemented in Kubernetes uses these existing CRIU
|
||||
integrations.
|
||||
-->
|
||||
在 CRIU 的帮助下，检查（checkpoint）和恢复容器成为可能。CRIU 集成在 runc、crun、CRI-O 和
|
||||
containerd 中，而在 Kubernetes 中实现的取证容器检查点使用这些现有的 CRIU 集成。
|
||||
在 CRIU 的帮助下，检查（checkpoint）和恢复容器成为可能。CRIU 集成在 runc、crun、CRI-O 和 containerd 中，
|
||||
而在 Kubernetes 中实现的取证容器检查点使用这些现有的 CRIU 集成。
|
||||
|
||||
<!--
|
||||
## Why is it important?
|
||||
-->
|
||||
## 为什么重要？
|
||||
## 这一特性为何重要？
|
||||
|
||||
<!--
|
||||
With the help of CRIU and the corresponding integrations it is possible to get
|
||||
|
|
@ -61,10 +60,9 @@ checkpoint and analysing the container in a sandboxed environment offers the
|
|||
possibility to inspect the container without the original container and maybe
|
||||
attacker being aware of the inspection.
|
||||
-->
|
||||
借助 CRIU 和相应的集成，可以获得磁盘上正在运行的容器的所有信息和状态，供以后进行取证分析。取证分析
|
||||
对于在不阻止或影响可疑容器的情况下，对其进行检查可能很重要。如果容器确实受到攻击，攻击者可能会检测到
|
||||
检查容器的企图。获取检查点并在沙箱环境中分析容器，提供了在原始容器和可能的攻击者不知道检查的情况下检
|
||||
查容器的可能性。
|
||||
借助 CRIU 和相应的集成，可以获得磁盘上正在运行的容器的所有信息和状态，供以后进行取证分析。
|
||||
取证分析对于在不阻止或影响可疑容器的情况下，对其进行检查可能很重要。如果容器确实受到攻击，攻击者可能会检测到检查容器的企图。
|
||||
获取检查点并在沙箱环境中分析容器，提供了在原始容器和可能的攻击者不知道检查的情况下检查容器的可能性。
|
||||
|
||||
<!--
|
||||
In addition to the forensic container checkpointing use case, it is also
|
||||
|
|
@ -73,8 +71,8 @@ the internal state. Especially for stateful containers with long initialization
|
|||
times restoring from a checkpoint might save time after a reboot or enable much
|
||||
faster startup times.
|
||||
-->
|
||||
除了取证容器检查点用例，还可以在不丢失内部状态的情况下，将容器从一个节点迁移到另一个节点。特别是对于
|
||||
初始化时间长的有状态容器，从检查点恢复，可能会节省重新启动后的时间，或者实现更快的启动时间。
|
||||
除了取证容器检查点用例，还可以在不丢失内部状态的情况下，将容器从一个节点迁移到另一个节点。
|
||||
特别是对于初始化时间长的有状态容器，从检查点恢复，可能会节省重新启动后的时间，或者实现更快的启动时间。
|
||||
|
||||
<!--
|
||||
## How do I use container checkpointing?
|
||||
|
|
@ -87,8 +85,7 @@ make sure to enable the `ContainerCheckpoint` gate before you can use the new
|
|||
feature.
|
||||
-->
|
||||
|
||||
该功能在[特性门][container-checkpoint-feature-gate]后面，因此在使用新功能之前，请确保启用
|
||||
ContainerCheckpoint 特性门。
|
||||
该功能在[特性门][container-checkpoint-feature-gate]后面，因此在使用新功能之前，请确保启用了 ContainerCheckpoint 特性门。
|
||||
|
||||
<!--
|
||||
The runtime must also support container checkpointing:
|
||||
|
|
@ -118,16 +115,15 @@ is also necessary to install CRIU. Usually runc or crun depend on CRIU and
|
|||
therefore it is installed automatically.
|
||||
-->
|
||||
要将取证容器检查点与 CRI-O 结合使用，需要使用命令行选项--enable-criu-support=true 启动运行时。
|
||||
Kubernetes 方面，你需要在启用 ContainerCheckpoint 特性门的情况下运行你的集群。由于检查点功能
|
||||
是由 CRIU 提供的，因此也有必要安装 CRIU。通常 runc 或 crun 依赖于 CRIU，因此它是自动安装的。
|
||||
Kubernetes 方面，你需要在启用 ContainerCheckpoint 特性门的情况下运行你的集群。由于检查点功能是由 CRIU 提供的，因此也有必要安装 CRIU。
|
||||
通常 runc 或 crun 依赖于 CRIU，因此它是自动安装的。
|
||||
|
||||
<!--
|
||||
It is also important to mention that at the time of writing the checkpointing functionality is
|
||||
to be considered as an alpha level feature in CRI-O and Kubernetes and the
|
||||
security implications are still under consideration.
|
||||
-->
|
||||
值得一提的是，在编写本文时，检查点功能被认为是 CRI-O 和 Kubernetes 中的一个 alpha 级特性，
|
||||
其安全影响仍在评估之中。
|
||||
值得一提的是，在编写本文时，检查点功能被认为是 CRI-O 和 Kubernetes 中的一个 alpha 级特性，其安全影响仍在评估之中。
|
||||
|
||||
<!--
|
||||
Once containers and pods are running it is possible to create a checkpoint.
|
||||
|
|
@ -140,8 +136,8 @@ checkpoint:
|
|||
curl -X POST "https://localhost:10250/checkpoint/namespace/podId/container"
|
||||
```
|
||||
-->
|
||||
一旦容器和 pod 开始运行，就可以创建一个检查点。[检查点][kubelet-checkpoint-api]目前只在
|
||||
kubelet 级别暴露。要检查一个容器，可以在运行该容器的节点上运行 curl，并触发一个检查点：
|
||||
一旦容器和 pod 开始运行，就可以创建一个检查点。[检查点][kubelet-checkpoint-api]目前只在 **kubelet** 级别暴露。
|
||||
要检查一个容器，可以在运行该容器的节点上运行 curl，并触发一个检查点：
|
||||
|
||||
```shell
|
||||
curl -X POST "https://localhost:10250/checkpoint/namespace/podId/container"
|
||||
|
|
@ -149,14 +145,13 @@ curl -X POST "https://localhost:10250/checkpoint/namespace/podId/container"
|
|||
|
||||
<!--
|
||||
For a container named *counter* in a pod named *counters* in a namespace named
|
||||
*default* the **kubelet** API endpoint is reachable at:
|
||||
*default* the *kubelet* API endpoint is reachable at:
|
||||
|
||||
```shell
|
||||
curl -X POST "https://localhost:10250/checkpoint/default/counters/counter"
|
||||
```
|
||||
-->
|
||||
对于名为 default 的命名空间中名为 counters 的 pod 中的名为 counter 的容器，kubelet API
|
||||
端点可在以下位置访问：
|
||||
对于名为 **default** 的命名空间中名为 **counters** 的 pod 中的名为 **counter** 的容器， **kubelet** API 端点可在以下位置访问：
|
||||
|
||||
```shell
|
||||
curl -X POST "https://localhost:10250/checkpoint/default/counters/counter"
|
||||
|
|
@ -171,8 +166,7 @@ use of the *kubelet* `checkpoint` API:
|
|||
--insecure --cert /var/run/kubernetes/client-admin.crt --key /var/run/kubernetes/client-admin.key
|
||||
```
|
||||
-->
|
||||
为了完整起见，以下 curl 命令行选项对于让 curl 接受 kubelet 的自签名证书并授权使用 kubelet
|
||||
检查点 API 是必要的：
|
||||
为了完整起见，以下 curl 命令行选项对于让 curl 接受 **kubelet** 的自签名证书并授权使用 **kubelet** 检查点 API 是必要的：
|
||||
|
||||
```shell
|
||||
--insecure --cert /var/run/kubernetes/client-admin.crt --key /var/run/kubernetes/client-admin.key
|
||||
|
|
@ -189,8 +183,8 @@ Once the checkpointing has finished the checkpoint should be available at
|
|||
|
||||
You could then use that tar archive to restore the container somewhere else.
|
||||
-->
|
||||
触发这个 kubelet API 将从 CRI-O 请求创建一个检查点，CRI-O 从你的低级运行时（例如，runc）
|
||||
请求一个检查点。看到这个请求，runc 调用 criu 工具来执行实际的检查点操作。
|
||||
触发这个 **kubelet** API 将从 CRI-O 请求创建一个检查点，CRI-O 从你的低级运行时（例如，runc）请求一个检查点。
|
||||
看到这个请求，runc 调用 criu 工具来执行实际的检查点操作。
|
||||
|
||||
检查点操作完成后，检查点应该位于
|
||||
`/var/lib/kubelet/checkpoints/checkpoint-<pod-name>_<namespace-name>-<container-name>-<timestamp>.tar`
|
||||
|
|
@ -210,9 +204,8 @@ during restore, I recommend that you use the latest version of CRI-O from the
|
|||
manually create certain directories Kubernetes would create before starting the
|
||||
container.
|
||||
-->
|
||||
使用检查点 tar 归档文件，可以在 Kubernetes 之外的 CRI-O 沙箱实例中恢复容器。为了在恢复过程中
|
||||
获得更好的用户体验，我建议你使用 main CRI-O GitHub 分支中最新版本的 CRI-O。如果你使用的是
|
||||
CRI-O v1.25，你需要在启动容器之前手动创建 Kubernetes 会创建的某些目录。
|
||||
使用检查点 tar 归档文件，可以在 Kubernetes 之外的 CRI-O 沙箱实例中恢复容器。为了在恢复过程中获得更好的用户体验，我建议你使用 CRI-O GitHub 的 **main** 分支中最新版本的 CRI-O。
|
||||
如果你使用的是 CRI-O v1.25，你需要在启动容器之前手动创建 Kubernetes 会创建的某些目录。
|
||||
|
||||
<!--
|
||||
The first step to restore a container outside of Kubernetes is to create a pod sandbox
|
||||
|
|
@ -222,7 +215,7 @@ using *crictl*:
|
|||
crictl runp pod-config.json
|
||||
```
|
||||
-->
|
||||
在 Kubernetes 外恢复容器的第一步是使用 crictl 创建一个 pod 沙箱：
|
||||
在 Kubernetes 外恢复容器的第一步是使用 **crictl** 创建一个 pod 沙箱：
|
||||
|
||||
```shell
|
||||
crictl runp pod-config.json
|
||||
|
|
@ -319,10 +312,10 @@ The security implications of starting CRI-O with CRIU support are not yet clear
|
|||
and therefore the functionality as well as the image format should be used with
|
||||
care.
|
||||
-->
|
||||
生成的镜像未经标准化，只能与 CRI-O 结合使用。请将此镜像格式视为 pre-alpha 格式。社区正在[讨论][image-spec-discussion]
|
||||
如何标准化这样的检查点镜像格式。重要的是要记住，这种尚未标准化的镜像格式只有在 CRI-O 已经用
|
||||
`--enable-criu-support=true` 启动时才有效。在 CRIU 支持下启动 CRI-O 的安全影响尚不清楚，
|
||||
因此应谨慎使用功能和镜像格式。
|
||||
生成的镜像未经标准化，只能与 CRI-O 结合使用。请将此镜像格式视为 pre-alpha 格式。
|
||||
社区正在[讨论][image-spec-discussion]如何标准化这样的检查点镜像格式。
|
||||
重要的是要记住，这种尚未标准化的镜像格式只有在 CRI-O 已经用`--enable-criu-support=true` 启动时才有效。
|
||||
在 CRIU 支持下启动 CRI-O 的安全影响尚不清楚，因此应谨慎使用功能和镜像格式。
|
||||
|
||||
<!--
|
||||
Now, you'll need to push that image to a container image registry. For example:
|
||||
|
|
@ -354,8 +347,7 @@ spec:
|
|||
nodeName: <destination-node>
|
||||
```
|
||||
-->
|
||||
要恢复此检查点镜像（container-image-registry.example/user/checkpoint-image:latest），
|
||||
该镜像需要在 Pod 的规范中列出。下面是一个清单示例：
|
||||
要恢复此检查点镜像（container-image-registry.example/user/checkpoint-image:latest），该镜像需要在 Pod 的规范中列出。下面是一个清单示例：
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
|
|
@ -379,26 +371,23 @@ instead of the usual steps to create and start a container,
|
|||
CRI-O fetches the checkpoint data and restores the container from that
|
||||
specified checkpoint.
|
||||
-->
|
||||
Kubernetes 将新的 Pod 调度到一个节点上。该节点上的 kubelet 指示容器运行时（本例中为 CRI-O）
|
||||
基于指定为 `registry/user/checkpoint-image:latest` 的镜像创建并启动容器。CRI-O 检测到
|
||||
`registry/user/checkpoint-image:latest` 是对检查点数据的引用，而不是容器镜像。然后，与创建
|
||||
和启动容器的通常步骤不同，CRI-O 获取检查点数据，并从指定的检查点恢复容器。
|
||||
Kubernetes 将新的 Pod 调度到一个节点上。该节点上的 kubelet 指示容器运行时（本例中为 CRI-O）基于指定为 `registry/user/checkpoint-image:latest` 的镜像创建并启动容器。
|
||||
CRI-O 检测到 `registry/user/checkpoint-image:latest` 是对检查点数据的引用，而不是容器镜像。
|
||||
然后，与创建和启动容器的通常步骤不同，CRI-O 获取检查点数据，并从指定的检查点恢复容器。
|
||||
|
||||
<!--
|
||||
The application in that Pod would continue running as if the checkpoint had not been taken;
|
||||
within the container, the application looks and behaves like any other container that had been
|
||||
started normally and not restored from a checkpoint.
|
||||
-->
|
||||
该 Pod 中的应用程序将继续运行，就像检查点未被获取一样；在该容器中，应用程序的外观和行为，与正常
|
||||
启动且未从检查点恢复的任何其他容器相似。
|
||||
该 Pod 中的应用程序将继续运行，就像检查点未被获取一样；在该容器中，应用程序的外观和行为，与正常启动且未从检查点恢复的任何其他容器相似。
|
||||
|
||||
<!--
|
||||
With these steps, it is possible to replace a Pod running on one node
|
||||
with a new equivalent Pod that is running on a different node,
|
||||
and without losing the state of the containers in that Pod.
|
||||
-->
|
||||
通过这些步骤，可以用在不同节点上运行的新的等效 Pod，替换在一个节点上运行的 Pod，而不会丢失该 Pod
|
||||
中容器的状态。
|
||||
通过这些步骤，可以用在不同节点上运行的新的等效 Pod，替换在一个节点上运行的 Pod，而不会丢失该 Pod中容器的状态。
|
||||
|
||||
<!--
|
||||
## How do I get involved?
|
||||
|
|
@ -428,7 +417,7 @@ can be analyzed.
|
|||
-->
|
||||
有关如何分析容器检查点的详细信息，请参阅后续文章[取证容器分析][forensic-container-analysis]。
|
||||
|
||||
[forensic-container-analysis]: /blog/2023/03/10/forensic-container-analysis/
|
||||
[forensic-container-analysis]: /zh-cn/blog/2023/03/10/forensic-container-analysis/
|
||||
[criu]: https://criu.org/
|
||||
[containerd-checkpoint-restore-pr]: https://github.com/containerd/containerd/pull/6965
|
||||
[container-checkpoint-feature-gate]: https://kubernetes.io/docs/reference/command-line-tools-reference/feature-gates/
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
---
|
||||
layout: blog
|
||||
title: "Kubernetes 的取证容器检查点"
|
||||
title: "Kubernetes 的容器检查点分析"
|
||||
date: 2023-03-10
|
||||
slug: forensic-container-analysis
|
||||
---
|
||||
|
|
@ -17,6 +17,7 @@ slug: forensic-container-analysis
|
|||
-->
|
||||
|
||||
**译者**：[Paco Xu](https://github.com/pacoxu) (Daocloud)
|
||||
|
||||
<!--
|
||||
In my previous article, [Forensic container checkpointing in
|
||||
Kubernetes][forensic-blog], I introduced checkpointing in Kubernetes
|
||||
|
|
@ -26,21 +27,22 @@ any details how to do the actual analysis of the checkpoint created by
|
|||
Kubernetes. In this article I want to provide details how the
|
||||
checkpoint can be analyzed.
|
||||
-->
|
||||
在我的上一篇文章中，[Kubernetes的取证容器检查点][forensic-blog]，我在 Kubernetes 中介绍了检查点
|
||||
以及如何设置以及如何使用它。 的名称
|
||||
功能是取证容器检查点，但我没有深入讨论
|
||||
任何详细信息如何对创建的检查点进行实际分析
|
||||
库伯内斯。 在这篇文章中，我想详细介绍如何
|
||||
可以分析检查点。
|
||||
|
||||
检查点仍然是 Kubernetes 中的 alpha 功能，本文
|
||||
希望提供该功能未来如何运作的预览。
|
||||
在我之前的文章 [Kubernetes 中的取证容器检查点][forensic-blog] 中，我介绍了检查点以及如何创建和使用它。
|
||||
该特性的名称是取证容器检查点，但我没有详细介绍如何对 Kubernetes 创建的检查点进行实际分析。
|
||||
在本文中，我想提供如何分析检查点的详细信息。
|
||||
|
||||
<!--
|
||||
Checkpointing is still an alpha feature in Kubernetes and this article
|
||||
wants to provide a preview how the feature might work in the future.
|
||||
-->
|
||||
检查点仍然是 Kubernetes 中的一个 alpha 功能，本文希望提供该功能未来如何工作的预览。
|
||||
|
||||
<!--
|
||||
## Preparation
|
||||
-->
|
||||
## 准备
|
||||
|
||||
<!--
|
||||
Details about how to configure Kubernetes and the underlying CRI implementation
|
||||
to enable checkpointing support can be found in my [Forensic container
|
||||
checkpointing in Kubernetes][forensic-blog] article.
|
||||
|
|
@ -49,8 +51,16 @@ As an example I prepared a container image (`quay.io/adrianreber/counter:blog`)
|
|||
which I want to checkpoint and then analyze in this article. This container allows
|
||||
me to create files in the container and also store information in memory which
|
||||
I later want to find in the checkpoint.
|
||||
-->
|
||||
有关如何配置 Kubernetes 和底层 CRI 实现以启用检查点支持的详细信息，请参阅 [Kubernetes 中的取证容器检查点][forensic-blog]文章。
|
||||
|
||||
作为示例，我准备了一个容器镜像（`quay.io/adrianreber/counter:blog`），我想对其进行检查点，然后在本文中进行分析。
|
||||
这个容器允许我在容器中创建文件，并将信息存储在内存中，稍后我想在检查点中找到这些信息。
|
||||
|
||||
<!--
|
||||
To run that container I need a pod, and for this example I am using the following Pod manifest:
|
||||
-->
|
||||
要运行该容器，我需要一个 pod，在本示例中，我使用以下 Pod 清单：
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
|
|
@ -63,10 +73,15 @@ spec:
|
|||
image: quay.io/adrianreber/counter:blog
|
||||
```
|
||||
|
||||
<!--
|
||||
This results in a container called `counter` running in a pod called `counters`.
|
||||
|
||||
Once the container is running I am performing following actions with that
|
||||
container:
|
||||
-->
|
||||
这会导致一个名为 `counter` 的容器在名为 `counters` 的 Pod 中运行。
|
||||
|
||||
容器运行后，我将对该容器执行以下操作：
|
||||
|
||||
```console
|
||||
$ kubectl get pod counters --template '{{.status.podIP}}'
|
||||
|
|
@ -76,6 +91,7 @@ $ curl 10.88.0.25:8088/secret?RANDOM_1432_KEY
|
|||
$ curl 10.88.0.25:8088
|
||||
```
|
||||
|
||||
<!--
|
||||
The first access creates a file called `test-file` with the content `test-file`
|
||||
in the container and the second access stores my secret information
|
||||
(`RANDOM_1432_KEY`) somewhere in the container's memory. The last access just
|
||||
|
|
@ -87,30 +103,53 @@ the checkpoint. As described in the previous article this requires access to the
|
|||
|
||||
For a container named *counter* in a pod named *counters* in a namespace named
|
||||
*default* the *kubelet* API endpoint is reachable at:
|
||||
-->
|
||||
1. 第一次访问在容器中创建一个名为 `test-file` 的文件，其内容为 `test-file`;
|
||||
2. 第二次访问将我的秘密信息（`RANDOM_1432_KEY`）存储在容器内存中的某处;
|
||||
3. 最后一次访问在内部日志文件中添加了一行。
|
||||
|
||||
在分析检查点之前的最后一步是告诉 Kubernetes 创建检查点。如上一篇文章所述，这需要访问 **kubelet** 唯一的“检查点” API 端点。
|
||||
|
||||
对于名为 **default** 的命名空间中名为 **counters** 的 pod 中的名为 **counter** 的容器， **kubelet** API 端点可在以下位置访问：
|
||||
|
||||
```shell
|
||||
# run this on the node where that Pod is executing
|
||||
curl -X POST "https://localhost:10250/checkpoint/default/counters/counter"
|
||||
```
|
||||
|
||||
<!--
|
||||
For completeness the following `curl` command-line options are necessary to
|
||||
have `curl` accept the *kubelet*'s self signed certificate and authorize the
|
||||
use of the *kubelet* `checkpoint` API:
|
||||
-->
|
||||
为了完整起见，以下 curl 命令行选项对于让 curl 接受 **kubelet** 的自签名证书并授权使用 **kubelet** 检查点 API 是必要的：
|
||||
|
||||
```shell
|
||||
--insecure --cert /var/run/kubernetes/client-admin.crt --key /var/run/kubernetes/client-admin.key
|
||||
```
|
||||
|
||||
<!--
|
||||
Once the checkpointing has finished the checkpoint should be available at
|
||||
`/var/lib/kubelet/checkpoints/checkpoint-<pod-name>_<namespace-name>-<container-name>-<timestamp>.tar`
|
||||
-->
|
||||
检查点操作完成后，检查点应该位于 `/var/lib/kubelet/checkpoints/checkpoint-<pod-name>_<namespace-name>-<container-name>-<timestamp>.tar`
|
||||
|
||||
<!--
|
||||
In the following steps of this article I will use the name `checkpoint.tar`
|
||||
when analyzing the checkpoint archive.
|
||||
-->
|
||||
在本文的以下步骤中，我将在分析检查点归档时使用名称 `checkpoint.tar`。
|
||||
|
||||
<!--
|
||||
## Checkpoint archive analysis using `checkpointctl`
|
||||
-->
|
||||
## 使用 `checkpointctl` 进行检查点归档分析
|
||||
|
||||
<!--
|
||||
To get some initial information about the checkpointed container I am using the
|
||||
tool [checkpointctl][checkpointctl] like this:
|
||||
-->
|
||||
我使用工具 [checkpointctl][checkpointctl] 获取有关检查点容器的一些初始信息，如下所示：
|
||||
|
||||
```console
|
||||
$ checkpointctl show checkpoint.tar --print-stats
|
||||
|
|
@ -127,21 +166,32 @@ CRIU dump statistics
|
|||
+---------------+-------------+--------------+---------------+---------------+---------------+
|
||||
```
|
||||
|
||||
<!--
|
||||
This gives me already some information about the checkpoint in that checkpoint
|
||||
archive. I can see the name of the container, information about the container
|
||||
runtime and container engine. It also lists the size of the checkpoint (`CHKPT
|
||||
SIZE`). This is mainly the size of the memory pages included in the checkpoint,
|
||||
but there is also information about the size of all changed files in the
|
||||
container (`ROOT FS DIFF SIZE`).
|
||||
-->
|
||||
这展示了有关该检查点归档中的检查点的一些信息。我们可以看到容器的名称、有关容器运行时和容器引擎的信息。它还列出了检查点的大小（`CHKPT SIZE`）。
|
||||
这主要是检查点中包含的内存页的大小，同时也有有关容器中所有更改文件的大小的信息（`ROOT FS DIFF SIZE`）。
|
||||
|
||||
<!--
|
||||
The additional parameter `--print-stats` decodes information in the checkpoint
|
||||
archive and displays them in the second table (*CRIU dump statistics*). This
|
||||
information is collected during checkpoint creation and gives an overview how much
|
||||
time CRIU needed to checkpoint the processes in the container and how many
|
||||
memory pages were analyzed and written during checkpoint creation.
|
||||
-->
|
||||
使用附加参数 `--print-stats` 可以解码检查点归档中的信息并将其显示在第二个表中（**CRIU 转储统计信息**）。
|
||||
此信息是在检查点创建期间收集的，并概述了 CRIU 对容器中的进程生成检查点所需的时间以及在检查点创建期间分析和写入了多少内存页。
|
||||
|
||||
<!--
|
||||
## Digging deeper
|
||||
|
||||
-->
|
||||
## 深入挖掘
|
||||
<!--
|
||||
With the help of `checkpointctl` I am able to get some high level information
|
||||
about the checkpoint archive. To be able to analyze the checkpoint archive
|
||||
further I have to extract it. The checkpoint archive is a *tar* archive and can
|
||||
|
|
@ -161,12 +211,30 @@ Extracting the checkpoint archive will result in following files and directories
|
|||
to display dump statistics (`--print-stats`)
|
||||
* `rootfs-diff.tar` - this file contains all changed files on the container's
|
||||
file-system
|
||||
-->
|
||||
借助 `checkpointctl`，我可以获得有关检查点归档的一些高级信息。为了能够进一步分析检查点归档，我必须将其提取。
|
||||
检查点归档是 **tar** 归档文件，可以借助 `tar xf checkpoint.tar` 进行解压。
|
||||
|
||||
展开检查点存档时，将创建以下文件和目录：
|
||||
|
||||
* `bind.mounts` - 该文件包含有关绑定挂载的信息，并且需要在恢复期间需要将所有外部文件和目录挂载到正确的位置。
|
||||
* `checkpoint/` - 该目录包含 CRIU 创建的实际检查点，
|
||||
* `config.dump` 和 `spec.dump` - 这些文件包含恢复期间所需的有关容器的元数据。
|
||||
* `dump.log` - 该文件包含在检查点期间创建的 CRIU 的调试输出。
|
||||
* `stats-dump` - 此文件包含 `checkpointctl` 用于通过 `--print-stats` 显示转储统计信息的数据。
|
||||
* `rootfs-diff.tar` - 该文件包含容器文件系统上所有已更改的文件。
|
||||
|
||||
<!--
|
||||
### File-system changes - `rootfs-diff.tar`
|
||||
-->
|
||||
### 更改文件系统 - `rootfs-diff.tar`
|
||||
|
||||
<!--
|
||||
The first step to analyze the container's checkpoint further is to look at
|
||||
the files that have changed in my container. This can be done by looking at the
|
||||
file `rootfs-diff.tar`:
|
||||
-->
|
||||
进一步分析容器检查点的第一步是查看容器内已更改的文件。这可以通过引用 `rootfs-diff.tar` 文件来完成。
|
||||
|
||||
```console
|
||||
$ tar xvf rootfs-diff.tar
|
||||
|
|
@ -174,7 +242,10 @@ home/counter/logfile
|
|||
home/counter/test-file
|
||||
```
|
||||
|
||||
<!--
|
||||
Now the files that changed in the container can be studied:
|
||||
-->
|
||||
现在你可以检查容器内已更改的文件。
|
||||
|
||||
```console
|
||||
$ cat home/counter/logfile
|
||||
|
|
@ -185,6 +256,7 @@ $ cat home/counter/test-file
|
|||
test-file
|
||||
```
|
||||
|
||||
<!--
|
||||
Compared to the container image (`quay.io/adrianreber/counter:blog`) this
|
||||
container is based on, I can see that the file `logfile` contains information
|
||||
about all access to the service the container provides and the file `test-file`
|
||||
|
|
@ -192,15 +264,28 @@ was created just as expected.
|
|||
|
||||
With the help of `rootfs-diff.tar` it is possible to inspect all files that
|
||||
were created or changed compared to the base image of the container.
|
||||
-->
|
||||
与该容器所基于的容器镜像（`quay.io/adrianreber/counter:blog`）相比，它包含容器提供的服务的所有访问信息以及预期创建的 `logfile` 可以检查 `test-file` 文件。
|
||||
|
||||
在 `rootfs-diff.tar` 的帮助下，可以根据容器的基本镜像检查所有创建或修改的文件。
|
||||
|
||||
<!--
|
||||
### Analyzing the checkpointed processes - `checkpoint/`
|
||||
-->
|
||||
### 分析检查点进程 - `checkpoint/`
|
||||
|
||||
<!--
|
||||
The directory `checkpoint/` contains data created by CRIU while checkpointing
|
||||
the processes in the container. The content in the directory `checkpoint/`
|
||||
consists of different [image files][image-files] which can be analyzed with the
|
||||
help of the tool [CRIT][crit] which is distributed as part of CRIU.
|
||||
|
||||
First lets get an overview of the processes inside of the container:
|
||||
-->
|
||||
目录 `checkpoint/` 包含 CRIU 在容器内对进程进行检查点时创建的数据。
|
||||
目录 `checkpoint/` 的内容由各种[镜像文件][image-files] 组成，可以使用作为 CRIU 一部分分发的 [CRIT][crit] 工具进行分析。
|
||||
|
||||
首先，我们先了解一下容器的内部流程。
|
||||
|
||||
```console
|
||||
$ crit show checkpoint/pstree.img | jq .entries[].pid
|
||||
|
|
@ -209,6 +294,7 @@ $ crit show checkpoint/pstree.img | jq .entries[].pid
|
|||
8
|
||||
```
|
||||
|
||||
<!--
|
||||
This output means that I have three processes inside of the container's PID
|
||||
namespace with the PIDs: 1, 7, 8
|
||||
|
||||
|
|
@ -217,6 +303,12 @@ restore exactly these PIDs will be recreated. From the outside of the
|
|||
container's PID namespace the PIDs will change after restore.
|
||||
|
||||
The next step is to get some additional information about these three processes:
|
||||
-->
|
||||
此输出意味着容器的 PID 命名空间内有 3 个进程（PID 为 1、7 和 8）。
|
||||
|
||||
这只是容器 PID 命名空间的内部视图。这些 PID 在恢复过程中会准确地重新创建。从容器的 PID 命名空间外部，PID 将在恢复后更改。
|
||||
|
||||
下一步是获取有关这三个进程的更多信息。
|
||||
|
||||
```console
|
||||
$ crit show checkpoint/core-1.img | jq .entries[0].tc.comm
|
||||
|
|
@ -227,11 +319,16 @@ $ crit show checkpoint/core-8.img | jq .entries[0].tc.comm
|
|||
"tee"
|
||||
```
|
||||
|
||||
<!--
|
||||
This means the three processes in my container are `bash`, `counter.py` (a Python
|
||||
interpreter) and `tee`. For details about the parent child relations of these processes there
|
||||
is more data to be analyzed in `checkpoint/pstree.img`.
|
||||
|
||||
Let's compare the so far collected information to the still running container:
|
||||
-->
|
||||
这意味着容器内的三个进程是 `bash`、`counter.py`（Python 解释器）和 `tee`。 `checkpoint/pstree.img` 中有更多数据可供分析，以获取有关进程起源的详细信息。
|
||||
|
||||
让我们将目前为止收集到的信息与仍在运行的容器进行比较。
|
||||
|
||||
```console
|
||||
$ crictl inspect --output go-template --template "{{(index .info.pid)}}" 059a219a22e56
|
||||
|
|
@ -248,10 +345,11 @@ $ cat /proc/722542/comm
|
|||
tee
|
||||
```
|
||||
|
||||
<!--
|
||||
In this output I am first retrieving the PID of the first process in the
|
||||
container and then I am looking for that PID and child processes on the system
|
||||
where the container is running. I am seeing three processes and the first one is
|
||||
"bash" which is PID 1 inside of the containers PID namespace. Then I am looking
|
||||
"bash" which is PID 1 inside of the containers PID namespace. Then I am looking
|
||||
at `/proc/<PID>/comm` and I can find the exact same value
|
||||
as in the checkpoint image.
|
||||
|
||||
|
|
@ -261,6 +359,13 @@ processes.
|
|||
|
||||
One last example of what `crit` can tell us about the container is the information
|
||||
about the UTS namespace:
|
||||
-->
|
||||
在此输出中，我们首先获取容器中第一个进程的 PID。在运行容器的系统上，它会查找其 PID 和子进程。你应该看到三个进程，第一个进程是 `bash`，容器 PID 命名空间中的 PID 为 1。
|
||||
然后查看 `/proc/<PID>/comm`，可以找到与检查点镜像完全相同的值。
|
||||
|
||||
需要记住的重点是，检查点包含容器的 PID 命名空间内的视图。因为这些信息对于恢复进程非常重要。
|
||||
|
||||
`crit` 告诉我们有关容器的最后一个例子是有关 UTS 命名空间的信息。
|
||||
|
||||
```console
|
||||
$ crit show checkpoint/utsns-12.img
|
||||
|
|
@ -275,42 +380,65 @@ $ crit show checkpoint/utsns-12.img
|
|||
}
|
||||
```
|
||||
|
||||
<!--
|
||||
This tells me that the hostname inside of the UTS namespace is `counters`.
|
||||
|
||||
For every resource CRIU collected during checkpointing the `checkpoint/`
|
||||
directory contains corresponding image files which can be analyzed with the help
|
||||
of `crit`.
|
||||
-->
|
||||
这里输出表示 UTS 命名空间中的主机名是 `counters`。
|
||||
|
||||
对于检查点创建期间收集的每个资源 CRIU，`checkpoint/` 目录包含相应的镜像文件。可以使用 `crit` 来分析该镜像文件。
|
||||
|
||||
<!--
|
||||
#### Looking at the memory pages
|
||||
-->
|
||||
#### 查看内存页面
|
||||
|
||||
<!--
|
||||
In addition to the information from CRIU that can be decoded with the help
|
||||
of CRIT, there are also files containing the raw memory pages written by
|
||||
CRIU to disk:
|
||||
-->
|
||||
除了可以借助 CRIT 解码的 CRIU 信息之外，还有包含 CRIU 写入磁盘的原始内存页的文件：
|
||||
|
||||
```console
|
||||
$ ls checkpoint/pages-*
|
||||
checkpoint/pages-1.img checkpoint/pages-2.img checkpoint/pages-3.img
|
||||
```
|
||||
|
||||
<!--
|
||||
When I initially used the container I stored a random key (`RANDOM_1432_KEY`)
|
||||
somewhere in the memory. Let see if I can find it:
|
||||
-->
|
||||
当我最初使用该容器时，我在内存中的某个位置存储了一个随机密钥。让我看看是否能找到它：
|
||||
|
||||
```console
|
||||
$ grep -ao RANDOM_1432_KEY checkpoint/pages-*
|
||||
checkpoint/pages-2.img:RANDOM_1432_KEY
|
||||
```
|
||||
|
||||
<!--
|
||||
And indeed, there is my data. This way I can easily look at the content
|
||||
of all memory pages of the processes in the container, but it is also
|
||||
important to remember that anyone that can access the checkpoint
|
||||
archive has access to all information that was stored in the memory of the
|
||||
container's processes.
|
||||
-->
|
||||
确实有我的数据。通过这种方式，我可以轻松查看容器中进程的所有内存页面的内容，但需要注意的是可以访问检查点存档的任何人都可以访问存储在容器进程内存中的所有信息。
|
||||
|
||||
<!--
|
||||
#### Using gdb for further analysis
|
||||
-->
|
||||
#### 使用 gdb 进行进一步分析
|
||||
|
||||
<!--
|
||||
Another possibility to look at the checkpoint images is `gdb`. The CRIU repository
|
||||
contains the script [coredump][criu-coredump] which can convert a checkpoint
|
||||
into a coredump file:
|
||||
-->
|
||||
查看检查点镜像的另一种方法是 `gdb`。CRIU 存储库包含脚本 [coredump][criu-coredump]，它可以将检查点转换为 coredump 文件：
|
||||
|
||||
```console
|
||||
$ /home/criu/coredump/coredump-python3
|
||||
|
|
@ -318,9 +446,12 @@ $ ls -al core*
|
|||
core.1 core.7 core.8
|
||||
```
|
||||
|
||||
<!--
|
||||
Running the `coredump-python3` script will convert the checkpoint images into
|
||||
one coredump file for each process in the container. Using `gdb` I can also look
|
||||
at the details of the processes:
|
||||
-->
|
||||
运行 `coredump-python3` 脚本会将检查点镜像转换为容器中每个进程一个的 coredump 文件。 使用 `gdb` 我还可以查看进程的详细信息：
|
||||
|
||||
```console
|
||||
$ echo info registers | gdb --core checkpoint/core.1 -q
|
||||
|
|
@ -361,8 +492,15 @@ In this example I can see the value of all registers as they were during
|
|||
checkpointing and I can also see the complete command-line of my container's PID
|
||||
1 process: `bash -c /home/counter/counter.py 2>&1 | tee /home/counter/logfile`
|
||||
|
||||
## Summary
|
||||
在这个例子中，我可以看到所有寄存器的值，因为它们在检查点，我还可以看到容器的 PID 1 进程的完整命令行：
|
||||
`bash -c /home/counter/counter.py 2>&1 | tee /home/counter/logfile`。
|
||||
|
||||
<!--
|
||||
## Summary
|
||||
-->
|
||||
## 总结
|
||||
|
||||
<!--
|
||||
With the help of container checkpointing, it is possible to create a
|
||||
checkpoint of a running container without stopping the container and without the
|
||||
container knowing that it was checkpointed. The result of checkpointing a
|
||||
|
|
@ -375,16 +513,27 @@ The different examples I have shown in this article how to analyze a checkpoint
|
|||
are just the starting point. Depending on your requirements it is possible to
|
||||
look at certain things in much more detail, but this article should give you an
|
||||
introduction how to start the analysis of your checkpoint.
|
||||
-->
|
||||
借助容器检查点，可以在不停止容器且在容器不知情的情况下，为正在运行的容器创建检查点。 在 Kubernetes 中对容器创建一个检查点的结果是检查点存档文件；
|
||||
使用不同的工具，如 `checkpointctl`、`tar`、`crit` 和 `gdb`，可以分析检查点。即使使用像 `grep` 这样的简单工具，也可以在检查点存档中找到信息。
|
||||
|
||||
我在本文中展示的如何分析检查点的不同示例，这只是一个起点。 根据你的需求，可以更详细地查看某些内容，本文向你介绍了如何开始进行检查点分析。
|
||||
|
||||
<!--
|
||||
## How do I get involved?
|
||||
-->
|
||||
## 如何参与？
|
||||
|
||||
<!--
|
||||
You can reach SIG Node by several means:
|
||||
-->
|
||||
你可以通过多种方式联系到 SIG Node。
|
||||
|
||||
* Slack: [#sig-node][slack-sig-node]
|
||||
* Slack: [#sig-security][slack-sig-security]
|
||||
* [Mailing list][sig-node-ml]
|
||||
* [邮件列表][sig-node-ml]
|
||||
|
||||
[forensic-blog]: https://kubernetes.io/blog/2022/12/05/forensic-container-checkpointing-alpha/
|
||||
[forensic-blog]: https://kubernetes.io/zh-cn/blog/2022/12/05/forensic-container-checkpointing-alpha/
|
||||
[checkpointctl]: https://github.com/checkpoint-restore/checkpointctl
|
||||
[image-files]: https://criu.org/Images
|
||||
[crit]: https://criu.org/CRIT
|
||||
|
|
|
|||
Loading…
Reference in New Issue