Merge pull request #31309 from neolit123/1.24-update-kubeadm-docs-for-dockershim
kubeadm: adapt docs for 1.24 and dockershim removalpull/31579/head
commit
5e35828808
|
@ -81,13 +81,8 @@ The user can skip specific preflight checks or all of them with the `--ignore-pr
|
|||
- Kubernetes system requirements:
|
||||
- if running on linux:
|
||||
- [error] if Kernel is older than the minimum required version
|
||||
- [error] if required cgroups subsystem aren't in set up
|
||||
- if using docker:
|
||||
- [warning/error] if Docker service does not exist, if it is disabled, if it is not active.
|
||||
- [error] if Docker endpoint does not exist or does not work
|
||||
- [warning] if docker version is not in the list of validated docker versions
|
||||
- If using other cri engine:
|
||||
- [error] if crictl socket does not answer
|
||||
- [error] if required cgroups subsystem aren't set up
|
||||
- [error] if the CRI endpoint does not answer
|
||||
- [error] if user is not root
|
||||
- [error] if the machine hostname is not a valid DNS subdomain
|
||||
- [warning] if the host name cannot be reached via network lookup
|
||||
|
@ -434,8 +429,7 @@ cluster startup problems.
|
|||
Please note that:
|
||||
|
||||
1. `kubeadm join` preflight checks are basically a subset `kubeadm init` preflight checks
|
||||
1. Starting from 1.9, kubeadm provides better support for CRI-generic functionality; in that case, docker specific controls
|
||||
are skipped or replaced by similar controls for crictl.
|
||||
1. Starting from 1.24, kubeadm uses crictl to communicate to all known CRI endpoints.
|
||||
1. Starting from 1.9, kubeadm provides support for joining nodes running on Windows; in that case, linux specific controls are skipped.
|
||||
1. In any case the user can skip specific preflight checks (or eventually all preflight checks) with the `--ignore-preflight-errors` option.
|
||||
|
||||
|
|
|
@ -111,10 +111,9 @@ for all control-plane nodes. Such an endpoint can be either a DNS name or an IP
|
|||
be passed to `kubeadm init`. Depending on which
|
||||
third-party provider you choose, you might need to set the `--pod-network-cidr` to
|
||||
a provider-specific value. See [Installing a Pod network add-on](#pod-network).
|
||||
1. (Optional) Since version 1.14, `kubeadm` tries to detect the container runtime on Linux
|
||||
by using a list of well known domain socket paths. To use different container runtime or
|
||||
if there are more than one installed on the provisioned node, specify the `--cri-socket`
|
||||
argument to `kubeadm init`. See
|
||||
1. (Optional) `kubeadm` tries to detect the container runtime by using a list of well
|
||||
known endpoints. To use different container runtime or if there are more than one installed
|
||||
on the provisioned node, specify the `--cri-socket` argument to `kubeadm`. See
|
||||
[Installing a runtime](/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#installing-runtime).
|
||||
1. (Optional) Unless otherwise specified, `kubeadm` uses the network interface associated
|
||||
with the default gateway to set the advertise address for this particular control-plane node's API server.
|
||||
|
@ -384,8 +383,8 @@ A few seconds later, you should notice this node in the output from `kubectl get
|
|||
nodes` when run on the control-plane node.
|
||||
|
||||
{{< note >}}
|
||||
As the cluster nodes are usually initialized sequentially, the CoreDNS Pods are likely to all run
|
||||
on the first control-plane node. To provide higher availability, please rebalance the CoreDNS Pods
|
||||
As the cluster nodes are usually initialized sequentially, the CoreDNS Pods are likely to all run
|
||||
on the first control-plane node. To provide higher availability, please rebalance the CoreDNS Pods
|
||||
with `kubectl -n kube-system rollout restart deployment coredns` after at least one new node is joined.
|
||||
{{< /note >}}
|
||||
|
||||
|
|
|
@ -79,51 +79,59 @@ The pod network plugin you use (see below) may also require certain ports to be
|
|||
open. Since this differs with each pod network plugin, please see the
|
||||
documentation for the plugins about what port(s) those need.
|
||||
|
||||
## Installing runtime {#installing-runtime}
|
||||
## Installing a container runtime {#installing-runtime}
|
||||
|
||||
To run containers in Pods, Kubernetes uses a
|
||||
{{< glossary_tooltip term_id="container-runtime" text="container runtime" >}}.
|
||||
|
||||
{{< tabs name="container_runtime" >}}
|
||||
{{% tab name="Linux nodes" %}}
|
||||
|
||||
By default, Kubernetes uses the
|
||||
{{< glossary_tooltip term_id="cri" text="Container Runtime Interface">}} (CRI)
|
||||
to interface with your chosen container runtime.
|
||||
|
||||
If you don't specify a runtime, kubeadm automatically tries to detect an installed
|
||||
container runtime by scanning through a list of well known Unix domain sockets.
|
||||
The following table lists container runtimes and their associated socket paths:
|
||||
container runtime by scanning through a list of known endpoints.
|
||||
|
||||
{{< table caption = "Container runtimes and their socket paths" >}}
|
||||
| Runtime | Path to Unix domain socket |
|
||||
|------------|-----------------------------------|
|
||||
| Docker | `/var/run/dockershim.sock` |
|
||||
| containerd | `/run/containerd/containerd.sock` |
|
||||
| CRI-O | `/var/run/crio/crio.sock` |
|
||||
If multiple or no container runtimes are detected kubeadm will throw an error
|
||||
and will request that you specify which one you want to use.
|
||||
|
||||
See [container runtimes](/docs/setup/production-environment/container-runtimes/)
|
||||
for more information.
|
||||
|
||||
{{< note >}}
|
||||
Docker Engine does not implement the [CRI](/docs/concepts/architecture/cri/)
|
||||
which is a requirement for a container runtime to work with Kubernetes.
|
||||
For that reason, an additional service [cri-dockerd](https://github.com/Mirantis/cri-dockerd)
|
||||
has to be installed. cri-dockerd is a project based on the legacy built-in
|
||||
Docker Engine support that was [removed](/dockershim) from the kubelet in version 1.24.
|
||||
{{< /note >}}
|
||||
|
||||
The tables below include the known endpoints for supported operating systems:
|
||||
|
||||
{{< tabs name="container_runtime" >}}
|
||||
{{% tab name="Linux" %}}
|
||||
|
||||
{{< table >}}
|
||||
| Runtime | Path to Unix domain socket |
|
||||
|------------------------------------|----------------------------------------------|
|
||||
| containerd | `unix:///var/run/containerd/containerd.sock` |
|
||||
| CRI-O | `unix:///var/run/crio/crio.sock` |
|
||||
| Docker Engine (using cri-dockerd) | `unix:///var/run/cri-dockerd.sock` |
|
||||
{{< /table >}}
|
||||
|
||||
<br />
|
||||
If both Docker and containerd are detected, Docker takes precedence. This is
|
||||
needed because Docker 18.09 ships with containerd and both are detectable even if you only
|
||||
installed Docker.
|
||||
If any other two or more runtimes are detected, kubeadm exits with an error.
|
||||
|
||||
The kubelet integrates with Docker through the built-in `dockershim` CRI implementation.
|
||||
|
||||
See [container runtimes](/docs/setup/production-environment/container-runtimes/)
|
||||
for more information.
|
||||
{{% /tab %}}
|
||||
{{% tab name="other operating systems" %}}
|
||||
By default, kubeadm uses {{< glossary_tooltip term_id="docker" >}} as the container runtime.
|
||||
The kubelet integrates with Docker through the built-in `dockershim` CRI implementation.
|
||||
|
||||
See [container runtimes](/docs/setup/production-environment/container-runtimes/)
|
||||
for more information.
|
||||
{{% tab name="Windows" %}}
|
||||
|
||||
{{< table >}}
|
||||
| Runtime | Path to Windows named pipe |
|
||||
|------------------------------------|----------------------------------------------|
|
||||
| containerd | `npipe:////./pipe/containerd-containerd` |
|
||||
| Docker Engine (using cri-dockerd) | `npipe:////./pipe/cri-dockerd` |
|
||||
{{< /table >}}
|
||||
|
||||
{{% /tab %}}
|
||||
{{< /tabs >}}
|
||||
|
||||
|
||||
## Installing kubeadm, kubelet and kubectl
|
||||
|
||||
You will install these packages on all of your machines:
|
||||
|
|
|
@ -36,7 +36,7 @@ using kubeadm, rather than managing the kubelet configuration for each Node manu
|
|||
### Propagating cluster-level configuration to each kubelet
|
||||
|
||||
You can provide the kubelet with default values to be used by `kubeadm init` and `kubeadm join`
|
||||
commands. Interesting examples include using a different CRI runtime or setting the default subnet
|
||||
commands. Interesting examples include using a different container runtime or setting the default subnet
|
||||
used by services.
|
||||
|
||||
If you want your services to use the subnet `10.96.0.0/12` as the default for services, you can pass
|
||||
|
@ -51,7 +51,7 @@ by the kubelet, using the `--cluster-dns` flag. This setting needs to be the sam
|
|||
on every manager and Node in the cluster. The kubelet provides a versioned, structured API object
|
||||
that can configure most parameters in the kubelet and push out this configuration to each running
|
||||
kubelet in the cluster. This object is called
|
||||
[`KubeletConfiguration`](/docs/reference/config-api/kubelet-config.v1beta1/).
|
||||
[`KubeletConfiguration`](/docs/reference/config-api/kubelet-config.v1beta1/).
|
||||
The `KubeletConfiguration` allows the user to specify flags such as the cluster DNS IP addresses expressed as
|
||||
a list of values to a camelCased key, illustrated by the following example:
|
||||
|
||||
|
@ -78,14 +78,12 @@ networking, or other host-specific parameters. The following list provides a few
|
|||
unless you are using a cloud provider. You can use the `--hostname-override` flag to override the
|
||||
default behavior if you need to specify a Node name different from the machine's hostname.
|
||||
|
||||
- Currently, the kubelet cannot automatically detect the cgroup driver used by the CRI runtime,
|
||||
but the value of `--cgroup-driver` must match the cgroup driver used by the CRI runtime to ensure
|
||||
- Currently, the kubelet cannot automatically detect the cgroup driver used by the container runtime,
|
||||
but the value of `--cgroup-driver` must match the cgroup driver used by the container runtime to ensure
|
||||
the health of the kubelet.
|
||||
|
||||
- Depending on the CRI runtime your cluster uses, you may need to specify different flags to the kubelet.
|
||||
For instance, when using Docker, you need to specify flags such as `--network-plugin=cni`, but if you
|
||||
are using an external runtime, you need to specify `--container-runtime=remote` and specify the CRI
|
||||
endpoint using the `--container-runtime-endpoint=<path>`.
|
||||
- To specify the container runtime you must set its endpoint with the
|
||||
`--container-runtime-endpoint=<path>` flag.
|
||||
|
||||
You can specify these flags by configuring an individual kubelet's configuration in your service manager,
|
||||
such as systemd.
|
||||
|
@ -123,7 +121,7 @@ KUBELET_KUBEADM_ARGS="--flag1=value1 --flag2=value2 ..."
|
|||
```
|
||||
|
||||
In addition to the flags used when starting the kubelet, the file also contains dynamic
|
||||
parameters such as the cgroup driver and whether to use a different CRI runtime socket
|
||||
parameters such as the cgroup driver and whether to use a different container runtime socket
|
||||
(`--cri-socket`).
|
||||
|
||||
After marshalling these two files to disk, kubeadm attempts to run the following two
|
||||
|
|
|
@ -97,7 +97,8 @@ and investigating each container by running `docker logs`. For other container r
|
|||
|
||||
## kubeadm blocks when removing managed containers
|
||||
|
||||
The following could happen if Docker halts and does not remove any Kubernetes-managed containers:
|
||||
The following could happen if the container runtime halts and does not remove
|
||||
any Kubernetes-managed containers:
|
||||
|
||||
```shell
|
||||
sudo kubeadm reset
|
||||
|
@ -111,18 +112,9 @@ sudo kubeadm reset
|
|||
(block)
|
||||
```
|
||||
|
||||
A possible solution is to restart the Docker service and then re-run `kubeadm reset`:
|
||||
|
||||
```shell
|
||||
sudo systemctl restart docker.service
|
||||
sudo kubeadm reset
|
||||
```
|
||||
|
||||
Inspecting the logs for docker may also be useful:
|
||||
|
||||
```shell
|
||||
journalctl -u docker
|
||||
```
|
||||
A possible solution is to restart the container runtime and then re-run `kubeadm reset`.
|
||||
You can also use `crictl` to debug the state of the container runtime. See
|
||||
[Debugging Kubernetes nodes with crictl](/docs/tasks/debug-application-cluster/crictl/).
|
||||
|
||||
## Pods in `RunContainerError`, `CrashLoopBackOff` or `Error` state
|
||||
|
||||
|
@ -136,10 +128,6 @@ Right after `kubeadm init` there should not be any pods in these states.
|
|||
it's very likely that the Pod Network add-on that you installed is somehow broken.
|
||||
You might have to grant it more RBAC privileges or use a newer version. Please file
|
||||
an issue in the Pod Network providers' issue tracker and get the issue triaged there.
|
||||
- If you install a version of Docker older than 1.12.1, remove the `MountFlags=slave` option
|
||||
when booting `dockerd` with `systemd` and restart `docker`. You can see the MountFlags in `/usr/lib/systemd/system/docker.service`.
|
||||
MountFlags can interfere with volumes mounted by Kubernetes, and put the Pods in `CrashLoopBackOff` state.
|
||||
The error happens when Kubernetes does not find `var/run/secrets/kubernetes.io/serviceaccount` files.
|
||||
|
||||
## `coredns` is stuck in the `Pending` state
|
||||
|
||||
|
|
|
@ -136,7 +136,7 @@ in the `flannel-host-gw.yml` or `flannel-overlay.yml` file and specify your inte
|
|||
curl -L https://github.com/kubernetes-sigs/sig-windows-tools/releases/latest/download/flannel-overlay.yml | sed 's/Ethernet/Ethernet0 2/g' | kubectl apply -f -
|
||||
```
|
||||
{{< /note >}}
|
||||
|
||||
|
||||
|
||||
|
||||
### Joining a Windows worker node
|
||||
|
@ -147,33 +147,7 @@ with elevated permissions (Administrator) on the Windows worker node.
|
|||
{{< /note >}}
|
||||
|
||||
{{< tabs name="tab-windows-kubeadm-runtime-installation" >}}
|
||||
{{% tab name="Docker EE" %}}
|
||||
|
||||
#### Install Docker EE
|
||||
|
||||
Install the `Containers` feature
|
||||
|
||||
```powershell
|
||||
Install-WindowsFeature -Name containers
|
||||
```
|
||||
|
||||
Install Docker
|
||||
Instructions to do so are available at [Install Docker Engine - Enterprise on Windows Servers](https://docs.microsoft.com/en-us/virtualization/windowscontainers/quick-start/set-up-environment?tabs=Windows-Server#install-docker).
|
||||
|
||||
#### Install wins, kubelet, and kubeadm
|
||||
|
||||
```PowerShell
|
||||
curl.exe -LO https://raw.githubusercontent.com/kubernetes-sigs/sig-windows-tools/master/kubeadm/scripts/PrepareNode.ps1
|
||||
.\PrepareNode.ps1 -KubernetesVersion {{< param "fullversion" >}}
|
||||
```
|
||||
|
||||
#### Run `kubeadm` to join the node
|
||||
|
||||
Use the command that was given to you when you ran `kubeadm init` on a control plane host.
|
||||
If you no longer have this command, or the token has expired, you can run `kubeadm token create --print-join-command`
|
||||
(on a control plane host) to generate a new token and join command.
|
||||
|
||||
{{% /tab %}}
|
||||
{{% tab name="CRI-containerD" %}}
|
||||
|
||||
#### Install containerD
|
||||
|
@ -191,9 +165,6 @@ To install a specific version of containerD specify the version with -ContainerD
|
|||
.\Install-Containerd.ps1 -ContainerDVersion 1.4.1
|
||||
```
|
||||
|
||||
{{< /note >}}
|
||||
|
||||
{{< note >}}
|
||||
If you're using a different interface rather than Ethernet (i.e. "Ethernet0 2") on the Windows nodes, specify the name with `-netAdapterName`.
|
||||
|
||||
```powershell
|
||||
|
@ -210,17 +181,59 @@ curl.exe -LO https://raw.githubusercontent.com/kubernetes-sigs/sig-windows-tools
|
|||
.\PrepareNode.ps1 -KubernetesVersion {{< param "fullversion" >}} -ContainerRuntime containerD
|
||||
```
|
||||
|
||||
[Install `crictl` from the cri-tools package](https://github.com/kubernetes-sigs/cri-tools)
|
||||
which is required so that kubeadm can talk to the CRI endpoint.
|
||||
|
||||
#### Run `kubeadm` to join the node
|
||||
|
||||
Use the command that was given to you when you ran `kubeadm init` on a control plane host.
|
||||
If you no longer have this command, or the token has expired, you can run `kubeadm token create --print-join-command`
|
||||
(on a control plane host) to generate a new token and join command.
|
||||
|
||||
{{% /tab %}}
|
||||
|
||||
{{% tab name="Docker Engine" %}}
|
||||
|
||||
#### Install Docker Engine
|
||||
|
||||
Install the `Containers` feature
|
||||
|
||||
```powershell
|
||||
Install-WindowsFeature -Name containers
|
||||
```
|
||||
|
||||
Install Docker
|
||||
Instructions to do so are available at [Install Docker Engine - Enterprise on Windows Servers](https://docs.microsoft.com/en-us/virtualization/windowscontainers/quick-start/set-up-environment?tabs=Windows-Server#install-docker).
|
||||
|
||||
[Install cri-dockerd](https://github.com/Mirantis/cri-dockerd) which is required so that the kubelet
|
||||
can communicate with Docker on a CRI compatible endpoint.
|
||||
|
||||
{{< note >}}
|
||||
If using **CRI-containerD** add `--cri-socket "npipe:////./pipe/containerd-containerd"` to the kubeadm call
|
||||
Docker Engine does not implement the [CRI](/docs/concepts/architecture/cri/)
|
||||
which is a requirement for a container runtime to work with Kubernetes.
|
||||
For that reason, an additional service [cri-dockerd](https://github.com/Mirantis/cri-dockerd)
|
||||
has to be installed. cri-dockerd is a project based on the legacy built-in
|
||||
Docker Engine support that was [removed](/dockershim) from the kubelet in version 1.24.
|
||||
{{< /note >}}
|
||||
|
||||
Install `crictl` from the [cri-tools project](https://github.com/kubernetes-sigs/cri-tools)
|
||||
which is required so that kubeadm can talk to the CRI endpoint.
|
||||
|
||||
#### Install wins, kubelet, and kubeadm
|
||||
|
||||
```PowerShell
|
||||
curl.exe -LO https://raw.githubusercontent.com/kubernetes-sigs/sig-windows-tools/master/kubeadm/scripts/PrepareNode.ps1
|
||||
.\PrepareNode.ps1 -KubernetesVersion {{< param "fullversion" >}}
|
||||
```
|
||||
|
||||
#### Run `kubeadm` to join the node
|
||||
|
||||
Use the command that was given to you when you ran `kubeadm init` on a control plane host.
|
||||
If you no longer have this command, or the token has expired, you can run `kubeadm token create --print-join-command`
|
||||
(on a control plane host) to generate a new token and join command.
|
||||
|
||||
{{% /tab %}}
|
||||
|
||||
{{< /tabs >}}
|
||||
|
||||
### Verifying your installation
|
||||
|
|
Loading…
Reference in New Issue