Improved the grammar and wording for the whole page.
parent
14e613de42
commit
dd225e174d
|
@ -11,12 +11,12 @@ weight: 100
|
||||||
|
|
||||||
{{< feature-state for_k8s_version="v1.10" state="stable" >}}
|
{{< feature-state for_k8s_version="v1.10" state="stable" >}}
|
||||||
|
|
||||||
`kubeadm init` and `kubeadm join` together provides a nice user experience for creating a
|
`kubeadm init` and `kubeadm join` together provide a nice user experience for creating a
|
||||||
best-practice but bare Kubernetes cluster from scratch.
|
bare Kubernetes cluster from scratch, that aligns with the best-practices.
|
||||||
However, it might not be obvious _how_ kubeadm does that.
|
However, it might not be obvious _how_ kubeadm does that.
|
||||||
|
|
||||||
This document provides additional details on what happen under the hood, with the aim of sharing
|
This document provides additional details on what happens under the hood, with the aim of sharing
|
||||||
knowledge on Kubernetes cluster best practices.
|
knowledge on the best practices for a Kubernetes cluster.
|
||||||
|
|
||||||
<!-- body -->
|
<!-- body -->
|
||||||
|
|
||||||
|
@ -47,9 +47,9 @@ In order to reduce complexity and to simplify development of higher level tools
|
||||||
limited set of constant values for well-known paths and file names.
|
limited set of constant values for well-known paths and file names.
|
||||||
|
|
||||||
The Kubernetes directory `/etc/kubernetes` is a constant in the application, since it is clearly the given path
|
The Kubernetes directory `/etc/kubernetes` is a constant in the application, since it is clearly the given path
|
||||||
in a majority of cases, and the most intuitive location; other constants paths and file names are:
|
in a majority of cases, and the most intuitive location; other constant paths and file names are:
|
||||||
|
|
||||||
- `/etc/kubernetes/manifests` as the path where kubelet should look for static Pod manifests.
|
- `/etc/kubernetes/manifests` as the path where the kubelet should look for static Pod manifests.
|
||||||
Names of static Pod manifests are:
|
Names of static Pod manifests are:
|
||||||
|
|
||||||
- `etcd.yaml`
|
- `etcd.yaml`
|
||||||
|
@ -78,9 +78,8 @@ in a majority of cases, and the most intuitive location; other constants paths a
|
||||||
|
|
||||||
## kubeadm init workflow internal design
|
## kubeadm init workflow internal design
|
||||||
|
|
||||||
The `kubeadm init` [internal workflow](/docs/reference/setup-tools/kubeadm/kubeadm-init/#init-workflow)
|
The `kubeadm init` consists of a sequence of atomic work tasks to perform,
|
||||||
consists of a sequence of atomic work tasks to perform,
|
as described in the `kubeadm init` [internal workflow](/docs/reference/setup-tools/kubeadm/kubeadm-init/#init-workflow).
|
||||||
as described in `kubeadm init`.
|
|
||||||
|
|
||||||
The [`kubeadm init phase`](/docs/reference/setup-tools/kubeadm/kubeadm-init-phase/) command allows
|
The [`kubeadm init phase`](/docs/reference/setup-tools/kubeadm/kubeadm-init-phase/) command allows
|
||||||
users to invoke each task individually, and ultimately offers a reusable and composable
|
users to invoke each task individually, and ultimately offers a reusable and composable
|
||||||
|
@ -142,7 +141,7 @@ Kubeadm generates certificate and private key pairs for different purposes:
|
||||||
|
|
||||||
- A serving certificate for the API server, generated using `ca.crt` as the CA, and saved into
|
- A serving certificate for the API server, generated using `ca.crt` as the CA, and saved into
|
||||||
`apiserver.crt` file with its private key `apiserver.key`. This certificate should contain
|
`apiserver.crt` file with its private key `apiserver.key`. This certificate should contain
|
||||||
following alternative names:
|
the following alternative names:
|
||||||
|
|
||||||
- The Kubernetes service's internal clusterIP (the first address in the services CIDR, e.g.
|
- The Kubernetes service's internal clusterIP (the first address in the services CIDR, e.g.
|
||||||
`10.96.0.1` if service subnet is `10.96.0.0/12`)
|
`10.96.0.1` if service subnet is `10.96.0.0/12`)
|
||||||
|
@ -163,7 +162,7 @@ Kubeadm generates certificate and private key pairs for different purposes:
|
||||||
- A certificate authority for the front proxy saved into `front-proxy-ca.crt` file with its key
|
- A certificate authority for the front proxy saved into `front-proxy-ca.crt` file with its key
|
||||||
`front-proxy-ca.key`
|
`front-proxy-ca.key`
|
||||||
|
|
||||||
- A client cert for the front proxy client, generate using `front-proxy-ca.crt` as the CA and
|
- A client certificate for the front proxy client, generated using `front-proxy-ca.crt` as the CA and
|
||||||
saved into `front-proxy-client.crt` file with its private key`front-proxy-client.key`
|
saved into `front-proxy-client.crt` file with its private key`front-proxy-client.key`
|
||||||
|
|
||||||
Certificates are stored by default in `/etc/kubernetes/pki`, but this directory is configurable
|
Certificates are stored by default in `/etc/kubernetes/pki`, but this directory is configurable
|
||||||
|
@ -171,16 +170,16 @@ using the `--cert-dir` flag.
|
||||||
|
|
||||||
Please note that:
|
Please note that:
|
||||||
|
|
||||||
1. If a given certificate and private key pair both exist, and its content is evaluated compliant with the above specs, the existing files will
|
1. If a given certificate and private key pair both exist, and their content is evaluated to be compliant with the above specs, the existing files will
|
||||||
be used and the generation phase for the given certificate skipped. This means the user can, for example, copy an existing CA to
|
be used and the generation phase for the given certificate will be skipped. This means the user can, for example, copy an existing CA to
|
||||||
`/etc/kubernetes/pki/ca.{crt,key}`, and then kubeadm will use those files for signing the rest of the certs.
|
`/etc/kubernetes/pki/ca.{crt,key}`, and then kubeadm will use those files for signing the rest of the certs.
|
||||||
See also [using custom certificates](/docs/tasks/administer-cluster/kubeadm/kubeadm-certs#custom-certificates)
|
See also [using custom certificates](/docs/tasks/administer-cluster/kubeadm/kubeadm-certs#custom-certificates)
|
||||||
1. Only for the CA, it is possible to provide the `ca.crt` file but not the `ca.key` file, if all other certificates and kubeconfig files
|
1. For the CA, it is possible to provide the `ca.crt` file but not the `ca.key` file. If all other certificates and kubeconfig files
|
||||||
already are in place kubeadm recognize this condition and activates the ExternalCA, which also implies the `csrsigner`controller in
|
are already in place, kubeadm recognizes this condition and activates the ExternalCA, which also implies the `csrsigner` controller in
|
||||||
controller-manager won't be started
|
controller-manager won't be started
|
||||||
1. If kubeadm is running in [external CA mode](/docs/tasks/administer-cluster/kubeadm/kubeadm-certs#external-ca-mode);
|
1. If kubeadm is running in [external CA mode](/docs/tasks/administer-cluster/kubeadm/kubeadm-certs#external-ca-mode);
|
||||||
all the certificates must be provided by the user, because kubeadm cannot generate them by itself
|
all the certificates must be provided by the user, because kubeadm cannot generate them by itself
|
||||||
1. In case of kubeadm is executed in the `--dry-run` mode, certificates files are written in a temporary folder
|
1. In case kubeadm is executed in the `--dry-run` mode, certificate files are written in a temporary folder
|
||||||
1. Certificate generation can be invoked individually with the
|
1. Certificate generation can be invoked individually with the
|
||||||
[`kubeadm init phase certs all`](/docs/reference/setup-tools/kubeadm/kubeadm-init-phase/#cmd-phase-certs) command
|
[`kubeadm init phase certs all`](/docs/reference/setup-tools/kubeadm/kubeadm-init-phase/#cmd-phase-certs) command
|
||||||
|
|
||||||
|
@ -189,23 +188,23 @@ Please note that:
|
||||||
Kubeadm generates kubeconfig files with identities for control plane components:
|
Kubeadm generates kubeconfig files with identities for control plane components:
|
||||||
|
|
||||||
- A kubeconfig file for the kubelet to use during TLS bootstrap -
|
- A kubeconfig file for the kubelet to use during TLS bootstrap -
|
||||||
/etc/kubernetes/bootstrap-kubelet.conf. Inside this file there is a bootstrap-token or embedded
|
/etc/kubernetes/bootstrap-kubelet.conf. Inside this file, there is a bootstrap-token or embedded
|
||||||
client certificates for authenticating this node with the cluster.
|
client certificates for authenticating this node with the cluster.
|
||||||
|
|
||||||
This client cert should:
|
This client certificate should:
|
||||||
|
|
||||||
- Be in the `system:nodes` organization, as required by the
|
- Be in the `system:nodes` organization, as required by the
|
||||||
[Node Authorization](/docs/reference/access-authn-authz/node/) module
|
[Node Authorization](/docs/reference/access-authn-authz/node/) module
|
||||||
- Have the Common Name (CN) `system:node:<hostname-lowercased>`
|
- Have the Common Name (CN) `system:node:<hostname-lowercased>`
|
||||||
|
|
||||||
- A kubeconfig file for controller-manager, `/etc/kubernetes/controller-manager.conf`; inside this
|
- A kubeconfig file for controller-manager, `/etc/kubernetes/controller-manager.conf`; inside this
|
||||||
file is embedded a client certificate with controller-manager identity. This client cert should
|
file is embedded a client certificate with controller-manager identity. This client certificate should
|
||||||
have the CN `system:kube-controller-manager`, as defined by default
|
have the CN `system:kube-controller-manager`, as defined by default
|
||||||
[RBAC core components roles](/docs/reference/access-authn-authz/rbac/#core-component-roles)
|
[RBAC core components roles](/docs/reference/access-authn-authz/rbac/#core-component-roles)
|
||||||
|
|
||||||
- A kubeconfig file for scheduler, `/etc/kubernetes/scheduler.conf`; inside this file is embedded
|
- A kubeconfig file for scheduler, `/etc/kubernetes/scheduler.conf`; inside this file is embedded
|
||||||
a client certificate with scheduler identity.
|
a client certificate with scheduler identity.
|
||||||
This client cert should have the CN `system:kube-scheduler`, as defined by default
|
This client certificate should have the CN `system:kube-scheduler`, as defined by default
|
||||||
[RBAC core components roles](/docs/reference/access-authn-authz/rbac/#core-component-roles)
|
[RBAC core components roles](/docs/reference/access-authn-authz/rbac/#core-component-roles)
|
||||||
|
|
||||||
Additionally, a kubeconfig file for kubeadm as an administrative entity is generated and stored
|
Additionally, a kubeconfig file for kubeadm as an administrative entity is generated and stored
|
||||||
|
@ -213,35 +212,35 @@ in `/etc/kubernetes/admin.conf`. This file includes a certificate with
|
||||||
`Subject: O = kubeadm:cluster-admins, CN = kubernetes-admin`. `kubeadm:cluster-admins`
|
`Subject: O = kubeadm:cluster-admins, CN = kubernetes-admin`. `kubeadm:cluster-admins`
|
||||||
is a group managed by kubeadm. It is bound to the `cluster-admin` ClusterRole during `kubeadm init`,
|
is a group managed by kubeadm. It is bound to the `cluster-admin` ClusterRole during `kubeadm init`,
|
||||||
by using the `super-admin.conf` file, which does not require RBAC.
|
by using the `super-admin.conf` file, which does not require RBAC.
|
||||||
This `admin.conf` file must remain on control plane nodes and not be shared with additional users.
|
This `admin.conf` file must remain on control plane nodes and should not be shared with additional users.
|
||||||
|
|
||||||
During `kubeadm init` another kubeconfig file is generated and stored in `/etc/kubernetes/super-admin.conf`.
|
During `kubeadm init` another kubeconfig file is generated and stored in `/etc/kubernetes/super-admin.conf`.
|
||||||
This file includes a certificate with `Subject: O = system:masters, CN = kubernetes-super-admin`.
|
This file includes a certificate with `Subject: O = system:masters, CN = kubernetes-super-admin`.
|
||||||
`system:masters` is a super user group that bypasses RBAC and makes `super-admin.conf` useful in case
|
`system:masters` is a superuser group that bypasses RBAC and makes `super-admin.conf` useful in case
|
||||||
of an emergency where a cluster is locked due to RBAC misconfiguration.
|
of an emergency where a cluster is locked due to RBAC misconfiguration.
|
||||||
The `super-admin.conf` file can be stored in a safe location and not shared with additional users.
|
The `super-admin.conf` file must be stored in a safe location and should not be shared with additional users.
|
||||||
|
|
||||||
See [RBAC user facing role bindings](/docs/reference/access-authn-authz/rbac/#user-facing-roles)
|
See [RBAC user facing role bindings](/docs/reference/access-authn-authz/rbac/#user-facing-roles)
|
||||||
for additional information RBAC and built-in ClusterRoles and groups.
|
for additional information on RBAC and built-in ClusterRoles and groups.
|
||||||
|
|
||||||
Please note that:
|
Please note that:
|
||||||
|
|
||||||
1. `ca.crt` certificate is embedded in all the kubeconfig files.
|
1. `ca.crt` certificate is embedded in all the kubeconfig files.
|
||||||
1. If a given kubeconfig file exists, and its content is evaluated compliant with the above specs,
|
1. If a given kubeconfig file exists, and its content is evaluated as compliant with the above specs,
|
||||||
the existing file will be used and the generation phase for the given kubeconfig skipped
|
the existing file will be used and the generation phase for the given kubeconfig will be skipped
|
||||||
1. If kubeadm is running in [ExternalCA mode](/docs/reference/setup-tools/kubeadm/kubeadm-init/#external-ca-mode),
|
1. If kubeadm is running in [ExternalCA mode](/docs/reference/setup-tools/kubeadm/kubeadm-init/#external-ca-mode),
|
||||||
all the required kubeconfig must be provided by the user as well, because kubeadm cannot
|
all the required kubeconfig must be provided by the user as well, because kubeadm cannot
|
||||||
generate any of them by itself
|
generate any of them by itself
|
||||||
1. In case of kubeadm is executed in the `--dry-run` mode, kubeconfig files are written in a temporary folder
|
1. In case kubeadm is executed in the `--dry-run` mode, kubeconfig files are written in a temporary folder
|
||||||
1. Kubeconfig files generation can be invoked individually with the
|
1. Generation of kubeconfig files can be invoked individually with the
|
||||||
[`kubeadm init phase kubeconfig all`](/docs/reference/setup-tools/kubeadm/kubeadm-init-phase/#cmd-phase-kubeconfig) command
|
[`kubeadm init phase kubeconfig all`](/docs/reference/setup-tools/kubeadm/kubeadm-init-phase/#cmd-phase-kubeconfig) command
|
||||||
|
|
||||||
### Generate static Pod manifests for control plane components
|
### Generate static Pod manifests for control plane components
|
||||||
|
|
||||||
Kubeadm writes static Pod manifest files for control plane components to
|
Kubeadm writes static Pod manifest files for control plane components to
|
||||||
`/etc/kubernetes/manifests`. The kubelet watches this directory for Pods to create on startup.
|
`/etc/kubernetes/manifests`. The kubelet watches this directory for Pods to be created on startup.
|
||||||
|
|
||||||
Static Pod manifest share a set of common properties:
|
Static Pod manifests share a set of common properties:
|
||||||
|
|
||||||
- All static Pods are deployed on `kube-system` namespace
|
- All static Pods are deployed on `kube-system` namespace
|
||||||
- All static Pods get `tier:control-plane` and `component:{component-name}` labels
|
- All static Pods get `tier:control-plane` and `component:{component-name}` labels
|
||||||
|
@ -249,8 +248,8 @@ Static Pod manifest share a set of common properties:
|
||||||
- `hostNetwork: true` is set on all static Pods to allow control plane startup before a network is
|
- `hostNetwork: true` is set on all static Pods to allow control plane startup before a network is
|
||||||
configured; as a consequence:
|
configured; as a consequence:
|
||||||
|
|
||||||
* The `address` that the controller-manager and the scheduler use to refer the API server is `127.0.0.1`
|
* The `address` that the controller-manager and the scheduler use to refer to the API server is `127.0.0.1`
|
||||||
* If using a local etcd server, `etcd-servers` address will be set to `127.0.0.1:2379`
|
* If the etcd server is set up locally, the `etcd-server` address will be set to `127.0.0.1:2379`
|
||||||
|
|
||||||
- Leader election is enabled for both the controller-manager and the scheduler
|
- Leader election is enabled for both the controller-manager and the scheduler
|
||||||
- Controller-manager and the scheduler will reference kubeconfig files with their respective, unique identities
|
- Controller-manager and the scheduler will reference kubeconfig files with their respective, unique identities
|
||||||
|
@ -263,22 +262,22 @@ Please note that:
|
||||||
1. All images will be pulled from registry.k8s.io by default.
|
1. All images will be pulled from registry.k8s.io by default.
|
||||||
See [using custom images](/docs/reference/setup-tools/kubeadm/kubeadm-init/#custom-images)
|
See [using custom images](/docs/reference/setup-tools/kubeadm/kubeadm-init/#custom-images)
|
||||||
for customizing the image repository
|
for customizing the image repository
|
||||||
1. In case of kubeadm is executed in the `--dry-run` mode, static Pods files are written in a
|
1. In case kubeadm is executed in the `--dry-run` mode, static Pod files are written in a
|
||||||
temporary folder
|
temporary folder
|
||||||
1. Static Pod manifest generation for control plane components can be invoked individually with
|
1. Static Pod manifest generation for control plane components can be invoked individually with
|
||||||
the [`kubeadm init phase control-plane all`](/docs/reference/setup-tools/kubeadm/kubeadm-init-phase/#cmd-phase-control-plane) command
|
the [`kubeadm init phase control-plane all`](/docs/reference/setup-tools/kubeadm/kubeadm-init-phase/#cmd-phase-control-plane) command
|
||||||
|
|
||||||
#### API server
|
#### API server
|
||||||
|
|
||||||
The static Pod manifest for the API server is affected by following parameters provided by the users:
|
The static Pod manifest for the API server is affected by the following parameters provided by the users:
|
||||||
|
|
||||||
- The `apiserver-advertise-address` and `apiserver-bind-port` to bind to; if not provided, those
|
- The `apiserver-advertise-address` and `apiserver-bind-port` to bind to; if not provided, those
|
||||||
value defaults to the IP address of the default network interface on the machine and port 6443
|
values default to the IP address of the default network interface on the machine and port 6443
|
||||||
- The `service-cluster-ip-range` to use for services
|
- The `service-cluster-ip-range` to use for services
|
||||||
- If an external etcd server is specified, the `etcd-servers` address and related TLS settings
|
- If an external etcd server is specified, the `etcd-servers` address and related TLS settings
|
||||||
(`etcd-cafile`, `etcd-certfile`, `etcd-keyfile`);
|
(`etcd-cafile`, `etcd-certfile`, `etcd-keyfile`);
|
||||||
if an external etcd server is not be provided, a local etcd will be used (via host network)
|
if an external etcd server is not provided, a local etcd will be used (via host network)
|
||||||
- If a cloud provider is specified, the corresponding `--cloud-provider` is configured, together
|
- If a cloud provider is specified, the corresponding `--cloud-provider` parameter is configured together
|
||||||
with the `--cloud-config` path if such file exists (this is experimental, alpha and will be
|
with the `--cloud-config` path if such file exists (this is experimental, alpha and will be
|
||||||
removed in a future version)
|
removed in a future version)
|
||||||
|
|
||||||
|
@ -345,7 +344,7 @@ the users:
|
||||||
- `--allocate-node-cidrs=true`
|
- `--allocate-node-cidrs=true`
|
||||||
- `--cluster-cidr` and `--node-cidr-mask-size` flags according to the given CIDR
|
- `--cluster-cidr` and `--node-cidr-mask-size` flags according to the given CIDR
|
||||||
|
|
||||||
- If a cloud provider is specified, the corresponding `--cloud-provider` is specified, together
|
- If a cloud provider is specified, the corresponding `--cloud-provider` is specified together
|
||||||
with the `--cloud-config` path if such configuration file exists (this is experimental, alpha
|
with the `--cloud-config` path if such configuration file exists (this is experimental, alpha
|
||||||
and will be removed in a future version)
|
and will be removed in a future version)
|
||||||
|
|
||||||
|
@ -370,7 +369,7 @@ The static Pod manifest for the scheduler is not affected by parameters provided
|
||||||
|
|
||||||
### Generate static Pod manifest for local etcd
|
### Generate static Pod manifest for local etcd
|
||||||
|
|
||||||
If you specified an external etcd this step will be skipped, otherwise kubeadm generates a
|
If you specified an external etcd, this step will be skipped, otherwise kubeadm generates a
|
||||||
static Pod manifest file for creating a local etcd instance running in a Pod with following attributes:
|
static Pod manifest file for creating a local etcd instance running in a Pod with following attributes:
|
||||||
|
|
||||||
- listen on `localhost:2379` and use `HostNetwork=true`
|
- listen on `localhost:2379` and use `HostNetwork=true`
|
||||||
|
@ -391,7 +390,7 @@ Please note that:
|
||||||
### Wait for the control plane to come up
|
### Wait for the control plane to come up
|
||||||
|
|
||||||
kubeadm waits (upto 4m0s) until `localhost:6443/healthz` (kube-apiserver liveness) returns `ok`.
|
kubeadm waits (upto 4m0s) until `localhost:6443/healthz` (kube-apiserver liveness) returns `ok`.
|
||||||
However in order to detect deadlock conditions, kubeadm fails fast if `localhost:10255/healthz`
|
However, in order to detect deadlock conditions, kubeadm fails fast if `localhost:10255/healthz`
|
||||||
(kubelet liveness) or `localhost:10255/healthz/syncloop` (kubelet readiness) don't return `ok`
|
(kubelet liveness) or `localhost:10255/healthz/syncloop` (kubelet readiness) don't return `ok`
|
||||||
within 40s and 60s respectively.
|
within 40s and 60s respectively.
|
||||||
|
|
||||||
|
@ -414,7 +413,7 @@ Please note that:
|
||||||
|
|
||||||
### Mark the node as control-plane
|
### Mark the node as control-plane
|
||||||
|
|
||||||
As soon as the control plane is available, kubeadm executes following actions:
|
As soon as the control plane is available, kubeadm executes the following actions:
|
||||||
|
|
||||||
- Labels the node as control-plane with `node-role.kubernetes.io/control-plane=""`
|
- Labels the node as control-plane with `node-role.kubernetes.io/control-plane=""`
|
||||||
- Taints the node with `node-role.kubernetes.io/control-plane:NoSchedule`
|
- Taints the node with `node-role.kubernetes.io/control-plane:NoSchedule`
|
||||||
|
@ -422,14 +421,6 @@ As soon as the control plane is available, kubeadm executes following actions:
|
||||||
Please note that the phase to mark the control-plane phase can be invoked
|
Please note that the phase to mark the control-plane phase can be invoked
|
||||||
individually with the [`kubeadm init phase mark-control-plane`](/docs/reference/setup-tools/kubeadm/kubeadm-init-phase/#cmd-phase-mark-control-plane) command.
|
individually with the [`kubeadm init phase mark-control-plane`](/docs/reference/setup-tools/kubeadm/kubeadm-init-phase/#cmd-phase-mark-control-plane) command.
|
||||||
|
|
||||||
- Taints the node with `node-role.kubernetes.io/master:NoSchedule` and
|
|
||||||
`node-role.kubernetes.io/control-plane:NoSchedule`
|
|
||||||
|
|
||||||
Please note that:
|
|
||||||
|
|
||||||
1. The `node-role.kubernetes.io/master` taint is deprecated and will be removed in kubeadm version 1.25
|
|
||||||
1. Mark control-plane phase can be invoked individually with the command
|
|
||||||
[`kubeadm init phase mark-control-plane`](/docs/reference/setup-tools/kubeadm/kubeadm-init-phase/#cmd-phase-mark-control-plane)
|
|
||||||
|
|
||||||
### Configure TLS-Bootstrapping for node joining
|
### Configure TLS-Bootstrapping for node joining
|
||||||
|
|
||||||
|
@ -450,9 +441,9 @@ alternatively, each step can be invoked individually.
|
||||||
|
|
||||||
#### Create a bootstrap token
|
#### Create a bootstrap token
|
||||||
|
|
||||||
`kubeadm init` create a first bootstrap token, either generated automatically or provided by the
|
`kubeadm init` creates a first bootstrap token, either generated automatically or provided by the
|
||||||
user with the `--token` flag; as documented in bootstrap token specification, token should be
|
user with the `--token` flag; as documented in bootstrap token specification, token should be
|
||||||
saved as secrets with name `bootstrap-token-<token-id>` under `kube-system` namespace.
|
saved as a secret with name `bootstrap-token-<token-id>` under `kube-system` namespace.
|
||||||
|
|
||||||
Please note that:
|
Please note that:
|
||||||
|
|
||||||
|
@ -461,7 +452,7 @@ Please note that:
|
||||||
`system:bootstrappers:kubeadm:default-node-token` group
|
`system:bootstrappers:kubeadm:default-node-token` group
|
||||||
1. The token has a limited validity, default 24 hours (the interval may be changed with the `—token-ttl` flag)
|
1. The token has a limited validity, default 24 hours (the interval may be changed with the `—token-ttl` flag)
|
||||||
1. Additional tokens can be created with the [`kubeadm token`](/docs/reference/setup-tools/kubeadm/kubeadm-token/)
|
1. Additional tokens can be created with the [`kubeadm token`](/docs/reference/setup-tools/kubeadm/kubeadm-token/)
|
||||||
command, that provide as well other useful functions for token management.
|
command, that provide other useful functions for token management as well.
|
||||||
|
|
||||||
#### Allow joining nodes to call CSR API
|
#### Allow joining nodes to call CSR API
|
||||||
|
|
||||||
|
@ -486,7 +477,7 @@ well, granting POST permission to
|
||||||
|
|
||||||
#### Set up nodes certificate rotation with auto approval
|
#### Set up nodes certificate rotation with auto approval
|
||||||
|
|
||||||
Kubeadm ensures that certificate rotation is enabled for nodes, and that new certificate request
|
Kubeadm ensures that certificate rotation is enabled for nodes, and that a new certificate request
|
||||||
for nodes will get its CSR request automatically approved by the csrapprover controller.
|
for nodes will get its CSR request automatically approved by the csrapprover controller.
|
||||||
|
|
||||||
This is implemented by creating ClusterRoleBinding named
|
This is implemented by creating ClusterRoleBinding named
|
||||||
|
@ -497,14 +488,14 @@ role `system:certificates.k8s.io:certificatesigningrequests:selfnodeclient`.
|
||||||
|
|
||||||
This phase creates the `cluster-info` ConfigMap in the `kube-public` namespace.
|
This phase creates the `cluster-info` ConfigMap in the `kube-public` namespace.
|
||||||
|
|
||||||
Additionally it creates a Role and a RoleBinding granting access to the ConfigMap for
|
Additionally, it creates a Role and a RoleBinding granting access to the ConfigMap for
|
||||||
unauthenticated users (i.e. users in RBAC group `system:unauthenticated`).
|
unauthenticated users (i.e. users in RBAC group `system:unauthenticated`).
|
||||||
|
|
||||||
{{< note >}}
|
{{< note >}}
|
||||||
The access to the `cluster-info` ConfigMap _is not_ rate-limited. This may or may not be a
|
The access to the `cluster-info` ConfigMap _is not_ rate-limited. This may or may not be a
|
||||||
problem if you expose your cluster's API server to the internet; worst-case scenario here is a
|
problem if you expose your cluster's API server to the internet; worst-case scenario here is a
|
||||||
DoS attack where an attacker uses all the in-flight requests the kube-apiserver can handle to
|
DoS attack where an attacker uses all the in-flight requests the kube-apiserver can handle to
|
||||||
serving the `cluster-info` ConfigMap.
|
serve the `cluster-info` ConfigMap.
|
||||||
{{< /note >}}
|
{{< /note >}}
|
||||||
|
|
||||||
### Install addons
|
### Install addons
|
||||||
|
@ -529,7 +520,7 @@ deployed as a DaemonSet:
|
||||||
|
|
||||||
- The CoreDNS service is named `kube-dns`. This is done to prevent any interruption
|
- The CoreDNS service is named `kube-dns`. This is done to prevent any interruption
|
||||||
in service when the user is switching the cluster DNS from kube-dns to CoreDNS
|
in service when the user is switching the cluster DNS from kube-dns to CoreDNS
|
||||||
the `--config` method described [here](/docs/reference/setup-tools/kubeadm/kubeadm-init-phase/#cmd-phase-addon).
|
through the `--config` method described [here](/docs/reference/setup-tools/kubeadm/kubeadm-init-phase/#cmd-phase-addon).
|
||||||
|
|
||||||
- A ServiceAccount for CoreDNS is created in the `kube-system` namespace.
|
- A ServiceAccount for CoreDNS is created in the `kube-system` namespace.
|
||||||
|
|
||||||
|
@ -556,7 +547,7 @@ preconditions and avoid common cluster startup problems.
|
||||||
|
|
||||||
Please note that:
|
Please note that:
|
||||||
|
|
||||||
1. `kubeadm join` preflight checks are basically a subset `kubeadm init` preflight checks
|
1. `kubeadm join` preflight checks are basically a subset of `kubeadm init` preflight checks
|
||||||
1. Starting from 1.24, kubeadm uses crictl to communicate to all known CRI endpoints.
|
1. Starting from 1.24, kubeadm uses crictl to communicate to all known CRI endpoints.
|
||||||
1. Starting from 1.9, kubeadm provides support for joining nodes running on Windows; in that case,
|
1. Starting from 1.9, kubeadm provides support for joining nodes running on Windows; in that case,
|
||||||
linux specific controls are skipped.
|
linux specific controls are skipped.
|
||||||
|
@ -578,16 +569,16 @@ node basically retrieves the cluster CA certificates from the `cluster-info` Con
|
||||||
In order to prevent "man in the middle" attacks, several steps are taken:
|
In order to prevent "man in the middle" attacks, several steps are taken:
|
||||||
|
|
||||||
- First, the CA certificate is retrieved via insecure connection (this is possible because
|
- First, the CA certificate is retrieved via insecure connection (this is possible because
|
||||||
`kubeadm init` granted access to `cluster-info` users for `system:unauthenticated`)
|
`kubeadm init` is granted access to `cluster-info` users for `system:unauthenticated`)
|
||||||
|
|
||||||
- Then the CA certificate goes trough following validation steps:
|
- Then the CA certificate goes through following validation steps:
|
||||||
|
|
||||||
- Basic validation: using the token ID against a JWT signature
|
- Basic validation: using the token ID against a JWT signature
|
||||||
- Pub key validation: using provided `--discovery-token-ca-cert-hash`. This value is available
|
- Pub key validation: using provided `--discovery-token-ca-cert-hash`. This value is available
|
||||||
in the output of `kubeadm init` or can be calculated using standard tools (the hash is
|
in the output of `kubeadm init` or can be calculated using standard tools (the hash is
|
||||||
calculated over the bytes of the Subject Public Key Info (SPKI) object as in RFC7469). The
|
calculated over the bytes of the Subject Public Key Info (SPKI) object as in RFC7469). The
|
||||||
`--discovery-token-ca-cert-hash flag` may be repeated multiple times to allow more than one public key.
|
`--discovery-token-ca-cert-hash flag` may be repeated multiple times to allow more than one public key.
|
||||||
- As a additional validation, the CA certificate is retrieved via secure connection and then
|
- As an additional validation, the CA certificate is retrieved via secure connection and then
|
||||||
compared with the CA retrieved initially
|
compared with the CA retrieved initially
|
||||||
|
|
||||||
{{< note >}}
|
{{< note >}}
|
||||||
|
@ -602,30 +593,30 @@ If `kubeadm join` is invoked with `--discovery-file`, file discovery is used; th
|
||||||
local file or downloaded via an HTTPS URL; in case of HTTPS, the host installed CA bundle is used
|
local file or downloaded via an HTTPS URL; in case of HTTPS, the host installed CA bundle is used
|
||||||
to verify the connection.
|
to verify the connection.
|
||||||
|
|
||||||
With file discovery, the cluster CA certificates is provided into the file itself; in fact, the
|
With file discovery, the cluster CA certificate is provided into the file itself; in fact, the
|
||||||
discovery file is a kubeconfig file with only `server` and `certificate-authority-data` attributes
|
discovery file is a kubeconfig file with only `server` and `certificate-authority-data` attributes
|
||||||
set, as described in [`kubeadm join`](/docs/reference/setup-tools/kubeadm/kubeadm-join/#file-or-https-based-discovery)
|
set, as described in the [`kubeadm join`](/docs/reference/setup-tools/kubeadm/kubeadm-join/#file-or-https-based-discovery)
|
||||||
reference doc; when the connection with the cluster is established, kubeadm try to access the
|
reference doc; when the connection with the cluster is established, kubeadm tries to access the
|
||||||
`cluster-info` ConfigMap, and if available, uses it.
|
`cluster-info` ConfigMap, and if available, uses it.
|
||||||
|
|
||||||
## TLS Bootstrap
|
## TLS Bootstrap
|
||||||
|
|
||||||
Once the cluster info are known, the file `bootstrap-kubelet.conf` is written, thus allowing
|
Once the cluster info is known, the file `bootstrap-kubelet.conf` is written, thus allowing
|
||||||
kubelet to do TLS Bootstrapping.
|
kubelet to do TLS Bootstrapping.
|
||||||
|
|
||||||
The TLS bootstrap mechanism uses the shared token to temporarily authenticate with the Kubernetes
|
The TLS bootstrap mechanism uses the shared token to temporarily authenticate with the Kubernetes
|
||||||
API server to submit a certificate signing request (CSR) for a locally created key pair.
|
API server to submit a certificate signing request (CSR) for a locally created key pair.
|
||||||
|
|
||||||
The request is then automatically approved and the operation completes saving `ca.crt` file and
|
The request is then automatically approved and the operation completes saving `ca.crt` file and
|
||||||
`kubelet.conf` file to be used by kubelet for joining the cluster, while`bootstrap-kubelet.conf`
|
`kubelet.conf` file to be used by the kubelet for joining the cluster, while `bootstrap-kubelet.conf`
|
||||||
is deleted.
|
is deleted.
|
||||||
|
|
||||||
{{< note >}}
|
{{< note >}}
|
||||||
- The temporary authentication is validated against the token saved during the `kubeadm init`
|
- The temporary authentication is validated against the token saved during the `kubeadm init`
|
||||||
process (or with additional tokens created with `kubeadm token`)
|
process (or with additional tokens created with `kubeadm token` command)
|
||||||
- The temporary authentication resolve to a user member of
|
- The temporary authentication resolves to a user member of
|
||||||
`system:bootstrappers:kubeadm:default-node-token` group which was granted access to CSR api
|
`system:bootstrappers:kubeadm:default-node-token` group which was granted access to the CSR api
|
||||||
during the `kubeadm init` process
|
during the `kubeadm init` process
|
||||||
- The automatic CSR approval is managed by the csrapprover controller, according with
|
- The automatic CSR approval is managed by the csrapprover controller, according to
|
||||||
configuration done the `kubeadm init` process
|
the configuration present in the `kubeadm init` process
|
||||||
{{< /note >}}
|
{{< /note >}}
|
||||||
|
|
Loading…
Reference in New Issue