Merge pull request #44305 from sftim/20231211_dev_1.29_merge_main

Merge main into dev-1.29

content/en/community/_index.html

@@ -122,9 +122,9 @@ community_styles_migrated: true
 
     <div id="twitter" class="community-resource">
       <a href="https://twitter.com/kubernetesio">
-        <img src="/images/community/twitter.png" alt="Twitter">
+        <img src="/images/community/x-org.png" alt="𝕏.org">
       </a>
-      <a href="https://twitter.com/kubernetesio">Twitter&nbsp;&#9654;</a>
+      <a href="https://twitter.com/kubernetesio">𝕏&nbsp;&#9654;</a>
       <p><em>#kubernetesio</em></p>
       <p>Real-time announcements of blog posts, events, news, ideas.</p>
     </div>

content/en/docs/concepts/services-networking/ingress-controllers.md

@@ -64,7 +64,7 @@ Kubernetes as a project supports and maintains [AWS](https://github.com/kubernet
 * The [Traefik Kubernetes Ingress provider](https://doc.traefik.io/traefik/providers/kubernetes-ingress/) is an
   ingress controller for the [Traefik](https://traefik.io/traefik/) proxy.
 * [Tyk Operator](https://github.com/TykTechnologies/tyk-operator) extends Ingress with Custom Resources to bring API Management capabilities to Ingress. Tyk Operator works with the Open Source Tyk Gateway & Tyk Cloud control plane.
-* [Voyager](https://appscode.com/products/voyager) is an ingress controller for
+* [Voyager](https://voyagermesh.com) is an ingress controller for
   [HAProxy](https://www.haproxy.org/#desc).
 * [Wallarm Ingress Controller](https://www.wallarm.com/solutions/waf-for-kubernetes) is an Ingress Controller that provides WAAP (WAF) and API Security capabilities.
 

content/en/docs/concepts/services-networking/service.md

@@ -887,10 +887,7 @@ finding a Service: environment variables and DNS.
 When a Pod is run on a Node, the kubelet adds a set of environment variables
 for each active Service. It adds `{SVCNAME}_SERVICE_HOST` and `{SVCNAME}_SERVICE_PORT` variables,
 where the Service name is upper-cased and dashes are converted to underscores.
-It also supports variables
+
-(see [makeLinkVariables](https://github.com/kubernetes/kubernetes/blob/dd2d12f6dc0e654c15d5db57a5f9f6ba61192726/pkg/kubelet/envvars/envvars.go#L72))
-that are compatible with Docker Engine's
-"_[legacy container links](https://docs.docker.com/network/links/)_" feature.
 
 For example, the Service `redis-primary` which exposes TCP port 6379 and has been
 allocated cluster IP address 10.0.0.11, produces the following environment

content/en/docs/concepts/workloads/pods/init-containers.md

@@ -14,6 +14,9 @@ Init containers can contain utilities or setup scripts not present in an app ima
 You can specify init containers in the Pod specification alongside the `containers`
 array (which describes app containers).
 
+In Kubernetes, a [sidecar container](/docs/concepts/workloads/pods/sidecar-containers/) is a container that
+starts before the main application container and _continues to run_. This document is about init containers:
+containers that run to completion during Pod initialization.
 
 <!-- body -->
 
@@ -48,14 +51,33 @@ including resource limits, [volumes](/docs/concepts/storage/volumes/), and secur
 resource requests and limits for an init container are handled differently,
 as documented in [Resource sharing within containers](#resource-sharing-within-containers).
 
-Also, init containers do not support `lifecycle`, `livenessProbe`, `readinessProbe`, or
+Regular init containers (in other words: excluding sidecar containers) do not support the
-`startupProbe` because they must run to completion before the Pod can be ready.
+`lifecycle`, `livenessProbe`, `readinessProbe`, or `startupProbe` fields. Init containers
+must run to completion before the Pod can be ready; sidecar containers continue running
+during a Pod's lifetime, and _do_ support some probes. See [sidecar container](/docs/concepts/workloads/pods/sidecar-containers/)
+for further details about sidecar containers.
 
 If you specify multiple init containers for a Pod, kubelet runs each init
 container sequentially. Each init container must succeed before the next can run.
 When all of the init containers have run to completion, kubelet initializes
 the application containers for the Pod and runs them as usual.
 
+### Differences from sidecar containers
+
+Init containers run and complete their tasks before the main application container starts.
+Unlike [sidecar containers](/docs/concepts/workloads/pods/sidecar-containers),
+init containers are not continuously running alongside the main containers.
+
+Init containers run to completion sequentially, and the main container does not start
+until all the init containers have successfully completed.
+
+init containers do not support `lifecycle`, `livenessProbe`, `readinessProbe`, or
+`startupProbe` whereas sidecar containers support all these [probes](/docs/concepts/workloads/pods/pod-lifecycle/#types-of-probe) to control their lifecycle.
+
+Init containers share the same resources (CPU, memory, network) with the main application
+containers but do not interact directly with them. They can, however, use shared volumes
+for data exchange.
+
 ## Using init containers
 
 Because init containers have separate images from app containers, they
@@ -289,51 +311,9 @@ The Pod which is already running correctly would be killed by `activeDeadlineSec
 The name of each app and init container in a Pod must be unique; a
 validation error is thrown for any container sharing a name with another.
 
-#### API for sidecar containers
+### Resource sharing within containers
 
-{{< feature-state for_k8s_version="v1.29" state="beta" >}}
+Given the order of execution for init, sidecar and app containers, the following rules
-
-Enabled by default with Kubernetes 1.29, a feature gate named `SidecarContainers`
-allows you to specify a `restartPolicy` for init containers which is independent of
-the Pod and other init containers. Container [probes](/docs/concepts/workloads/pods/pod-lifecycle/#types-of-probe)
-can also be added to control their lifecycle.
-
-If an init container is created with its `restartPolicy` set to `Always`, it will
-start and remain running during the entire life of the Pod, which is useful for
-running supporting services separated from the main application containers.
-
-If a `readinessProbe` is specified for this init container, its result will be used
-to determine the `ready` state of the Pod.
-
-Since these containers are defined as init containers, they benefit from the same
-ordering and sequential guarantees as other init containers, allowing them to
-be mixed with other init containers into complex Pod initialization flows.
-
-Compared to regular init containers, sidecar-style init containers continue to
-run and the next init container can begin starting once the kubelet has set
-the `started` container status for the sidecar-style init container to true.
-That status either becomes true because there is a process running in the
-container and no startup probe defined, or
-as a result of its `startupProbe` succeeding.
-
-This feature can be used to implement the sidecar container pattern in a more
-robust way, as the kubelet always restarts a sidecar container if it fails.
-
-Here's an example of a Deployment with two containers, one of which is a sidecar:
-
-{{% code_sample language="yaml" file="application/deployment-sidecar.yaml" %}}
-
-This feature is also useful for running Jobs with sidecars, as the sidecar
-container will not prevent the Job from completing after the main container
-has finished.
-
-Here's an example of a Job with two containers, one of which is a sidecar:
-
-{{% code_sample language="yaml" file="application/job/job-sidecar.yaml" %}}
-
-#### Resource sharing within containers
-
-Given the ordering and execution for init containers, the following rules
 for resource usage apply:
 
 * The highest of any particular resource request or limit defined on all init
@@ -354,6 +334,10 @@ limit.
 Pod level control groups (cgroups) are based on the effective Pod request and
 limit, the same as the scheduler.
 
+{{< comment >}}
+This section also present under [sidecar containers](/docs/concepts/workloads/pods/sidecar-containers/) page.
+If you're editing this section, change both places.
+{{< /comment >}}
 
 ### Pod restart reasons
 
@@ -373,7 +357,9 @@ Kubernetes, consult the documentation for the version you are using.
 
 ## {{% heading "whatsnext" %}}
 
-* Read about [creating a Pod that has an init container](/docs/tasks/configure-pod-container/configure-pod-initialization/#create-a-pod-that-has-an-init-container)
+Learn more about the following:
-* Learn how to [debug init containers](/docs/tasks/debug/debug-application/debug-init-containers/)
+* [Creating a Pod that has an init container](/docs/tasks/configure-pod-container/configure-pod-initialization/#create-a-pod-that-has-an-init-container).
-* Read about an overview of [kubelet](/docs/reference/command-line-tools-reference/kubelet/) and [kubectl](/docs/reference/kubectl/)
+* [Debug init containers](/docs/tasks/debug/debug-application/debug-init-containers/).
-* Learn about the [types of probes](/docs/concepts/workloads/pods/pod-lifecycle/#types-of-probe): liveness, readiness, startup probe.
+* Overview of [kubelet](/docs/reference/command-line-tools-reference/kubelet/) and [kubectl](/docs/reference/kubectl/).
+* [Types of probes](/docs/concepts/workloads/pods/pod-lifecycle/#types-of-probe): liveness, readiness, startup probe.
+* [Sidecar containers](/docs/concepts/workloads/pods/sidecar-containers).

content/en/docs/concepts/workloads/pods/pod-lifecycle.md

@@ -150,11 +150,22 @@ the `Terminated` state.
 The `spec` of a Pod has a `restartPolicy` field with possible values Always, OnFailure,
 and Never. The default value is Always.
 
-The `restartPolicy` applies to all containers in the Pod. `restartPolicy` only
+The `restartPolicy` for a Pod applies to {{< glossary_tooltip text="app containers" term_id="app-container" >}}
-refers to restarts of the containers by the kubelet on the same node. After containers
+in the Pod and to regular [init containers](/docs/concepts/workloads/pods/init-containers/).
-in a Pod exit, the kubelet restarts them with an exponential back-off delay (10s, 20s,
+[Sidecar containers](/docs/concepts/workloads/pods/sidecar-containers/)
-40s, …), that is capped at five minutes. Once a container has executed for 10 minutes
+ignore the Pod-level `restartPolicy` field: in Kubernetes, a sidecar is defined as an
-without any problems, the kubelet resets the restart backoff timer for that container.
+entry inside `initContainers` that has its container-level `restartPolicy` set to `Always`.
+For init containers that exit with an error, the kubelet restarts the init container if
+the Pod level `restartPolicy` is either `OnFailure` or `Always`.
+
+When the kubelet is handling container restarts according to the configured restart
+policy, that only applies to restarts that make replacement containers inside the
+same Pod and running on the same node. After containers in a Pod exit, the kubelet
+restarts them with an exponential back-off delay (10s, 20s,40s, …), that is capped at
+five minutes. Once a container has executed for 10 minutes without any problems, the
+kubelet resets the restart backoff timer for that container.
+[Sidecar containers and Pod lifecycle](/docs/concepts/workloads/pods/sidecar-containers/#sidecar-containers-and-pod-lifecycle)
+explains the behaviour of `init containers` when specify `restartpolicy` field on it.
 
 ## Pod conditions
 
@@ -600,6 +611,8 @@ for more details.
 
 * Learn more about [container lifecycle hooks](/docs/concepts/containers/container-lifecycle-hooks/).
 
+* Learn more about [sidecar containers](/docs/concepts/workloads/pods/sidecar-containers/).
+
 * For detailed information about Pod and container status in the API, see
   the API reference documentation covering
   [`status`](/docs/reference/kubernetes-api/workload-resources/pod-v1/#PodStatus) for Pod.

content/en/docs/concepts/workloads/pods/sidecar-containers.md

@@ -0,0 +1,126 @@
+---
+title: Sidecar Containers
+content_type: concept
+weight: 50
+---
+
+<!-- overview -->
+{{< feature-state for_k8s_version="v1.29" state="beta" >}}
+
+Sidecar containers are the secondary containers that run along with the main
+application container within the same {{< glossary_tooltip text="Pod" term_id="pod" >}}.
+These containers are used to enhance or to extend the functionality of the main application
+container by providing additional services, or functionality such as logging, monitoring,
+security, or data synchronization, without directly altering the primary application code.
+
+<!-- body -->
+
+## Enabling sidecar containers
+
+Starting with Kubernetes 1.28, a
+[feature gate](/docs/reference/command-line-tools-reference/feature-gates/) named
+`SidecarContainers` allows you to specify a `restartPolicy` for containers listed in a
+Pod's `initContainers` field. These restartable _sidecar_ containers are independent with
+other [init containers](/docs/concepts/workloads/pods/init-containers/) and main
+application container within the same pod. These can be started, stopped, or restarted
+without effecting the main application container and other init containers.
+
+## Sidecar containers and Pod lifecycle
+
+If an init container is created with its `restartPolicy` set to `Always`, it will
+start and remain running during the entire life of the Pod. This can be helpful for
+running supporting services separated from the main application containers.
+
+If a `readinessProbe` is specified for this init container, its result will be used
+to determine the `ready` state of the Pod.
+
+Since these containers are defined as init containers, they benefit from the same
+ordering and sequential guarantees as other init containers, allowing them to
+be mixed with other init containers into complex Pod initialization flows.
+
+Compared to regular init containers, sidecars defined within `initContainers` continue to
+run after they have started. This is important when there is more than one entry inside
+`.spec.initContainers` for a Pod. After a sidecar-style init container is running (the kubelet
+has set the `started` status for that init container to true), the kubelet then starts the
+next init container from the ordered `.spec.initContainers` list.
+That status either becomes true because there is a process running in the
+container and no startup probe defined, or as a result of its `startupProbe` succeeding.
+
+Here's an example of a Deployment with two containers, one of which is a sidecar:
+
+{{% code_sample language="yaml" file="application/deployment-sidecar.yaml" %}}
+
+This feature is also useful for running Jobs with sidecars, as the sidecar
+container will not prevent the Job from completing after the main container
+has finished.
+
+Here's an example of a Job with two containers, one of which is a sidecar:
+
+{{% code_sample language="yaml" file="application/job/job-sidecar.yaml" %}}
+By default, this feature is not available in Kubernetes. To avail this feature, you
+need to enable the [feature gate](/docs/reference/command-line-tools-reference/feature-gates/)
+named `SidecarContainers`.
+
+## Differences from regular containers
+
+Sidecar containers run alongside regular containers in the same pod. However, they do not
+execute the primary application logic; instead, they provide supporting functionality to
+the main application.
+
+Sidecar containers have their own independent lifecycles. They can be started, stopped,
+and restarted independently of regular containers. This means you can update, scale, or
+maintain sidecar containers without affecting the primary application.
+
+Sidecar containers share the same network and storage namespaces with the primary
+container This co-location allows them to interact closely and share resources.
+
+## Differences from init containers
+
+Sidecar containers work alongside the main container, extending its functionality and
+providing additional services.
+
+Sidecar containers run concurrently with the main application container. They are active
+throughout the lifecycle of the pod and can be started and stopped independently of the
+main container. Unlike [init containers](/docs/concepts/workloads/pods/init-containers/),
+sidecar containers support [probes](/docs/concepts/workloads/pods/pod-lifecycle/#types-of-probe) to control their lifecycle.
+
+These containers can interact directly with the main application containers, sharing
+the same network namespace, filesystem, and environment variables. They work closely
+together to provide additional functionality.
+
+## Resource sharing within containers
+
+{{< comment >}}
+This section is also present in the [init containers](/docs/concepts/workloads/pods/init-containers/) page.
+If you're editing this section, change both places.
+{{< /comment >}}
+
+Given the order of execution for init, sidecar and app containers, the following rules
+for resource usage apply:
+
+* The highest of any particular resource request or limit defined on all init
+  containers is the *effective init request/limit*. If any resource has no
+  resource limit specified this is considered as the highest limit.
+* The Pod's *effective request/limit* for a resource is the sum of
+[pod overhead](/docs/concepts/scheduling-eviction/pod-overhead/) and the higher of:
+  * the sum of all non-init containers(app and sidecar containers) request/limit for a
+  resource
+  * the effective init request/limit for a resource
+* Scheduling is done based on effective requests/limits, which means
+  init containers can reserve resources for initialization that are not used
+  during the life of the Pod.
+* The QoS (quality of service) tier of the Pod's *effective QoS tier* is the
+  QoS tier for all init, sidecar and app containers alike.
+
+Quota and limits are applied based on the effective Pod request and
+limit.
+
+Pod level control groups (cgroups) are based on the effective Pod request and
+limit, the same as the scheduler.
+
+## {{% heading "whatsnext" %}}
+
+* Read a blog post on [native sidecar containers](/blog/2023/08/25/native-sidecar-containers/).
+* Read about [creating a Pod that has an init container](/docs/tasks/configure-pod-container/configure-pod-initialization/#create-a-pod-that-has-an-init-container).
+* Learn about the [types of probes](/docs/concepts/workloads/pods/pod-lifecycle/#types-of-probe): liveness, readiness, startup probe.
+* Learn about [pod overhead](/docs/concepts/scheduling-eviction/pod-overhead/).

content/en/docs/reference/setup-tools/kubeadm/kubeadm-certs.md

@@ -61,7 +61,9 @@ For more details see
 ## kubeadm certs generate-csr {#cmd-certs-generate-csr}
 
 This command can be used to generate keys and CSRs for all control-plane certificates and kubeconfig files.
-The user can then sign the CSRs with a CA of their choice.
+The user can then sign the CSRs with a CA of their choice. To read more information
+on how to use the command see
+[Signing certificate signing requests (CSR) generated by kubeadm](/docs/tasks/administer-cluster/kubeadm/kubeadm-certs#signing-csr).
 
 {{< tabs name="tab-certs-generate-csr" >}}
 {{< tab name="generate-csr" include="generated/kubeadm_certs_generate-csr.md" />}}

content/en/docs/reference/using-api/api-concepts.md

@@ -722,13 +722,13 @@ When you **delete** a resource this takes place in two phases.
   "kind": "ConfigMap",
   "apiVersion": "v1",
   "metadata": {
-    "finalizers": {"url.io/neat-finalization", "other-url.io/my-finalizer"},
+    "finalizers": ["url.io/neat-finalization", "other-url.io/my-finalizer"],
     "deletionTimestamp": nil,
   }
 }
 ```
 
-When a client first sends a **delete** to request removal of a resource, the `.metadata.deletionTimestamp` is set to the current time.
+When a client first sends a **delete** to request the removal of a resource, the `.metadata.deletionTimestamp` is set to the current time.
 Once the `.metadata.deletionTimestamp` is set, external controllers that act on finalizers
 may start performing their cleanup work at any time, in any order.
 

content/en/docs/tasks/administer-cluster/kubeadm/kubeadm-certs.md

@@ -16,6 +16,7 @@ to kubeadm certificate management.
 
 ## {{% heading "prerequisites" %}}
 
+
 You should be familiar with [PKI certificates and requirements in Kubernetes](/docs/setup/best-practices/certificates/).
 
 <!-- steps -->
@@ -70,7 +71,6 @@ etcd-peer                  Dec 30, 2020 23:36 UTC   364d            etcd-ca
 etcd-server                Dec 30, 2020 23:36 UTC   364d            etcd-ca                 no
 front-proxy-client         Dec 30, 2020 23:36 UTC   364d            front-proxy-ca          no
 scheduler.conf             Dec 30, 2020 23:36 UTC   364d                                    no
-super-admin.conf           Dec 30, 2020 23:36 UTC   364d                                    no
 
 CERTIFICATE AUTHORITY   EXPIRES                  RESIDUAL TIME   EXTERNALLY MANAGED
 ca                      Dec 28, 2029 23:36 UTC   9y              no
@@ -81,7 +81,6 @@ front-proxy-ca          Dec 28, 2029 23:36 UTC   9y              no
 The command shows expiration/residual time for the client certificates in the
 `/etc/kubernetes/pki` folder and for the client certificate embedded in the kubeconfig files used
 by kubeadm (`admin.conf`, `controller-manager.conf` and `scheduler.conf`).
-If missing, `super-admin.conf` will not cause an error.
 
 Additionally, kubeadm informs the user if the certificate is externally managed; in this case, the
 user should take care of managing certificate renewal manually/using other tools.
@@ -226,28 +225,11 @@ A CSR represents a request to a CA for a signed certificate for a client.
 In kubeadm terms, any certificate that would normally be signed by an on-disk CA can be produced
 as a CSR instead. A CA, however, cannot be produced as a CSR.
 
-### Create certificate signing requests (CSR)
+### Renewal by using certificate signing requests (CSR)
 
-You can create certificate signing requests with `kubeadm certs renew --csr-only`.
+Renewal of ceritficates is possible by generating new CSRs and signing them with the external CA.
-
+For more details about working with CSRs generated by kubeadm see the section
-Both the CSR and the accompanying private key are given in the output.
+[Signing certificate signing requests (CSR) generated by kubeadm](#signing-csr).
-You can pass in a directory with `--csr-dir` to output the CSRs to the specified location.
-If `--csr-dir` is not specified, the default certificate directory (`/etc/kubernetes/pki`) is used.
-
-Certificates can be renewed with `kubeadm certs renew --csr-only`.
-As with `kubeadm init`, an output directory can be specified with the `--csr-dir` flag.
-
-A CSR contains a certificate's name, domains, and IPs, but it does not specify usages.
-It is the responsibility of the CA to specify [the correct cert usages](/docs/setup/best-practices/certificates/#all-certificates)
-when issuing a certificate.
-
-* In `openssl` this is done with the
-  [`openssl ca` command](https://superuser.com/questions/738612/openssl-ca-keyusage-extension).
-* In `cfssl` you specify
-  [usages in the config file](https://github.com/cloudflare/cfssl/blob/master/doc/cmd/cfssl.txt#L170).
-
-After a certificate is signed using your preferred method, the certificate and the private key
-must be copied to the PKI directory (by default `/etc/kubernetes/pki`).
 
 ## Certificate authority (CA) rotation {#certificate-authority-rotation}
 
@@ -327,18 +309,14 @@ CSRs requesting serving certificates for any IP or domain name.
 
 ## Generating kubeconfig files for additional users {#kubeconfig-additional-users}
 
-The kubeconfig file `admin.conf` that `kubeadm init` generates contains a certificate with
+During cluster creation, kubeadm signs the certificate in the `admin.conf` to have
-`Subject: O = kubeadm:cluster-admins, CN = kubernetes-admin`. The group `kubeadm:cluster-admins`
+`Subject: O = system:masters, CN = kubernetes-admin`.
-is bound to the built-in `cluster-admin` ClusterRole.
+[`system:masters`](/docs/reference/access-authn-authz/rbac/#user-facing-roles)
-Do not share the `admin.conf` file with anyone.
+is a break-glass, super user group that bypasses the authorization layer (for example,
+[RBAC](/docs/reference/access-authn-authz/rbac/)).
+Sharing the `admin.conf` with additional users is **not recommended**!
 
-`kubeadm init` generates another kubeconfig file `super-admin.conf` that contains a certificate with
+Instead, you can use the [`kubeadm kubeconfig user`](/docs/reference/setup-tools/kubeadm/kubeadm-kubeconfig)
-`Subject: O = system:masters, CN = kubernetes-super-admin`.
-`system:masters` is a break-glass, super user group that bypasses the authorization layer (for example RBAC).
-Do not share the `super-admin.conf` file with anyone. It is recommended to move the file to a safe location.
-
-Instead of sharing these files, you can use the
-[`kubeadm kubeconfig user`](/docs/reference/setup-tools/kubeadm/kubeadm-kubeconfig)
 command to generate kubeconfig files for additional users.
 The command accepts a mixture of command line flags and
 [kubeadm configuration](/docs/reference/config-api/kubeadm-config.v1beta3/) options.
@@ -373,34 +351,251 @@ for a new user `johndoe` that is part of the `appdevs` group:
 kubeadm kubeconfig user --config example.yaml --org appdevs --client-name johndoe --validity-period 24h
 ```
 
-The following example binds the `cluster-admin` ClusterRole to a new group called
+The following example will generate a kubeconfig file with administrator credentials valid for 1 week:
-`my-cluster-admins`. It then creates a new kubeconfig file for an admin called `my-admin` that
-is part of the `my-cluster-admins` group. The kubeconfig is valid for 1 week:
 
 ```shell
-kubectl create clusterrolebinding my-cluster-admins-binding --clusterrole cluster-admin --group my-cluster-admins
+kubeadm kubeconfig user --config example.yaml --client-name admin --validity-period 168h
-kubeadm kubeconfig user --config example.yaml --client-name my-admin --org my-cluster-admins --validity-period 168h
 ```
 
-Alternatively, you can create a ClusterRoleBinding for individual users. The following example
+## Signing certificate signing requests (CSR) generated by kubeadm {#signing-csr}
-binds the `cluster-admin` ClusterRole to the new user `my-admin` and then creates a kubeconfig
+
-for the user:
+You can create certificate signing requests with `kubeadm certs generate-csr`.
+Calling this command will generate `.csr` / `.key` file pairs for regular
+certificates. For certificates embedded in kubeconfig files, the command will
+generate a `.csr` / `.conf` pair where the key is already embedded in the `.conf` file.
+
+A CSR file contains all relevant information for a CA to sign a certificate.
+kubeadm uses a
+[well defined specification](/docs/setup/best-practices/certificates/#all-certificates)
+for all its certificates and CSRs.
+
+The default certificate directory is `/etc/kubernetes/pki`, while the default
+directory for kubeconfig files is `/etc/kubernetes`. These defaults can be
+overridden with the flags `--cert-dir` and `--kubeconfig-dir`, respectively.
+
+To pass custom options to `kubeadm certs generate-csr` use the `--config` flag,
+which accepts a [kubeadm configuration](/docs/reference/config-api/kubeadm-config.v1beta3/)
+file, similarly to commands such as `kubeadm init`. Any specification such
+as extra SANs and custom IP addresses must be stored in the same configuration
+file and used for all relevant kubeadm commands by passing it as `--config`.
+
+{{< note >}}
+This guide will cover the usage of the `openssl` command for singing the CSRs,
+but you can use your preferred tools.
+{{< /note >}}
+
+{{< note >}}
+This guide will use the default Kubernetes directory `/etc/kubernetes`, which requires
+a super user. If you are following this guide with permissive directories
+(by passing `--cert-dir` and `--kubeconfig-dir`) you can omit the `sudo` command).
+But note that the resulted files must be copied to the `/etc/kubernetes` tree,
+so that `kubeadm init` or `kubeadm join` will find them.
+{{< /note >}}
+
+### Preparing CA and service account files
+
+On the primary control plane node, where `kubeadm init` will be executed, call the following
+commands:
 
 ```shell
-kubectl create clusterrolebinding my-admin-binding --clusterrole cluster-admin --user my-admin
+sudo kubeadm init phase certs ca
-kubeadm kubeconfig user --config example.yaml --client-name my-admin ---validity-period 168h
+sudo kubeadm init phase certs etcd-ca
+sudo kubeadm init phase certs front-proxy-ca
+sudo kubeadm init phase certs sa
 ```
 
-Removing the ClusterRoleBinding for a group of users or individual users can act
+This will populate the folders `/etc/kubernetes/pki` and `/etc/kubernetes/pki/etcd`
-as permission revocation:
+with all self-signed CA files (certificates and keys) and service account (public and
+private keys) that kubeadm needs for a control plane node.
+
+{{< note >}}
+If you are using an external CA, you must generate the same files out of band and manually
+copy them to the primary control plane node in `/etc/kubernetes`. Once all CSRs
+are signed, you can delete the root CA key (`ca.key`) as noted in the
+[External CA mode](#external-ca-mode) section.
+{{< /note >}}
+
+For secondary control plane nodes (`kubeadm join --control-plane`) there is no need to call
+the above commands. Depending on how you setup the
+[High Availability](/docs/setup/production-environment/tools/kubeadm/high-availability)
+cluster, you either have to manually copy the same files from the primary
+control plane node, or use the automated `--upload-certs` functionality of `kubeadm init`.
+
+### Generate CSRs
+
+The `kubeadm certs generate-csr` command generates CSRs for all known certificates
+managed by kubeadm. Once the command is done you must manually delete `.csr`, `.conf`
+or `.key` files that you don't need.
+
+#### Considerations for kubelet.conf {#considerations-kubelet-conf}
+
+This section applies to both control plane and worker nodes.
+
+If you have deleted the `ca.key` file from control plane nodes
+([External CA mode](#external-ca-mode)), the active kube-controller-manager in
+this cluster will not be able to sign kubelet client certificates. If no external
+method for signing these certificates exists in your setup (such as an
+[external signer](#set-up-a-signer), you could manually sign the `kubelet.conf.csr`
+as explained in this guide.
+
+Note that this also means that the automatic
+[kubelet client certificate rotation](/docs/tasks/tls/certificate-rotation/#enabling-client-certificate-rotation)
+will be disabled. If so, close to certificate expiration, you must generate
+a new `kubelet.conf.csr`, sign the certificate, embed it in `kubelet.conf`
+and restart the kubelet.
+
+If this does not apply to your setup, you can skip processing the `kubelet.conf.csr`
+on secondary control plane and on workers nodes (all nodes tha call `kubeadm join ...`).
+That is because the active kube-controller-manager will be responsible
+for signing new kubelet client certificates.
+
+{{< note >}}
+Processing the `kubelet.conf.csr` on the primary control plane node
+(`kubeadm init`) is required, because that is considered the node that
+bootstraps the cluster and a pre-populated `kubelet.conf` is needed.
+{{< /note >}}
+
+#### Control plane nodes
+
+Execute the following command on primary (`kubeadm init`) and secondary
+(`kubeadm join --control-plane`) control plane nodes to generate all CSR files:
 
 ```shell
-kubectl delete clusterrolebinding <name-of-clusterrolebinding>
+sudo kubeadm certs generate-csr
 ```
 
-{{< warning >}}
+If external etcd is to be used, follow the
-Creating kubeconfig files for additional users that are part of the default kubeadm group
+[External etcd with kubeadm](docs/setup/production-environment/tools/kubeadm/high-availability/#external-etcd-nodes)
-`kubeadm:cluster-admins` or the built-in Kubernetes group `system:masters` is not recommended.
+guide to understand what CSR files are needed on the kubeadm and etcd nodes. Other
-Ideally, these groups should only be used for the users stored in `admin.conf` and `super-admin.conf` -
+`.csr` and `.key` files under `/etc/kubernetes/pki/etcd` can be removed.
-`kubernetes-admin` and `kubernetes-super-admin`, respectively.
+
-{{< /warning >}}
+Based on the explanation in
+[Considerations for kubelet.conf](#considerations-kubelet-conf) keep or delete
+the `kubelet.conf` and `kubelet.conf.csr` files.
+
+#### Worker nodes
+
+Based on the explanation in
+[Considerations for kubelet.conf](#considerations-kubelet-conf), optionally call:
+
+```shell
+sudo kubeadm certs generate-csr
+```
+
+and keep only the `kubelet.conf` and `kubelet.conf.csr` files. Alternatively skip
+the steps for worker nodes entirely.
+
+### Signing CSRs for all certificates
+
+{{< note >}}
+If you are using external CA and already have CA serial number files (`.srl`) for
+`openssl` you can copy such files to a kubeadm node where CSRs will be processed.
+`.srl` files to copy are `/etc/kubernetes/pki/ca.srl`,
+`/etc/kubernetes/pki/front-proxy-ca.srl` and `/etc/kubernetes/pki/etcd/ca.srl`.
+The files can be then moved to a new node where CSR files will be processed.
+
+If a `.srl` file is missing for a CA on a node, the script below will generate a new SRL file
+with a random starting serial number.
+
+To read more about `.srl` files see the
+[`openssl`](https://www.openssl.org/docs/man3.0/man1/openssl-x509.html)
+documentation for the `--CAserial` flag.
+{{< /note >}}
+
+Repeat this step for all nodes that have CSR files.
+
+Write the following script in the `/etc/kubernetes` directory, navigate to the directory
+and execute the script. The script will generate certificates for all CSR files that are
+present in the `/etc/kubernetes` tree.
+
+```bash
+#!/bin/bash
+
+# Set certificate expiration time in days
+DAYS=365
+
+# Process all CSR files except those for front-proxy and etcd
+find ./ -name "*.csr" | grep -v "pki/etcd" | grep -v "front-proxy" | while read -r FILE;
+do
+    echo "* Processing ${FILE} ..."
+    FILE=${FILE%.*} # Trim the extension
+    if [ -f "./pki/ca.srl" ]; then
+        SERIAL_FLAG="-CAserial ./pki/ca.srl"
+    else
+        SERIAL_FLAG="-CAcreateserial"
+    fi
+    openssl x509 -req -days "${DAYS}" -CA ./pki/ca.crt -CAkey ./pki/ca.key ${SERIAL_FLAG} \
+        -in "${FILE}.csr" -out "${FILE}.crt"
+    sleep 2
+done
+
+# Process all etcd CSRs
+find ./pki/etcd -name "*.csr" | while read -r FILE;
+do
+    echo "* Processing ${FILE} ..."
+    FILE=${FILE%.*} # Trim the extension
+    if [ -f "./pki/etcd/ca.srl" ]; then
+        SERIAL_FLAG=-CAserial ./pki/etcd/ca.srl
+    else
+        SERIAL_FLAG=-CAcreateserial
+    fi
+    openssl x509 -req -days "${DAYS}" -CA ./pki/etcd/ca.crt -CAkey ./pki/etcd/ca.key ${SERIAL_FLAG} \
+        -in "${FILE}.csr" -out "${FILE}.crt"
+done
+
+# Process front-proxy CSRs
+echo "* Processing ./pki/front-proxy-client.csr ..."
+openssl x509 -req -days "${DAYS}" -CA ./pki/front-proxy-ca.crt -CAkey ./pki/front-proxy-ca.key -CAcreateserial \
+    -in ./pki/front-proxy-client.csr -out ./pki/front-proxy-client.crt
+```
+
+### Embedding certificates in kubeconfig files
+
+Repeat this step for all nodes that have CSR files.
+
+Write the following script in the `/etc/kubernetes` directory, navigate to the directory
+and execute the script. The script will take the `.crt` files that were signed for
+kubeconfig files from CSRs in the previous step and will embed them in the kubeconfig files.
+
+```bash
+#!/bin/bash
+
+CLUSTER=kubernetes
+find ./ -name "*.conf" | while read -r FILE;
+do
+    echo "* Processing ${FILE} ..."
+    KUBECONFIG="${FILE}" kubectl config set-cluster "${CLUSTER}" --certificate-authority ./pki/ca.crt --embed-certs
+    USER=$(KUBECONFIG="${FILE}" kubectl config view -o jsonpath='{.users[0].name}')
+    KUBECONFIG="${FILE}" kubectl config set-credentials "${USER}" --client-certificate "${FILE}.crt" --embed-certs
+done
+```
+
+### Performing cleanup {#post-csr-cleanup}
+
+Perform this step on all nodes that have CSR files.
+
+Write the following script in the `/etc/kubernetes` directory, navigate to the directory
+and execute the script.
+
+```bash
+#!/bin/bash
+
+# Cleanup CSR files
+rm -f ./*.csr ./pki/*.csr ./pki/etcd/*.csr # Clean all CSR files
+
+# Cleanup CRT files that were already embedded in kubeconfig files
+rm -f ./*.crt
+```
+
+Optionally, move `.srl` files to the next node to be processed.
+
+Optionally, if using external CA remove the `/etc/kubernetes/pki/ca.key` file,
+as explained in the [External CA node](#external-ca-mode) section.
+
+### kubeadm node initialization
+
+Once CSR files have been signed and required certificates are in place on the hosts
+you want to use as nodes, you can use the commands `kubeadm init` and `kubeadm join`
+to create a Kubernetes cluster from these nodes. During `init` and `join`, kubeadm
+uses existing certificates, encryption keys and kubeconfig files that it finds in the
+`/etc/kubernetes` tree on the host's local filesystem.

content/en/docs/tutorials/kubernetes-basics/public/images/module_05_scaling1.svg

@@ -24,13 +24,13 @@
 	.st18{fill:none;stroke:#06F7C9;stroke-width:2;stroke-miterlimit:10;}
 	.st19{fill:none;stroke:#06F7C9;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.3749,1.5832;}
 	.st20{fill:none;stroke:#06F7C9;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.4006,1.6004;}
-	.st21{fill:none;stroke:#EEF406;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.4,1.6;}
+	.st21{fill:none;stroke:#F29202;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.4,1.6;}
-	.st22{fill:none;stroke:#EEF406;stroke-width:2;stroke-miterlimit:10;}
+	.st22{fill:none;stroke:#F29202;stroke-width:2;stroke-miterlimit:10;}
-	.st23{fill:none;stroke:#EEF406;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.3975,1.5984;}
+	.st23{fill:none;stroke:#F29202;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.3975,1.5984;}
-	.st24{fill:none;stroke:#EEF406;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.395,1.5966;}
+	.st24{fill:none;stroke:#F29202;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.395,1.5966;}
-	.st25{fill:none;stroke:#EEF406;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.3963,1.5976;}
+	.st25{fill:none;stroke:#F29202;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.3963,1.5976;}
-	.st26{opacity:0.1;fill:#EEF406;}
+	.st26{opacity:0.1;fill:#F29202;}
-	.st27{opacity:2.000000e-02;fill:#EEF406;}
+	.st27{opacity:2.000000e-02;fill:#F29202;}
 	.st28{opacity:0.1;fill:#06F7C9;}
 	.st29{fill:none;stroke:#006DE9;stroke-width:0.8;stroke-miterlimit:10;}
 	.st30{opacity:0.1;fill:url(#SVGID_3_);}
@@ -43,12 +43,12 @@
 	.st37{opacity:0.1;fill:url(#SVGID_9_);}
 	.st38{opacity:0.1;fill:url(#SVGID_10_);}
 	.st39{fill:none;stroke:#326DE6;stroke-width:2;stroke-miterlimit:10;}
-	.st40{opacity:0.4;fill:none;stroke:#EEF406;stroke-width:2;stroke-miterlimit:10;}
+	.st40{opacity:0.4;fill:none;stroke:#F29202;stroke-width:2;stroke-miterlimit:10;}
-	.st41{fill:none;stroke:#EEF406;stroke-width:2.4596;stroke-miterlimit:10;}
+	.st41{fill:none;stroke:#F29202;stroke-width:2.4596;stroke-miterlimit:10;}
 	.st42{fill:#011F38;}
 	.st43{opacity:0.4;}
 	.st44{opacity:0.1;}
-	.st45{fill:#326DE6;stroke:#EEF406;stroke-width:2;stroke-miterlimit:10;}
+	.st45{fill:#326DE6;stroke:#F29202;stroke-width:2;stroke-miterlimit:10;}
 	.st46{fill:none;stroke:#FFFFFF;stroke-width:1.2;stroke-linecap:round;stroke-linejoin:round;}
 	.st47{fill:#06F7C9;stroke:#FFFFFF;stroke-width:0.3;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:10;}
 	.st48{fill:none;stroke:#011F38;stroke-width:1.2;stroke-linecap:round;stroke-linejoin:round;}
@@ -57,10 +57,10 @@
 	.st51{fill:#8115FF;stroke:#011F38;stroke-width:0.8;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:10;}
 	.st52{opacity:0.3;}
 	.st53{opacity:0.2;fill:#6D6E71;}
-	.st54{fill:#EEF406;}
+	.st54{fill:#F29202;}
 	.st55{fill:#06F7C9;}
 	.st56{fill:#FFFFFF;stroke:#06F7C9;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.4,1.6;}
-	.st57{fill:#FFFFFF;stroke:#EEF406;stroke-width:1.6;stroke-miterlimit:10;stroke-dasharray:2.4,1.6;}
+	.st57{fill:#FFFFFF;stroke:#F29202;stroke-width:1.6;stroke-miterlimit:10;stroke-dasharray:2.4,1.6;}
 	.st58{fill:none;stroke:#06F7C9;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.4938,1.6626;}
 	.st59{fill:none;stroke:#06F7C9;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.0084,1.3389;}
 	.st60{fill:none;stroke:#06F7C9;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.724,1.816;}

content/en/docs/tutorials/kubernetes-basics/public/images/module_05_scaling2.svg

@@ -24,13 +24,13 @@
 	.st18{fill:none;stroke:#06F7C9;stroke-width:2;stroke-miterlimit:10;}
 	.st19{fill:none;stroke:#06F7C9;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.3749,1.5832;}
 	.st20{fill:none;stroke:#06F7C9;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.4006,1.6004;}
-	.st21{fill:none;stroke:#EEF406;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.4,1.6;}
+	.st21{fill:none;stroke:#F29202;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.4,1.6;}
-	.st22{fill:none;stroke:#EEF406;stroke-width:2;stroke-miterlimit:10;}
+	.st22{fill:none;stroke:#F29202;stroke-width:2;stroke-miterlimit:10;}
-	.st23{fill:none;stroke:#EEF406;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.3975,1.5984;}
+	.st23{fill:none;stroke:#F29202;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.3975,1.5984;}
-	.st24{fill:none;stroke:#EEF406;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.395,1.5966;}
+	.st24{fill:none;stroke:#F29202;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.395,1.5966;}
-	.st25{fill:none;stroke:#EEF406;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.3963,1.5976;}
+	.st25{fill:none;stroke:#F29202;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.3963,1.5976;}
-	.st26{opacity:0.1;fill:#EEF406;}
+	.st26{opacity:0.1;fill:#F29202;}
-	.st27{opacity:2.000000e-02;fill:#EEF406;}
+	.st27{opacity:2.000000e-02;fill:#F29202;}
 	.st28{opacity:0.1;fill:#06F7C9;}
 	.st29{fill:none;stroke:#006DE9;stroke-width:0.8;stroke-miterlimit:10;}
 	.st30{opacity:0.1;fill:url(#SVGID_3_);}
@@ -43,12 +43,12 @@
 	.st37{opacity:0.1;fill:url(#SVGID_9_);}
 	.st38{opacity:0.1;fill:url(#SVGID_10_);}
 	.st39{fill:none;stroke:#326DE6;stroke-width:2;stroke-miterlimit:10;}
-	.st40{opacity:0.4;fill:none;stroke:#EEF406;stroke-width:2;stroke-miterlimit:10;}
+	.st40{opacity:0.4;fill:none;stroke:#F29202;stroke-width:2;stroke-miterlimit:10;}
-	.st41{fill:none;stroke:#EEF406;stroke-width:2.4596;stroke-miterlimit:10;}
+	.st41{fill:none;stroke:#F29202;stroke-width:2.4596;stroke-miterlimit:10;}
 	.st42{fill:#011F38;}
 	.st43{opacity:0.4;}
 	.st44{opacity:0.1;}
-	.st45{fill:#326DE6;stroke:#EEF406;stroke-width:2;stroke-miterlimit:10;}
+	.st45{fill:#326DE6;stroke:#F29202;stroke-width:2;stroke-miterlimit:10;}
 	.st46{fill:none;stroke:#FFFFFF;stroke-width:1.2;stroke-linecap:round;stroke-linejoin:round;}
 	.st47{fill:#06F7C9;stroke:#FFFFFF;stroke-width:0.3;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:10;}
 	.st48{fill:none;stroke:#011F38;stroke-width:1.2;stroke-linecap:round;stroke-linejoin:round;}
@@ -57,10 +57,10 @@
 	.st51{fill:#8115FF;stroke:#011F38;stroke-width:0.8;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:10;}
 	.st52{opacity:0.3;}
 	.st53{opacity:0.2;fill:#6D6E71;}
-	.st54{fill:#EEF406;}
+	.st54{fill:#F29202;}
 	.st55{fill:#06F7C9;}
 	.st56{fill:#FFFFFF;stroke:#06F7C9;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.4,1.6;}
-	.st57{fill:#FFFFFF;stroke:#EEF406;stroke-width:1.6;stroke-miterlimit:10;stroke-dasharray:2.4,1.6;}
+	.st57{fill:#FFFFFF;stroke:#F29202;stroke-width:1.6;stroke-miterlimit:10;stroke-dasharray:2.4,1.6;}
 	.st58{fill:none;stroke:#06F7C9;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.4938,1.6626;}
 	.st59{fill:none;stroke:#06F7C9;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.0084,1.3389;}
 	.st60{fill:none;stroke:#06F7C9;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.724,1.816;}

content/en/docs/tutorials/kubernetes-basics/public/images/module_06_rollingupdates1.svg

@@ -24,13 +24,13 @@
 	.st18{fill:none;stroke:#06F7C9;stroke-width:2;stroke-miterlimit:10;}
 	.st19{fill:none;stroke:#06F7C9;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.3749,1.5832;}
 	.st20{fill:none;stroke:#06F7C9;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.4006,1.6004;}
-	.st21{fill:none;stroke:#EEF406;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.4,1.6;}
+	.st21{fill:none;stroke:#F29202;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.4,1.6;}
-	.st22{fill:none;stroke:#EEF406;stroke-width:2;stroke-miterlimit:10;}
+	.st22{fill:none;stroke:#F29202;stroke-width:2;stroke-miterlimit:10;}
-	.st23{fill:none;stroke:#EEF406;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.3975,1.5984;}
+	.st23{fill:none;stroke:#F29202;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.3975,1.5984;}
-	.st24{fill:none;stroke:#EEF406;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.395,1.5966;}
+	.st24{fill:none;stroke:#F29202;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.395,1.5966;}
-	.st25{fill:none;stroke:#EEF406;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.3963,1.5976;}
+	.st25{fill:none;stroke:#F29202;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.3963,1.5976;}
-	.st26{opacity:0.1;fill:#EEF406;}
+	.st26{opacity:0.1;fill:#F29202;}
-	.st27{opacity:2.000000e-02;fill:#EEF406;}
+	.st27{opacity:2.000000e-02;fill:#F29202;}
 	.st28{opacity:0.1;fill:#06F7C9;}
 	.st29{fill:none;stroke:#006DE9;stroke-width:0.8;stroke-miterlimit:10;}
 	.st30{opacity:0.1;fill:url(#SVGID_3_);}
@@ -43,12 +43,12 @@
 	.st37{opacity:0.1;fill:url(#SVGID_9_);}
 	.st38{opacity:0.1;fill:url(#SVGID_10_);}
 	.st39{fill:none;stroke:#326DE6;stroke-width:2;stroke-miterlimit:10;}
-	.st40{opacity:0.4;fill:none;stroke:#EEF406;stroke-width:2;stroke-miterlimit:10;}
+	.st40{opacity:0.4;fill:none;stroke:#F29202;stroke-width:2;stroke-miterlimit:10;}
-	.st41{fill:none;stroke:#EEF406;stroke-width:2.4596;stroke-miterlimit:10;}
+	.st41{fill:none;stroke:#F29202;stroke-width:2.4596;stroke-miterlimit:10;}
 	.st42{fill:#011F38;}
 	.st43{opacity:0.4;}
 	.st44{opacity:0.1;}
-	.st45{fill:#326DE6;stroke:#EEF406;stroke-width:2;stroke-miterlimit:10;}
+	.st45{fill:#326DE6;stroke:#F29202;stroke-width:2;stroke-miterlimit:10;}
 	.st46{fill:none;stroke:#FFFFFF;stroke-width:1.2;stroke-linecap:round;stroke-linejoin:round;}
 	.st47{fill:#06F7C9;stroke:#FFFFFF;stroke-width:0.3;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:10;}
 	.st48{fill:none;stroke:#011F38;stroke-width:1.2;stroke-linecap:round;stroke-linejoin:round;}
@@ -57,10 +57,10 @@
 	.st51{fill:#8115FF;stroke:#011F38;stroke-width:0.8;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:10;}
 	.st52{opacity:0.3;}
 	.st53{opacity:0.2;fill:#6D6E71;}
-	.st54{fill:#EEF406;}
+	.st54{fill:#F29202;}
 	.st55{fill:#06F7C9;}
 	.st56{fill:#FFFFFF;stroke:#06F7C9;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.4,1.6;}
-	.st57{fill:#FFFFFF;stroke:#EEF406;stroke-width:1.6;stroke-miterlimit:10;stroke-dasharray:2.4,1.6;}
+	.st57{fill:#FFFFFF;stroke:#F29202;stroke-width:1.6;stroke-miterlimit:10;stroke-dasharray:2.4,1.6;}
 	.st58{fill:none;stroke:#06F7C9;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.4938,1.6626;}
 	.st59{fill:none;stroke:#06F7C9;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.0084,1.3389;}
 	.st60{fill:none;stroke:#06F7C9;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.724,1.816;}

content/en/docs/tutorials/kubernetes-basics/public/images/module_06_rollingupdates2.svg

@@ -24,13 +24,13 @@
 	.st18{fill:none;stroke:#06F7C9;stroke-width:2;stroke-miterlimit:10;}
 	.st19{fill:none;stroke:#06F7C9;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.3749,1.5832;}
 	.st20{fill:none;stroke:#06F7C9;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.4006,1.6004;}
-	.st21{fill:none;stroke:#EEF406;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.4,1.6;}
+	.st21{fill:none;stroke:#F29202;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.4,1.6;}
-	.st22{fill:none;stroke:#EEF406;stroke-width:2;stroke-miterlimit:10;}
+	.st22{fill:none;stroke:#F29202;stroke-width:2;stroke-miterlimit:10;}
-	.st23{fill:none;stroke:#EEF406;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.3975,1.5984;}
+	.st23{fill:none;stroke:#F29202;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.3975,1.5984;}
-	.st24{fill:none;stroke:#EEF406;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.395,1.5966;}
+	.st24{fill:none;stroke:#F29202;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.395,1.5966;}
-	.st25{fill:none;stroke:#EEF406;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.3963,1.5976;}
+	.st25{fill:none;stroke:#F29202;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.3963,1.5976;}
-	.st26{opacity:0.1;fill:#EEF406;}
+	.st26{opacity:0.1;fill:#F29202;}
-	.st27{opacity:2.000000e-02;fill:#EEF406;}
+	.st27{opacity:2.000000e-02;fill:#F29202;}
 	.st28{opacity:0.1;fill:#06F7C9;}
 	.st29{fill:none;stroke:#006DE9;stroke-width:0.8;stroke-miterlimit:10;}
 	.st30{opacity:0.1;fill:url(#SVGID_3_);}
@@ -43,12 +43,12 @@
 	.st37{opacity:0.1;fill:url(#SVGID_9_);}
 	.st38{opacity:0.1;fill:url(#SVGID_10_);}
 	.st39{fill:none;stroke:#326DE6;stroke-width:2;stroke-miterlimit:10;}
-	.st40{opacity:0.4;fill:none;stroke:#EEF406;stroke-width:2;stroke-miterlimit:10;}
+	.st40{opacity:0.4;fill:none;stroke:#F29202;stroke-width:2;stroke-miterlimit:10;}
-	.st41{fill:none;stroke:#EEF406;stroke-width:2.4596;stroke-miterlimit:10;}
+	.st41{fill:none;stroke:#F29202;stroke-width:2.4596;stroke-miterlimit:10;}
 	.st42{fill:#011F38;}
 	.st43{opacity:0.4;}
 	.st44{opacity:0.1;}
-	.st45{fill:#326DE6;stroke:#EEF406;stroke-width:2;stroke-miterlimit:10;}
+	.st45{fill:#326DE6;stroke:#F29202;stroke-width:2;stroke-miterlimit:10;}
 	.st46{fill:none;stroke:#FFFFFF;stroke-width:1.2;stroke-linecap:round;stroke-linejoin:round;}
 	.st47{fill:#06F7C9;stroke:#FFFFFF;stroke-width:0.3;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:10;}
 	.st48{fill:none;stroke:#011F38;stroke-width:1.2;stroke-linecap:round;stroke-linejoin:round;}
@@ -57,10 +57,10 @@
 	.st51{fill:#8115FF;stroke:#011F38;stroke-width:0.8;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:10;}
 	.st52{opacity:0.3;}
 	.st53{opacity:0.2;fill:#6D6E71;}
-	.st54{fill:#EEF406;}
+	.st54{fill:#F29202;}
 	.st55{fill:#06F7C9;}
 	.st56{fill:#FFFFFF;stroke:#06F7C9;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.4,1.6;}
-	.st57{fill:#FFFFFF;stroke:#EEF406;stroke-width:1.6;stroke-miterlimit:10;stroke-dasharray:2.4,1.6;}
+	.st57{fill:#FFFFFF;stroke:#F29202;stroke-width:1.6;stroke-miterlimit:10;stroke-dasharray:2.4,1.6;}
 	.st58{fill:none;stroke:#06F7C9;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.4938,1.6626;}
 	.st59{fill:none;stroke:#06F7C9;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.0084,1.3389;}
 	.st60{fill:none;stroke:#06F7C9;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.724,1.816;}

content/en/docs/tutorials/kubernetes-basics/public/images/module_06_rollingupdates3.svg

@@ -24,13 +24,13 @@
 	.st18{fill:none;stroke:#06F7C9;stroke-width:2;stroke-miterlimit:10;}
 	.st19{fill:none;stroke:#06F7C9;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.3749,1.5832;}
 	.st20{fill:none;stroke:#06F7C9;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.4006,1.6004;}
-	.st21{fill:none;stroke:#EEF406;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.4,1.6;}
+	.st21{fill:none;stroke:#F29202;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.4,1.6;}
-	.st22{fill:none;stroke:#EEF406;stroke-width:2;stroke-miterlimit:10;}
+	.st22{fill:none;stroke:#F29202;stroke-width:2;stroke-miterlimit:10;}
-	.st23{fill:none;stroke:#EEF406;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.3975,1.5984;}
+	.st23{fill:none;stroke:#F29202;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.3975,1.5984;}
-	.st24{fill:none;stroke:#EEF406;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.395,1.5966;}
+	.st24{fill:none;stroke:#F29202;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.395,1.5966;}
-	.st25{fill:none;stroke:#EEF406;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.3963,1.5976;}
+	.st25{fill:none;stroke:#F29202;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.3963,1.5976;}
-	.st26{opacity:0.1;fill:#EEF406;}
+	.st26{opacity:0.1;fill:#F29202;}
-	.st27{opacity:2.000000e-02;fill:#EEF406;}
+	.st27{opacity:2.000000e-02;fill:#F29202;}
 	.st28{opacity:0.1;fill:#06F7C9;}
 	.st29{fill:none;stroke:#006DE9;stroke-width:0.8;stroke-miterlimit:10;}
 	.st30{opacity:0.1;fill:url(#SVGID_3_);}
@@ -43,12 +43,12 @@
 	.st37{opacity:0.1;fill:url(#SVGID_9_);}
 	.st38{opacity:0.1;fill:url(#SVGID_10_);}
 	.st39{fill:none;stroke:#326DE6;stroke-width:2;stroke-miterlimit:10;}
-	.st40{opacity:0.4;fill:none;stroke:#EEF406;stroke-width:2;stroke-miterlimit:10;}
+	.st40{opacity:0.4;fill:none;stroke:#F29202;stroke-width:2;stroke-miterlimit:10;}
-	.st41{fill:none;stroke:#EEF406;stroke-width:2.4596;stroke-miterlimit:10;}
+	.st41{fill:none;stroke:#F29202;stroke-width:2.4596;stroke-miterlimit:10;}
 	.st42{fill:#011F38;}
 	.st43{opacity:0.4;}
 	.st44{opacity:0.1;}
-	.st45{fill:#326DE6;stroke:#EEF406;stroke-width:2;stroke-miterlimit:10;}
+	.st45{fill:#326DE6;stroke:#F29202;stroke-width:2;stroke-miterlimit:10;}
 	.st46{fill:none;stroke:#FFFFFF;stroke-width:1.2;stroke-linecap:round;stroke-linejoin:round;}
 	.st47{fill:#06F7C9;stroke:#FFFFFF;stroke-width:0.3;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:10;}
 	.st48{fill:none;stroke:#011F38;stroke-width:1.2;stroke-linecap:round;stroke-linejoin:round;}
@@ -57,10 +57,10 @@
 	.st51{fill:#8115FF;stroke:#011F38;stroke-width:0.8;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:10;}
 	.st52{opacity:0.3;}
 	.st53{opacity:0.2;fill:#6D6E71;}
-	.st54{fill:#EEF406;}
+	.st54{fill:#F29202;}
 	.st55{fill:#06F7C9;}
 	.st56{fill:#FFFFFF;stroke:#06F7C9;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.4,1.6;}
-	.st57{fill:#FFFFFF;stroke:#EEF406;stroke-width:1.6;stroke-miterlimit:10;stroke-dasharray:2.4,1.6;}
+	.st57{fill:#FFFFFF;stroke:#F29202;stroke-width:1.6;stroke-miterlimit:10;stroke-dasharray:2.4,1.6;}
 	.st58{fill:none;stroke:#06F7C9;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.4938,1.6626;}
 	.st59{fill:none;stroke:#06F7C9;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.0084,1.3389;}
 	.st60{fill:none;stroke:#06F7C9;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.724,1.816;}

content/en/docs/tutorials/kubernetes-basics/public/images/module_06_rollingupdates4.svg

@@ -24,13 +24,13 @@
 	.st18{fill:none;stroke:#06F7C9;stroke-width:2;stroke-miterlimit:10;}
 	.st19{fill:none;stroke:#06F7C9;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.3749,1.5832;}
 	.st20{fill:none;stroke:#06F7C9;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.4006,1.6004;}
-	.st21{fill:none;stroke:#EEF406;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.4,1.6;}
+	.st21{fill:none;stroke:#F29202;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.4,1.6;}
-	.st22{fill:none;stroke:#EEF406;stroke-width:2;stroke-miterlimit:10;}
+	.st22{fill:none;stroke:#F29202;stroke-width:2;stroke-miterlimit:10;}
-	.st23{fill:none;stroke:#EEF406;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.3975,1.5984;}
+	.st23{fill:none;stroke:#F29202;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.3975,1.5984;}
-	.st24{fill:none;stroke:#EEF406;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.395,1.5966;}
+	.st24{fill:none;stroke:#F29202;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.395,1.5966;}
-	.st25{fill:none;stroke:#EEF406;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.3963,1.5976;}
+	.st25{fill:none;stroke:#F29202;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.3963,1.5976;}
-	.st26{opacity:0.1;fill:#EEF406;}
+	.st26{opacity:0.1;fill:#F29202;}
-	.st27{opacity:2.000000e-02;fill:#EEF406;}
+	.st27{opacity:2.000000e-02;fill:#F29202;}
 	.st28{opacity:0.1;fill:#06F7C9;}
 	.st29{fill:none;stroke:#006DE9;stroke-width:0.8;stroke-miterlimit:10;}
 	.st30{opacity:0.1;fill:url(#SVGID_3_);}
@@ -43,12 +43,12 @@
 	.st37{opacity:0.1;fill:url(#SVGID_9_);}
 	.st38{opacity:0.1;fill:url(#SVGID_10_);}
 	.st39{fill:none;stroke:#326DE6;stroke-width:2;stroke-miterlimit:10;}
-	.st40{opacity:0.4;fill:none;stroke:#EEF406;stroke-width:2;stroke-miterlimit:10;}
+	.st40{opacity:0.4;fill:none;stroke:#F29202;stroke-width:2;stroke-miterlimit:10;}
-	.st41{fill:none;stroke:#EEF406;stroke-width:2.4596;stroke-miterlimit:10;}
+	.st41{fill:none;stroke:#F29202;stroke-width:2.4596;stroke-miterlimit:10;}
 	.st42{fill:#011F38;}
 	.st43{opacity:0.4;}
 	.st44{opacity:0.1;}
-	.st45{fill:#326DE6;stroke:#EEF406;stroke-width:2;stroke-miterlimit:10;}
+	.st45{fill:#326DE6;stroke:#F29202;stroke-width:2;stroke-miterlimit:10;}
 	.st46{fill:none;stroke:#FFFFFF;stroke-width:1.2;stroke-linecap:round;stroke-linejoin:round;}
 	.st47{fill:#06F7C9;stroke:#FFFFFF;stroke-width:0.3;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:10;}
 	.st48{fill:none;stroke:#011F38;stroke-width:1.2;stroke-linecap:round;stroke-linejoin:round;}
@@ -57,10 +57,10 @@
 	.st51{fill:#8115FF;stroke:#011F38;stroke-width:0.8;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:10;}
 	.st52{opacity:0.3;}
 	.st53{opacity:0.2;fill:#6D6E71;}
-	.st54{fill:#EEF406;}
+	.st54{fill:#F29202;}
 	.st55{fill:#06F7C9;}
 	.st56{fill:#FFFFFF;stroke:#06F7C9;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.4,1.6;}
-	.st57{fill:#FFFFFF;stroke:#EEF406;stroke-width:1.6;stroke-miterlimit:10;stroke-dasharray:2.4,1.6;}
+	.st57{fill:#FFFFFF;stroke:#F29202;stroke-width:1.6;stroke-miterlimit:10;stroke-dasharray:2.4,1.6;}
 	.st58{fill:none;stroke:#06F7C9;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.4938,1.6626;}
 	.st59{fill:none;stroke:#06F7C9;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.0084,1.3389;}
 	.st60{fill:none;stroke:#06F7C9;stroke-width:2;stroke-miterlimit:10;stroke-dasharray:2.724,1.816;}

content/pt-br/docs/concepts/extend-kubernetes/operator.md

@@ -93,7 +93,7 @@ A seguir estão algumas bibliotecas e ferramentas que você pode usar para escre
 ## {{% heading "whatsnext" %}} 
 
 
-* Leia o [whitepaper sobre operadores](https://github.com/cncf/tag-app-delivery/blob/eece8f7307f2970f46f100f51932db106db46968/operator-wg/whitepaper/Operator-WhitePaper_v1-0.md) da {{< glossary_tooltip text="CNCF" term_id="cncf" >}}
+* Leia o [whitepaper sobre operadores](https://github.com/cncf/tag-app-delivery/blob/163962c4b1cd70d085107fc579e3e04c2e14d59c/operator-wg/whitepaper/Operator-WhitePaper_v1-0.md) da {{< glossary_tooltip text="CNCF" term_id="cncf" >}}
 * Saiba mais sobre [Custom Resources](/docs/concepts/extend-kubernetes/api-extension/custom-resources/)
 * Encontre operadores prontos em [OperatorHub.io](https://operatorhub.io/) para atender ao seu caso de uso
 * [Publique](https://operatorhub.io/) seu operador para outras pessoas usarem

content/zh-cn/_index.html

@@ -70,14 +70,14 @@ Kubernetes 是开源系统，可以自由地部署在企业内部，私有云、
         <button id="desktopShowVideoButton" onclick="kub.showVideo()">观看视频</button>
         <br>
         <br>
-        <!-- <a href="https://events.linuxfoundation.org/kubecon-cloudnativecon-north-america/" button id="desktopKCButton">Attend KubeCon + CloudNativeCon North America on November 6-9, 2023</a> -->
-        <a href="https://events.linuxfoundation.org/kubecon-cloudnativecon-north-america/" button id="desktopKCButton">参加 2023 年 11 月 6-9 日的北美 KubeCon + CloudNativeCon</a>
-        <br>
-        <br>
-        <br>
-        <br>
         <!-- <a href="https://events.linuxfoundation.org/kubecon-cloudnativecon-europe/" button id="desktopKCButton">Attend KubeCon + CloudNativeCon Europe on March 19-22, 2024</a> -->
         <a href="https://events.linuxfoundation.org/kubecon-cloudnativecon-europe/" button id="desktopKCButton">参加 2024 年 3 月 19-22 日的欧洲 KubeCon + CloudNativeCon</a>
+        <br>
+        <br>
+        <br>
+        <br>
+        <!-- <a href="https://events.linuxfoundation.org/kubecon-cloudnativecon-north-america-2024/" button id="desktopKCButton">Attend KubeCon + CloudNativeCon North America on November 12-15, 2024</a> -->
+        <a href="https://events.linuxfoundation.org/kubecon-cloudnativecon-north-america-2024/" button id="desktopKCButton">参加 2024 年 11 月 12-15 日的北美 KubeCon + CloudNativeCon</a>
 </div>
 <div id="videoPlayer">
     <iframe data-url="https://www.youtube.com/embed/H06qrNmGqyE?autoplay=1" frameborder="0" allowfullscreen></iframe>

content/zh-cn/docs/concepts/configuration/secret.md

@@ -705,7 +705,7 @@ TLS Secret 的一种典型用法是为 [Ingress](/zh-cn/docs/concepts/services-n
 <!--
 The TLS Secret type is provided only for convenience.
 You can create an `Opaque` type for credentials used for TLS authentication.
-However, using the defined and public Secret type (`kubernetes.io/ssh-auth`)
+However, using the defined and public Secret type (`kubernetes.io/tls`)
 helps ensure the consistency of Secret format in your project. The API server
 verifies if the required keys are set for a Secret of this type.
 
@@ -713,7 +713,7 @@ To create a TLS Secret using `kubectl`, use the `tls` subcommand:
 -->
 提供 TLS 类型的 Secret 仅仅是出于方便性考虑。
 你可以创建 `Opaque` 类型的 Secret 来保存用于 TLS 身份认证的凭据。
-不过，使用已定义和公开的 Secret 类型有助于确保你自己项目中的 Secret 格式的一致性。
+不过，使用已定义和公开的 Secret 类型（`kubernetes.io/tls`）有助于确保你自己项目中的 Secret 格式的一致性。
 API 服务器会验证这种类型的 Secret 是否设定了所需的主键。
 
 要使用 `kubectl` 创建 TLS Secret，你可以使用 `tls` 子命令：

content/zh-cn/docs/reference/kubectl/cheatsheet.md

@@ -700,7 +700,7 @@ kubectl logs my-pod                                 # dump pod logs (stdout)
 kubectl logs -l name=myLabel                        # dump pod logs, with label name=myLabel (stdout)
 kubectl logs my-pod --previous                      # dump pod logs (stdout) for a previous instantiation of a container
 kubectl logs my-pod -c my-container                 # dump pod container logs (stdout, multi-container case)
-kubectl logs -l name=myLabel -c my-container        # dump pod logs, with label name=myLabel (stdout)
+kubectl logs -l name=myLabel -c my-container        # dump pod container logs, with label name=myLabel (stdout)
 kubectl logs my-pod -c my-container --previous      # dump pod container logs (stdout, multi-container case) for a previous instantiation of a container
 kubectl logs -f my-pod                              # stream pod logs (stdout)
 kubectl logs -f my-pod -c my-container              # stream pod container logs (stdout, multi-container case)

content/zh-cn/examples/application/hpa/php-apache.yaml

@@ -1,4 +1,4 @@
-apiVersion: autoscaling/v1
+apiVersion: autoscaling/v2
 kind: HorizontalPodAutoscaler
 metadata:
   name: php-apache
@@ -9,4 +9,10 @@ spec:
     name: php-apache
   minReplicas: 1
   maxReplicas: 10
-  targetCPUUtilizationPercentage: 50
+  metrics:
+  - type: Resource
+    resource:
+      name: cpu
+      target:
+        type: Utilization
+        averageUtilization: 50