kubernetes
/
website
mirror of https://github.com/kubernetes/website.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

Merge remote-tracking branch 'upstream/main' into dev-1.29

pull/43682/head
Oluebube Princes Egbuna 2023-10-25 09:39:05 +01:00
parent 3218e72755 01867cedde
commit 0fbfc94968
358 changed files with 16573 additions and 6520 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
.editorconfig
View File

@ -16,5 +16,8 @@ indent_size = 2
indent_style = space
indent_size = 4
[*.{yaml}]
insert_final_newline = true
[Makefile]
indent_style = tab

2
README-pl.md
View File

@ -43,7 +43,7 @@ make container-image
make container-serve
```
Jeśli widzisz błędy, prawdopodobnie kontener z Hugo nie dysponuje wystarczającymi zasobami. Aby rozwiązać ten problem, zwiększ ilość dostępnych zasobów CPU i pamięci dla Dockera na Twojej maszynie ([MacOSX](https://docs.docker.com/docker-for-mac/#resources) i [Windows](https://docs.docker.com/docker-for-windows/#resources)).
Jeśli widzisz błędy, prawdopodobnie kontener z Hugo nie dysponuje wystarczającymi zasobami. Aby rozwiązać ten problem, zwiększ ilość dostępnych zasobów CPU i pamięci dla Dockera na Twojej maszynie ([MacOS](https://docs.docker.com/desktop/settings/mac/) i [Windows](https://docs.docker.com/desktop/settings/windows/)).
Aby obejrzeć zawartość serwisu, otwórz w przeglądarce adres <http://localhost:1313>. Po każdej zmianie plików źródłowych, Hugo automatycznie aktualizuje stronę i odświeża jej widok w przeglądarce.

2
README-pt.md
View File

@ -49,7 +49,7 @@ Para executar o build do website em um contêiner, execute o comando abaixo:
make container-serve
```
Caso ocorram erros, é provável que o contêiner que está executando o Hugo não tenha recursos suficientes. A solução é aumentar a quantidade de CPU e memória disponível para o Docker ([MacOSX](https://docs.docker.com/docker-for-mac/#resources) e [Windows](https://docs.docker.com/docker-for-windows/#resources)).
Caso ocorram erros, é provável que o contêiner que está executando o Hugo não tenha recursos suficientes. A solução é aumentar a quantidade de CPU e memória disponível para o Docker ([MacOS](https://docs.docker.com/desktop/settings/mac/) e [Windows](https://docs.docker.com/desktop/settings/windows/)).
Abra seu navegador em http://localhost:1313 para visualizar o website. Conforme você faz alterações nos arquivos fontes, o Hugo atualiza o website e força a atualização do navegador.

7
assets/scss/_base.scss
View File

@ -902,9 +902,16 @@ section#cncf {
margin: 0;
}
//Table Content
.tab-content table{
border-collapse: separate;
border-spacing: 6px;
}
.tab-pane {
border-radius: 0.25rem;
padding: 0 16px 16px;
overflow: auto;
border: 1px solid #dee2e6;
&:first-of-type.active {

2
assets/scss/_case-studies.scss
View File

@ -1,7 +1,7 @@
// SASS for Case Studies pages go here:
hr {
background-color: #999999;
background-color: #303030;
margin-top: 0;
}

4
content/de/docs/tasks/tools/install-kubectl-linux.md
View File

@ -51,7 +51,7 @@ Um kubectl auf Linux zu installieren, gibt es die folgenden Möglichkeiten:
Download der kubectl Checksum-Datei:
```bash
curl -LO "https://dl.k8s.io/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl.sha256"
curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl.sha256"
```
Kubectl Binary mit der Checksum-Datei validieren:
@ -236,7 +236,7 @@ Untenstehend ist beschrieben, wie die Autovervollständigungen für Fish und Zsh
Download der kubectl-convert Checksum-Datei:
```bash
curl -LO "https://dl.k8s.io/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl-convert.sha256"
curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl-convert.sha256"
```
Kubectl-convert Binary mit der Checksum-Datei validieren:

10
content/en/_index.html
View File

@ -47,12 +47,12 @@ To download Kubernetes, visit the [download](/releases/download/) section.
<button id="desktopShowVideoButton" onclick="kub.showVideo()">Watch Video</button>
<br>
<br>
<a href="https://events.linuxfoundation.org/kubecon-cloudnativecon-europe/" button id="desktopKCButton">Attend KubeCon + CloudNativeCon Europe on April 18-21, 2023</a>
<br>
<br>
<br>
<br>
<a href="https://events.linuxfoundation.org/kubecon-cloudnativecon-north-america/" button id="desktopKCButton">Attend KubeCon + CloudNativeCon North America on November 6-9, 2023</a>
<br>
<br>
<br>
<br>
<a href="https://events.linuxfoundation.org/kubecon-cloudnativecon-europe/" button id="desktopKCButton">Attend KubeCon + CloudNativeCon Europe on March 19-22, 2024</a>
</div>
<div id="videoPlayer">
<iframe data-url="https://www.youtube.com/embed/H06qrNmGqyE?autoplay=1" frameborder="0" allowfullscreen></iframe>

2
content/en/blog/_posts/2020-09-03-warnings/index.md
View File

@ -63,7 +63,7 @@ This metric has labels for the API `group`, `version`, `resource`, and `subresou
and a `removed_release` label that indicates the Kubernetes release in which the API will no longer be served.
This is an example query using `kubectl`, [prom2json](https://github.com/prometheus/prom2json),
and [jq](https://stedolan.github.io/jq/) to determine which deprecated APIs have been requested
and [jq](https://jqlang.github.io/jq/) to determine which deprecated APIs have been requested
from the current instance of the API server:
```sh

2
content/en/blog/_posts/2021-10-18-kpng-specialized-proxiers.md
View File

@ -210,7 +210,7 @@ podip=$(cat /tmp/out | jq -r '.Endpoints[]|select(.Local == true)|select(.IPs.V6
ip6tables -t nat -A PREROUTING -d $xip/128 -j DNAT --to-destination $podip
```
Assuming the JSON output above is stored in `/tmp/out` ([jq](https://stedolan.github.io/jq/) is an *awesome* program!).
Assuming the JSON output above is stored in `/tmp/out` ([jq](https://jqlang.github.io/jq/) is an *awesome* program!).
As this is an example we make it really simple for ourselves by using

4
content/en/blog/_posts/2022-08-31-cgroupv2-ga.md
View File

@ -118,8 +118,8 @@ Scenarios in which you might need to update to cgroup v2 include the following:
DaemonSet for monitoring pods and containers, update it to v0.43.0 or later.
* If you deploy Java applications, prefer to use versions which fully support cgroup v2:
* [OpenJDK / HotSpot](https://bugs.openjdk.org/browse/JDK-8230305): jdk8u372, 11.0.16, 15 and later
* [IBM Semeru Runtimes](https://www.eclipse.org/openj9/docs/version0.33/#control-groups-v2-support): jdk8u345-b01, 11.0.16.0, 17.0.4.0, 18.0.2.0 and later
* [IBM Java](https://www.ibm.com/docs/en/sdk-java-technology/8?topic=new-service-refresh-7#whatsnew_sr7__fp15): 8.0.7.15 and later
* [IBM Semeru Runtimes](https://www.ibm.com/support/pages/apar/IJ46681): 8.0.382.0, 11.0.20.0, 17.0.8.0, and later
* [IBM Java](https://www.ibm.com/support/pages/apar/IJ46681): 8.0.8.6 and later
## Learn more

480
content/en/blog/_posts/2023-05-05-memory-qos-cgroups-v2/container-memory-high-best-effort.svg
View File

@ -1,87 +1,395 @@
<?xml version='1.0' encoding='UTF-8'?>
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!-- Generated by CodeCogs with dvisvgm 2.13.3 -->
<svg version='1.1' xmlns='http://www.w3.org/2000/svg' xmlns:xlink='http://www.w3.org/1999/xlink' width='412.017841pt' height='12.401574pt' viewBox='-.299738 -.255124 412.017841 12.401574'>
<defs>
<path id='g2-61' d='M8.069738-3.873474C8.237111-3.873474 8.452304-3.873474 8.452304-4.088667C8.452304-4.315816 8.249066-4.315816 8.069738-4.315816H1.028144C.860772-4.315816 .645579-4.315816 .645579-4.100623C.645579-3.873474 .848817-3.873474 1.028144-3.873474H8.069738ZM8.069738-1.649813C8.237111-1.649813 8.452304-1.649813 8.452304-1.865006C8.452304-2.092154 8.249066-2.092154 8.069738-2.092154H1.028144C.860772-2.092154 .645579-2.092154 .645579-1.876961C.645579-1.649813 .848817-1.649813 1.028144-1.649813H8.069738Z'/>
<path id='g0-3' d='M3.287671-5.104857C3.299626-5.272229 3.299626-5.559153 2.988792-5.559153C2.797509-5.559153 2.642092-5.403736 2.677958-5.248319V-5.092902L2.84533-3.239851L1.315068-4.351681C1.207472-4.411457 1.183562-4.435367 1.099875-4.435367C.932503-4.435367 .777086-4.267995 .777086-4.100623C.777086-3.90934 .896638-3.861519 1.016189-3.801743L2.713823-2.988792L1.06401-2.187796C.872727-2.092154 .777086-2.044334 .777086-1.865006S.932503-1.530262 1.099875-1.530262C1.183562-1.530262 1.207472-1.530262 1.506351-1.75741L2.84533-2.725778L2.666002-.71731C2.666002-.466252 2.881196-.406476 2.976837-.406476C3.120299-.406476 3.299626-.490162 3.299626-.71731L3.120299-2.725778L4.65056-1.613948C4.758157-1.554172 4.782067-1.530262 4.865753-1.530262C5.033126-1.530262 5.188543-1.697634 5.188543-1.865006C5.188543-2.044334 5.080946-2.10411 4.937484-2.175841C4.220174-2.534496 4.196264-2.534496 3.251806-2.976837L4.901619-3.777833C5.092902-3.873474 5.188543-3.921295 5.188543-4.100623S5.033126-4.435367 4.865753-4.435367C4.782067-4.435367 4.758157-4.435367 4.459278-4.208219L3.120299-3.239851L3.287671-5.104857Z'/>
<path id='g1-58' d='M2.199751-.573848C2.199751-.920548 1.912827-1.159651 1.625903-1.159651C1.279203-1.159651 1.0401-.872727 1.0401-.585803C1.0401-.239103 1.327024 0 1.613948 0C1.960648 0 2.199751-.286924 2.199751-.573848Z'/>
<path id='g1-65' d='M2.032379-1.327024C1.613948-.621669 1.207472-.382565 .633624-.3467C.502117-.334745 .406476-.334745 .406476-.119552C.406476-.047821 .466252 0 .549938 0C.765131 0 1.303113-.02391 1.518306-.02391C1.865006-.02391 2.247572 0 2.582316 0C2.654047 0 2.797509 0 2.797509-.227148C2.797509-.334745 2.701868-.3467 2.630137-.3467C2.355168-.37061 2.12802-.466252 2.12802-.753176C2.12802-.920548 2.199751-1.052055 2.355168-1.315068L3.263761-2.82142H6.312329C6.324284-2.713823 6.324284-2.618182 6.336239-2.510585C6.372105-2.199751 6.515567-.956413 6.515567-.729265C6.515567-.37061 5.905853-.3467 5.71457-.3467C5.583064-.3467 5.451557-.3467 5.451557-.131507C5.451557 0 5.559153 0 5.630884 0C5.834122 0 6.073225-.02391 6.276463-.02391H6.957908C7.687173-.02391 8.2132 0 8.225156 0C8.308842 0 8.440349 0 8.440349-.227148C8.440349-.3467 8.332752-.3467 8.153425-.3467C7.49589-.3467 7.483935-.454296 7.44807-.812951L6.718804-8.272976C6.694894-8.51208 6.647073-8.53599 6.515567-8.53599C6.396015-8.53599 6.324284-8.51208 6.216687-8.332752L2.032379-1.327024ZM3.466999-3.16812L5.869988-7.185056L6.276463-3.16812H3.466999Z'/>
<path id='g1-70' d='M3.550685-3.897385H4.698381C5.606974-3.897385 5.678705-3.694147 5.678705-3.347447C5.678705-3.19203 5.654795-3.024658 5.595019-2.761644C5.571108-2.713823 5.559153-2.654047 5.559153-2.630137C5.559153-2.546451 5.606974-2.49863 5.69066-2.49863C5.786301-2.49863 5.798257-2.546451 5.846077-2.737733L6.539477-5.523288C6.539477-5.571108 6.503611-5.642839 6.419925-5.642839C6.312329-5.642839 6.300374-5.595019 6.252553-5.391781C6.001494-4.495143 5.762391-4.244085 4.722291-4.244085H3.634371L4.411457-7.340473C4.519054-7.758904 4.542964-7.79477 5.033126-7.79477H6.635118C8.129514-7.79477 8.344707-7.352428 8.344707-6.503611C8.344707-6.43188 8.344707-6.168867 8.308842-5.858032C8.296887-5.810212 8.272976-5.654795 8.272976-5.606974C8.272976-5.511333 8.332752-5.475467 8.404483-5.475467C8.488169-5.475467 8.53599-5.523288 8.5599-5.738481L8.810959-7.830635C8.810959-7.866501 8.834869-7.986052 8.834869-8.009963C8.834869-8.141469 8.727273-8.141469 8.51208-8.141469H2.84533C2.618182-8.141469 2.49863-8.141469 2.49863-7.926276C2.49863-7.79477 2.582316-7.79477 2.785554-7.79477C3.526775-7.79477 3.526775-7.711083 3.526775-7.579577C3.526775-7.519801 3.514819-7.47198 3.478954-7.340473L1.865006-.884682C1.75741-.466252 1.733499-.3467 .896638-.3467C.669489-.3467 .549938-.3467 .549938-.131507C.549938 0 .657534 0 .729265 0C.956413 0 1.195517-.02391 1.422665-.02391H2.976837C3.239851-.02391 3.526775 0 3.789788 0C3.897385 0 4.040847 0 4.040847-.215193C4.040847-.3467 3.969116-.3467 3.706102-.3467C2.761644-.3467 2.737733-.430386 2.737733-.609714C2.737733-.669489 2.761644-.765131 2.785554-.848817L3.550685-3.897385Z'/>
<path id='g1-77' d='M10.855293-7.292653C10.962889-7.699128 10.9868-7.81868 11.835616-7.81868C12.062765-7.81868 12.170361-7.81868 12.170361-8.045828C12.170361-8.16538 12.086675-8.16538 11.859527-8.16538H10.424907C10.126027-8.16538 10.114072-8.153425 9.982565-7.962142L5.618929-1.06401L4.722291-7.902366C4.686426-8.16538 4.674471-8.16538 4.363636-8.16538H2.881196C2.654047-8.16538 2.546451-8.16538 2.546451-7.938232C2.546451-7.81868 2.654047-7.81868 2.833375-7.81868C3.56264-7.81868 3.56264-7.723039 3.56264-7.591532C3.56264-7.567621 3.56264-7.49589 3.514819-7.316563L1.984558-1.219427C1.841096-.645579 1.566127-.382565 .765131-.3467C.729265-.3467 .585803-.334745 .585803-.131507C.585803 0 .6934 0 .74122 0C.980324 0 1.590037-.02391 1.829141-.02391H2.402989C2.570361-.02391 2.773599 0 2.940971 0C3.024658 0 3.156164 0 3.156164-.227148C3.156164-.334745 3.036613-.3467 2.988792-.3467C2.594271-.358655 2.211706-.430386 2.211706-.860772C2.211706-.980324 2.211706-.992279 2.259527-1.159651L3.90934-7.746949H3.921295L4.913574-.32279C4.94944-.035866 4.961395 0 5.068991 0C5.200498 0 5.260274-.095641 5.32005-.203238L10.126027-7.806725H10.137983L8.404483-.884682C8.296887-.466252 8.272976-.3467 7.436115-.3467C7.208966-.3467 7.089415-.3467 7.089415-.131507C7.089415 0 7.197011 0 7.268742 0C7.47198 0 7.711083-.02391 7.914321-.02391H9.325031C9.528269-.02391 9.779328 0 9.982565 0C10.078207 0 10.209714 0 10.209714-.227148C10.209714-.3467 10.102117-.3467 9.92279-.3467C9.193524-.3467 9.193524-.442341 9.193524-.561893C9.193524-.573848 9.193524-.657534 9.217435-.753176L10.855293-7.292653Z'/>
<path id='g1-78' d='M8.846824-6.910087C8.978331-7.424159 9.169614-7.782814 10.078207-7.81868C10.114072-7.81868 10.257534-7.830635 10.257534-8.033873C10.257534-8.16538 10.149938-8.16538 10.102117-8.16538C9.863014-8.16538 9.2533-8.141469 9.014197-8.141469H8.440349C8.272976-8.141469 8.057783-8.16538 7.890411-8.16538C7.81868-8.16538 7.675218-8.16538 7.675218-7.938232C7.675218-7.81868 7.770859-7.81868 7.854545-7.81868C8.571856-7.79477 8.619676-7.519801 8.619676-7.304608C8.619676-7.197011 8.607721-7.161146 8.571856-6.993773L7.220922-1.601993L4.662516-7.962142C4.578829-8.153425 4.566874-8.16538 4.303861-8.16538H2.84533C2.606227-8.16538 2.49863-8.16538 2.49863-7.938232C2.49863-7.81868 2.582316-7.81868 2.809465-7.81868C2.86924-7.81868 3.574595-7.81868 3.574595-7.711083C3.574595-7.687173 3.550685-7.591532 3.53873-7.555666L1.948692-1.219427C1.80523-.633624 1.518306-.382565 .729265-.3467C.669489-.3467 .549938-.334745 .549938-.119552C.549938 0 .669489 0 .705355 0C.944458 0 1.554172-.02391 1.793275-.02391H2.367123C2.534496-.02391 2.737733 0 2.905106 0C2.988792 0 3.120299 0 3.120299-.227148C3.120299-.334745 3.000747-.3467 2.952927-.3467C2.558406-.358655 2.175841-.430386 2.175841-.860772C2.175841-.956413 2.199751-1.06401 2.223661-1.159651L3.837609-7.555666C3.90934-7.436115 3.90934-7.412204 3.957161-7.304608L6.802491-.215193C6.862267-.071731 6.886177 0 6.993773 0C7.113325 0 7.12528-.035866 7.173101-.239103L8.846824-6.910087Z'/>
<path id='g1-84' d='M4.985305-7.292653C5.057036-7.579577 5.080946-7.687173 5.260274-7.734994C5.355915-7.758904 5.750436-7.758904 6.001494-7.758904C7.197011-7.758904 7.758904-7.711083 7.758904-6.77858C7.758904-6.599253 7.711083-6.144956 7.639352-5.702615L7.627397-5.559153C7.627397-5.511333 7.675218-5.439601 7.746949-5.439601C7.866501-5.439601 7.866501-5.499377 7.902366-5.69066L8.249066-7.806725C8.272976-7.914321 8.272976-7.938232 8.272976-7.974097C8.272976-8.105604 8.201245-8.105604 7.962142-8.105604H1.422665C1.147696-8.105604 1.135741-8.093649 1.06401-7.878456L.334745-5.726526C.32279-5.702615 .286924-5.571108 .286924-5.559153C.286924-5.499377 .334745-5.439601 .406476-5.439601C.502117-5.439601 .526027-5.487422 .573848-5.642839C1.075965-7.089415 1.327024-7.758904 2.917061-7.758904H3.718057C4.004981-7.758904 4.124533-7.758904 4.124533-7.627397C4.124533-7.591532 4.124533-7.567621 4.064757-7.352428L2.462765-.932503C2.343213-.466252 2.319303-.3467 1.052055-.3467C.753176-.3467 .669489-.3467 .669489-.119552C.669489 0 .800996 0 .860772 0C1.159651 0 1.470486-.02391 1.769365-.02391H3.634371C3.93325-.02391 4.25604 0 4.554919 0C4.686426 0 4.805978 0 4.805978-.227148C4.805978-.3467 4.722291-.3467 4.411457-.3467C3.335492-.3467 3.335492-.454296 3.335492-.633624C3.335492-.645579 3.335492-.729265 3.383313-.920548L4.985305-7.292653Z'/>
<path id='g1-97' d='M3.598506-1.422665C3.53873-1.219427 3.53873-1.195517 3.371357-.968369C3.108344-.633624 2.582316-.119552 2.020423-.119552C1.530262-.119552 1.255293-.561893 1.255293-1.267248C1.255293-1.924782 1.625903-3.263761 1.853051-3.765878C2.259527-4.60274 2.82142-5.033126 3.287671-5.033126C4.076712-5.033126 4.23213-4.052802 4.23213-3.957161C4.23213-3.945205 4.196264-3.789788 4.184309-3.765878L3.598506-1.422665ZM4.363636-4.483188C4.23213-4.794022 3.90934-5.272229 3.287671-5.272229C1.936737-5.272229 .478207-3.526775 .478207-1.75741C.478207-.573848 1.171606 .119552 1.984558 .119552C2.642092 .119552 3.203985-.394521 3.53873-.789041C3.658281-.083686 4.220174 .119552 4.578829 .119552S5.224408-.095641 5.439601-.526027C5.630884-.932503 5.798257-1.661768 5.798257-1.709589C5.798257-1.769365 5.750436-1.817186 5.678705-1.817186C5.571108-1.817186 5.559153-1.75741 5.511333-1.578082C5.332005-.872727 5.104857-.119552 4.614695-.119552C4.267995-.119552 4.244085-.430386 4.244085-.669489C4.244085-.944458 4.27995-1.075965 4.387547-1.542217C4.471233-1.841096 4.531009-2.10411 4.62665-2.450809C5.068991-4.244085 5.176588-4.674471 5.176588-4.746202C5.176588-4.913574 5.045081-5.045081 4.865753-5.045081C4.483188-5.045081 4.387547-4.62665 4.363636-4.483188Z'/>
<path id='g1-98' d='M2.761644-7.998007C2.773599-8.045828 2.797509-8.117559 2.797509-8.177335C2.797509-8.296887 2.677958-8.296887 2.654047-8.296887C2.642092-8.296887 2.211706-8.261021 1.996513-8.237111C1.793275-8.225156 1.613948-8.201245 1.398755-8.18929C1.111831-8.16538 1.028144-8.153425 1.028144-7.938232C1.028144-7.81868 1.147696-7.81868 1.267248-7.81868C1.876961-7.81868 1.876961-7.711083 1.876961-7.591532C1.876961-7.507846 1.78132-7.161146 1.733499-6.945953L1.446575-5.798257C1.327024-5.32005 .645579-2.606227 .597758-2.391034C.537983-2.092154 .537983-1.888917 .537983-1.733499C.537983-.514072 1.219427 .119552 1.996513 .119552C3.383313 .119552 4.817933-1.661768 4.817933-3.395268C4.817933-4.495143 4.196264-5.272229 3.299626-5.272229C2.677958-5.272229 2.116065-4.758157 1.888917-4.519054L2.761644-7.998007ZM2.008468-.119552C1.625903-.119552 1.207472-.406476 1.207472-1.338979C1.207472-1.733499 1.243337-1.960648 1.458531-2.797509C1.494396-2.952927 1.685679-3.718057 1.733499-3.873474C1.75741-3.969116 2.462765-5.033126 3.275716-5.033126C3.801743-5.033126 4.040847-4.507098 4.040847-3.88543C4.040847-3.311582 3.706102-1.960648 3.407223-1.338979C3.108344-.6934 2.558406-.119552 2.008468-.119552Z'/>
<path id='g1-99' d='M4.674471-4.495143C4.447323-4.495143 4.339726-4.495143 4.172354-4.351681C4.100623-4.291905 3.969116-4.112578 3.969116-3.921295C3.969116-3.682192 4.148443-3.53873 4.375592-3.53873C4.662516-3.53873 4.985305-3.777833 4.985305-4.25604C4.985305-4.829888 4.435367-5.272229 3.610461-5.272229C2.044334-5.272229 .478207-3.56264 .478207-1.865006C.478207-.824907 1.123786 .119552 2.343213 .119552C3.969116 .119552 4.99726-1.147696 4.99726-1.303113C4.99726-1.374844 4.925529-1.43462 4.877709-1.43462C4.841843-1.43462 4.829888-1.422665 4.722291-1.315068C3.957161-.298879 2.82142-.119552 2.367123-.119552C1.542217-.119552 1.279203-.836862 1.279203-1.43462C1.279203-1.853051 1.482441-3.012702 1.912827-3.825654C2.223661-4.387547 2.86924-5.033126 3.622416-5.033126C3.777833-5.033126 4.435367-5.009215 4.674471-4.495143Z'/>
<path id='g1-100' d='M6.01345-7.998007C6.025405-8.045828 6.049315-8.117559 6.049315-8.177335C6.049315-8.296887 5.929763-8.296887 5.905853-8.296887C5.893898-8.296887 5.308095-8.249066 5.248319-8.237111C5.045081-8.225156 4.865753-8.201245 4.65056-8.18929C4.351681-8.16538 4.267995-8.153425 4.267995-7.938232C4.267995-7.81868 4.363636-7.81868 4.531009-7.81868C5.116812-7.81868 5.128767-7.711083 5.128767-7.591532C5.128767-7.519801 5.104857-7.424159 5.092902-7.388294L4.363636-4.483188C4.23213-4.794022 3.90934-5.272229 3.287671-5.272229C1.936737-5.272229 .478207-3.526775 .478207-1.75741C.478207-.573848 1.171606 .119552 1.984558 .119552C2.642092 .119552 3.203985-.394521 3.53873-.789041C3.658281-.083686 4.220174 .119552 4.578829 .119552S5.224408-.095641 5.439601-.526027C5.630884-.932503 5.798257-1.661768 5.798257-1.709589C5.798257-1.769365 5.750436-1.817186 5.678705-1.817186C5.571108-1.817186 5.559153-1.75741 5.511333-1.578082C5.332005-.872727 5.104857-.119552 4.614695-.119552C4.267995-.119552 4.244085-.430386 4.244085-.669489C4.244085-.71731 4.244085-.968369 4.327771-1.303113L6.01345-7.998007ZM3.598506-1.422665C3.53873-1.219427 3.53873-1.195517 3.371357-.968369C3.108344-.633624 2.582316-.119552 2.020423-.119552C1.530262-.119552 1.255293-.561893 1.255293-1.267248C1.255293-1.924782 1.625903-3.263761 1.853051-3.765878C2.259527-4.60274 2.82142-5.033126 3.287671-5.033126C4.076712-5.033126 4.23213-4.052802 4.23213-3.957161C4.23213-3.945205 4.196264-3.789788 4.184309-3.765878L3.598506-1.422665Z'/>
<path id='g1-101' d='M2.139975-2.773599C2.462765-2.773599 3.275716-2.797509 3.849564-3.012702C4.758157-3.359402 4.841843-4.052802 4.841843-4.267995C4.841843-4.794022 4.387547-5.272229 3.598506-5.272229C2.343213-5.272229 .537983-4.136488 .537983-2.008468C.537983-.753176 1.255293 .119552 2.343213 .119552C3.969116 .119552 4.99726-1.147696 4.99726-1.303113C4.99726-1.374844 4.925529-1.43462 4.877709-1.43462C4.841843-1.43462 4.829888-1.422665 4.722291-1.315068C3.957161-.298879 2.82142-.119552 2.367123-.119552C1.685679-.119552 1.327024-.657534 1.327024-1.542217C1.327024-1.709589 1.327024-2.008468 1.506351-2.773599H2.139975ZM1.566127-3.012702C2.080199-4.853798 3.21594-5.033126 3.598506-5.033126C4.124533-5.033126 4.483188-4.722291 4.483188-4.267995C4.483188-3.012702 2.570361-3.012702 2.068244-3.012702H1.566127Z'/>
<path id='g1-103' d='M4.040847-1.518306C3.993026-1.327024 3.969116-1.279203 3.813699-1.099875C3.323537-.466252 2.82142-.239103 2.450809-.239103C2.056289-.239103 1.685679-.549938 1.685679-1.374844C1.685679-2.008468 2.044334-3.347447 2.307347-3.88543C2.654047-4.554919 3.19203-5.033126 3.694147-5.033126C4.483188-5.033126 4.638605-4.052802 4.638605-3.981071L4.60274-3.813699L4.040847-1.518306ZM4.782067-4.483188C4.62665-4.829888 4.291905-5.272229 3.694147-5.272229C2.391034-5.272229 .908593-3.634371 .908593-1.853051C.908593-.609714 1.661768 0 2.426899 0C3.060523 0 3.622416-.502117 3.837609-.74122L3.574595 .334745C3.407223 .992279 3.335492 1.291158 2.905106 1.709589C2.414944 2.199751 1.960648 2.199751 1.697634 2.199751C1.338979 2.199751 1.0401 2.175841 .74122 2.080199C1.123786 1.972603 1.219427 1.637858 1.219427 1.506351C1.219427 1.315068 1.075965 1.123786 .812951 1.123786C.526027 1.123786 .215193 1.362889 .215193 1.75741C.215193 2.247572 .705355 2.438854 1.721544 2.438854C3.263761 2.438854 4.064757 1.446575 4.220174 .800996L5.547198-4.554919C5.583064-4.698381 5.583064-4.722291 5.583064-4.746202C5.583064-4.913574 5.451557-5.045081 5.272229-5.045081C4.985305-5.045081 4.817933-4.805978 4.782067-4.483188Z'/>
<path id='g1-104' d='M3.359402-7.998007C3.371357-8.045828 3.395268-8.117559 3.395268-8.177335C3.395268-8.296887 3.275716-8.296887 3.251806-8.296887C3.239851-8.296887 2.654047-8.249066 2.594271-8.237111C2.391034-8.225156 2.211706-8.201245 1.996513-8.18929C1.697634-8.16538 1.613948-8.153425 1.613948-7.938232C1.613948-7.81868 1.709589-7.81868 1.876961-7.81868C2.462765-7.81868 2.47472-7.711083 2.47472-7.591532C2.47472-7.519801 2.450809-7.424159 2.438854-7.388294L.705355-.466252C.657534-.286924 .657534-.263014 .657534-.191283C.657534 .071731 .860772 .119552 .980324 .119552C1.183562 .119552 1.338979-.035866 1.398755-.167372L1.936737-2.331258C1.996513-2.594271 2.068244-2.84533 2.12802-3.108344C2.259527-3.610461 2.259527-3.622416 2.486675-3.969116S3.251806-5.033126 4.172354-5.033126C4.65056-5.033126 4.817933-4.674471 4.817933-4.196264C4.817933-3.526775 4.351681-2.223661 4.088667-1.506351C3.981071-1.219427 3.921295-1.06401 3.921295-.848817C3.921295-.310834 4.291905 .119552 4.865753 .119552C5.977584 .119552 6.396015-1.637858 6.396015-1.709589C6.396015-1.769365 6.348194-1.817186 6.276463-1.817186C6.168867-1.817186 6.156912-1.78132 6.097136-1.578082C5.822167-.621669 5.379826-.119552 4.901619-.119552C4.782067-.119552 4.590785-.131507 4.590785-.514072C4.590785-.824907 4.734247-1.207472 4.782067-1.338979C4.99726-1.912827 5.535243-3.323537 5.535243-4.016936C5.535243-4.734247 5.116812-5.272229 4.208219-5.272229C3.526775-5.272229 2.929016-4.94944 2.438854-4.327771L3.359402-7.998007Z'/>
<path id='g1-105' d='M3.383313-1.709589C3.383313-1.769365 3.335492-1.817186 3.263761-1.817186C3.156164-1.817186 3.144209-1.78132 3.084433-1.578082C2.773599-.490162 2.283437-.119552 1.888917-.119552C1.745455-.119552 1.578082-.155417 1.578082-.514072C1.578082-.836862 1.721544-1.195517 1.853051-1.554172L2.689913-3.777833C2.725778-3.873474 2.809465-4.088667 2.809465-4.315816C2.809465-4.817933 2.450809-5.272229 1.865006-5.272229C.765131-5.272229 .32279-3.53873 .32279-3.443088C.32279-3.395268 .37061-3.335492 .454296-3.335492C.561893-3.335492 .573848-3.383313 .621669-3.550685C.908593-4.554919 1.362889-5.033126 1.829141-5.033126C1.936737-5.033126 2.139975-5.021171 2.139975-4.638605C2.139975-4.327771 1.984558-3.93325 1.888917-3.670237L1.052055-1.446575C.980324-1.255293 .908593-1.06401 .908593-.848817C.908593-.310834 1.279203 .119552 1.853051 .119552C2.952927 .119552 3.383313-1.625903 3.383313-1.709589ZM3.287671-7.460025C3.287671-7.639352 3.144209-7.854545 2.881196-7.854545C2.606227-7.854545 2.295392-7.591532 2.295392-7.280697C2.295392-6.981818 2.546451-6.886177 2.689913-6.886177C3.012702-6.886177 3.287671-7.197011 3.287671-7.460025Z'/>
<path id='g1-108' d='M3.036613-7.998007C3.048568-8.045828 3.072478-8.117559 3.072478-8.177335C3.072478-8.296887 2.952927-8.296887 2.929016-8.296887C2.917061-8.296887 2.486675-8.261021 2.271482-8.237111C2.068244-8.225156 1.888917-8.201245 1.673724-8.18929C1.3868-8.16538 1.303113-8.153425 1.303113-7.938232C1.303113-7.81868 1.422665-7.81868 1.542217-7.81868C2.15193-7.81868 2.15193-7.711083 2.15193-7.591532C2.15193-7.543711 2.15193-7.519801 2.092154-7.304608L.609714-1.374844C.573848-1.243337 .549938-1.147696 .549938-.956413C.549938-.358655 .992279 .119552 1.601993 .119552C1.996513 .119552 2.259527-.143462 2.450809-.514072C2.654047-.908593 2.82142-1.661768 2.82142-1.709589C2.82142-1.769365 2.773599-1.817186 2.701868-1.817186C2.594271-1.817186 2.582316-1.75741 2.534496-1.578082C2.319303-.753176 2.10411-.119552 1.625903-.119552C1.267248-.119552 1.267248-.502117 1.267248-.669489C1.267248-.71731 1.267248-.968369 1.350934-1.303113L3.036613-7.998007Z'/>
<path id='g1-109' d='M2.462765-3.502864C2.486675-3.574595 2.785554-4.172354 3.227895-4.554919C3.53873-4.841843 3.945205-5.033126 4.411457-5.033126C4.889664-5.033126 5.057036-4.674471 5.057036-4.196264C5.057036-4.124533 5.057036-3.88543 4.913574-3.323537L4.614695-2.092154C4.519054-1.733499 4.291905-.848817 4.267995-.71731C4.220174-.537983 4.148443-.227148 4.148443-.179328C4.148443-.011955 4.27995 .119552 4.459278 .119552C4.817933 .119552 4.877709-.155417 4.985305-.585803L5.702615-3.443088C5.726526-3.53873 6.348194-5.033126 7.663263-5.033126C8.141469-5.033126 8.308842-4.674471 8.308842-4.196264C8.308842-3.526775 7.84259-2.223661 7.579577-1.506351C7.47198-1.219427 7.412204-1.06401 7.412204-.848817C7.412204-.310834 7.782814 .119552 8.356663 .119552C9.468493 .119552 9.886924-1.637858 9.886924-1.709589C9.886924-1.769365 9.839103-1.817186 9.767372-1.817186C9.659776-1.817186 9.647821-1.78132 9.588045-1.578082C9.313076-.621669 8.870735-.119552 8.392528-.119552C8.272976-.119552 8.081694-.131507 8.081694-.514072C8.081694-.824907 8.225156-1.207472 8.272976-1.338979C8.488169-1.912827 9.026152-3.323537 9.026152-4.016936C9.026152-4.734247 8.607721-5.272229 7.699128-5.272229C6.898132-5.272229 6.252553-4.817933 5.774346-4.112578C5.738481-4.758157 5.34396-5.272229 4.447323-5.272229C3.383313-5.272229 2.82142-4.519054 2.606227-4.220174C2.570361-4.901619 2.080199-5.272229 1.554172-5.272229C1.207472-5.272229 .932503-5.104857 .705355-4.65056C.490162-4.220174 .32279-3.490909 .32279-3.443088S.37061-3.335492 .454296-3.335492C.549938-3.335492 .561893-3.347447 .633624-3.622416C.812951-4.327771 1.0401-5.033126 1.518306-5.033126C1.793275-5.033126 1.888917-4.841843 1.888917-4.483188C1.888917-4.220174 1.769365-3.753923 1.685679-3.383313L1.350934-2.092154C1.303113-1.865006 1.171606-1.327024 1.111831-1.111831C1.028144-.800996 .896638-.239103 .896638-.179328C.896638-.011955 1.028144 .119552 1.207472 .119552C1.350934 .119552 1.518306 .047821 1.613948-.131507C1.637858-.191283 1.745455-.609714 1.80523-.848817L2.068244-1.924782L2.462765-3.502864Z'/>
<path id='g1-110' d='M2.462765-3.502864C2.486675-3.574595 2.785554-4.172354 3.227895-4.554919C3.53873-4.841843 3.945205-5.033126 4.411457-5.033126C4.889664-5.033126 5.057036-4.674471 5.057036-4.196264C5.057036-3.514819 4.566874-2.15193 4.327771-1.506351C4.220174-1.219427 4.160399-1.06401 4.160399-.848817C4.160399-.310834 4.531009 .119552 5.104857 .119552C6.216687 .119552 6.635118-1.637858 6.635118-1.709589C6.635118-1.769365 6.587298-1.817186 6.515567-1.817186C6.40797-1.817186 6.396015-1.78132 6.336239-1.578082C6.06127-.597758 5.606974-.119552 5.140722-.119552C5.021171-.119552 4.829888-.131507 4.829888-.514072C4.829888-.812951 4.961395-1.171606 5.033126-1.338979C5.272229-1.996513 5.774346-3.335492 5.774346-4.016936C5.774346-4.734247 5.355915-5.272229 4.447323-5.272229C3.383313-5.272229 2.82142-4.519054 2.606227-4.220174C2.570361-4.901619 2.080199-5.272229 1.554172-5.272229C1.171606-5.272229 .908593-5.045081 .705355-4.638605C.490162-4.208219 .32279-3.490909 .32279-3.443088S.37061-3.335492 .454296-3.335492C.549938-3.335492 .561893-3.347447 .633624-3.622416C.824907-4.351681 1.0401-5.033126 1.518306-5.033126C1.793275-5.033126 1.888917-4.841843 1.888917-4.483188C1.888917-4.220174 1.769365-3.753923 1.685679-3.383313L1.350934-2.092154C1.303113-1.865006 1.171606-1.327024 1.111831-1.111831C1.028144-.800996 .896638-.239103 .896638-.179328C.896638-.011955 1.028144 .119552 1.207472 .119552C1.350934 .119552 1.518306 .047821 1.613948-.131507C1.637858-.191283 1.745455-.609714 1.80523-.848817L2.068244-1.924782L2.462765-3.502864Z'/>
<path id='g1-111' d='M5.451557-3.287671C5.451557-4.423412 4.710336-5.272229 3.622416-5.272229C2.044334-5.272229 .490162-3.550685 .490162-1.865006C.490162-.729265 1.231382 .119552 2.319303 .119552C3.90934 .119552 5.451557-1.601993 5.451557-3.287671ZM2.331258-.119552C1.733499-.119552 1.291158-.597758 1.291158-1.43462C1.291158-1.984558 1.578082-3.203985 1.912827-3.801743C2.450809-4.722291 3.120299-5.033126 3.610461-5.033126C4.196264-5.033126 4.65056-4.554919 4.65056-3.718057C4.65056-3.239851 4.399502-1.960648 3.945205-1.231382C3.455044-.430386 2.797509-.119552 2.331258-.119552Z'/>
<path id='g1-114' d='M4.65056-4.889664C4.27995-4.817933 4.088667-4.554919 4.088667-4.291905C4.088667-4.004981 4.315816-3.90934 4.483188-3.90934C4.817933-3.90934 5.092902-4.196264 5.092902-4.554919C5.092902-4.937484 4.722291-5.272229 4.124533-5.272229C3.646326-5.272229 3.096389-5.057036 2.594271-4.327771C2.510585-4.961395 2.032379-5.272229 1.554172-5.272229C1.08792-5.272229 .848817-4.913574 .705355-4.65056C.502117-4.220174 .32279-3.502864 .32279-3.443088C.32279-3.395268 .37061-3.335492 .454296-3.335492C.549938-3.335492 .561893-3.347447 .633624-3.622416C.812951-4.339726 1.0401-5.033126 1.518306-5.033126C1.80523-5.033126 1.888917-4.829888 1.888917-4.483188C1.888917-4.220174 1.769365-3.753923 1.685679-3.383313L1.350934-2.092154C1.303113-1.865006 1.171606-1.327024 1.111831-1.111831C1.028144-.800996 .896638-.239103 .896638-.179328C.896638-.011955 1.028144 .119552 1.207472 .119552C1.338979 .119552 1.566127 .035866 1.637858-.203238C1.673724-.298879 2.116065-2.10411 2.187796-2.379078C2.247572-2.642092 2.319303-2.893151 2.379078-3.156164C2.426899-3.323537 2.47472-3.514819 2.510585-3.670237C2.546451-3.777833 2.86924-4.363636 3.16812-4.62665C3.311582-4.758157 3.622416-5.033126 4.112578-5.033126C4.303861-5.033126 4.495143-4.99726 4.65056-4.889664Z'/>
<path id='g1-116' d='M2.402989-4.805978H3.502864C3.730012-4.805978 3.849564-4.805978 3.849564-5.021171C3.849564-5.152677 3.777833-5.152677 3.53873-5.152677H2.486675L2.929016-6.898132C2.976837-7.065504 2.976837-7.089415 2.976837-7.173101C2.976837-7.364384 2.82142-7.47198 2.666002-7.47198C2.570361-7.47198 2.295392-7.436115 2.199751-7.053549L1.733499-5.152677H.609714C.37061-5.152677 .263014-5.152677 .263014-4.925529C.263014-4.805978 .3467-4.805978 .573848-4.805978H1.637858L.848817-1.649813C.753176-1.231382 .71731-1.111831 .71731-.956413C.71731-.394521 1.111831 .119552 1.78132 .119552C2.988792 .119552 3.634371-1.625903 3.634371-1.709589C3.634371-1.78132 3.58655-1.817186 3.514819-1.817186C3.490909-1.817186 3.443088-1.817186 3.419178-1.769365C3.407223-1.75741 3.395268-1.745455 3.311582-1.554172C3.060523-.956413 2.510585-.119552 1.817186-.119552C1.458531-.119552 1.43462-.418431 1.43462-.681445C1.43462-.6934 1.43462-.920548 1.470486-1.06401L2.402989-4.805978Z'/>
<path id='g1-121' d='M3.144209 1.338979C2.82142 1.793275 2.355168 2.199751 1.769365 2.199751C1.625903 2.199751 1.052055 2.175841 .872727 1.625903C.908593 1.637858 .968369 1.637858 .992279 1.637858C1.350934 1.637858 1.590037 1.327024 1.590037 1.052055S1.362889 .681445 1.183562 .681445C.992279 .681445 .573848 .824907 .573848 1.41071C.573848 2.020423 1.08792 2.438854 1.769365 2.438854C2.964882 2.438854 4.172354 1.338979 4.507098 .011955L5.678705-4.65056C5.69066-4.710336 5.71457-4.782067 5.71457-4.853798C5.71457-5.033126 5.571108-5.152677 5.391781-5.152677C5.284184-5.152677 5.033126-5.104857 4.937484-4.746202L4.052802-1.231382C3.993026-1.016189 3.993026-.992279 3.897385-.860772C3.658281-.526027 3.263761-.119552 2.689913-.119552C2.020423-.119552 1.960648-.777086 1.960648-1.099875C1.960648-1.78132 2.283437-2.701868 2.606227-3.56264C2.737733-3.90934 2.809465-4.076712 2.809465-4.315816C2.809465-4.817933 2.450809-5.272229 1.865006-5.272229C.765131-5.272229 .32279-3.53873 .32279-3.443088C.32279-3.395268 .37061-3.335492 .454296-3.335492C.561893-3.335492 .573848-3.383313 .621669-3.550685C.908593-4.554919 1.362889-5.033126 1.829141-5.033126C1.936737-5.033126 2.139975-5.033126 2.139975-4.638605C2.139975-4.327771 2.008468-3.981071 1.829141-3.526775C1.243337-1.960648 1.243337-1.566127 1.243337-1.279203C1.243337-.143462 2.056289 .119552 2.654047 .119552C3.000747 .119552 3.431133 .011955 3.849564-.430386L3.861519-.418431C3.682192 .286924 3.56264 .753176 3.144209 1.338979Z'/>
</defs>
<g id='page1' transform='matrix(1.13 0 0 1.13 -80.23 -68.28816)'>
<use x='70.734745' y='68.742217' xlink:href='#g1-109'/>
<use x='80.974012' y='68.742217' xlink:href='#g1-101'/>
<use x='86.399452' y='68.742217' xlink:href='#g1-109'/>
<use x='96.638719' y='68.742217' xlink:href='#g1-111'/>
<use x='102.266156' y='68.742217' xlink:href='#g1-114'/>
<use x='107.86663' y='68.742217' xlink:href='#g1-121'/>
<use x='114.003281' y='68.742217' xlink:href='#g1-58'/>
<use x='117.254943' y='68.742217' xlink:href='#g1-104'/>
<use x='123.993498' y='68.742217' xlink:href='#g1-105'/>
<use x='127.98693' y='68.742217' xlink:href='#g1-103'/>
<use x='134.021186' y='68.742217' xlink:href='#g1-104'/>
<use x='144.08057' y='68.742217' xlink:href='#g2-61'/>
<use x='156.506051' y='68.742217' xlink:href='#g1-77'/>
<use x='169.079658' y='68.742217' xlink:href='#g1-101'/>
<use x='174.505098' y='68.742217' xlink:href='#g1-109'/>
<use x='184.744365' y='68.742217' xlink:href='#g1-111'/>
<use x='190.371803' y='68.742217' xlink:href='#g1-114'/>
<use x='195.972276' y='68.742217' xlink:href='#g1-121'/>
<use x='202.108928' y='68.742217' xlink:href='#g1-84'/>
<use x='210.595764' y='68.742217' xlink:href='#g1-104'/>
<use x='217.334319' y='68.742217' xlink:href='#g1-114'/>
<use x='222.934792' y='68.742217' xlink:href='#g1-111'/>
<use x='228.56223' y='68.742217' xlink:href='#g1-116'/>
<use x='232.789389' y='68.742217' xlink:href='#g1-116'/>
<use x='237.016549' y='68.742217' xlink:href='#g1-105'/>
<use x='241.009981' y='68.742217' xlink:href='#g1-108'/>
<use x='244.75979' y='68.742217' xlink:href='#g1-110'/>
<use x='251.747395' y='68.742217' xlink:href='#g1-103'/>
<use x='257.781652' y='68.742217' xlink:href='#g1-70'/>
<use x='266.985267' y='68.742217' xlink:href='#g1-97'/>
<use x='273.130211' y='68.742217' xlink:href='#g1-99'/>
<use x='278.1682' y='68.742217' xlink:href='#g1-116'/>
<use x='282.395359' y='68.742217' xlink:href='#g1-111'/>
<use x='288.022797' y='68.742217' xlink:href='#g1-114'/>
<use x='296.224599' y='68.742217' xlink:href='#g0-3'/>
<use x='304.803535' y='68.742217' xlink:href='#g1-78'/>
<use x='315.426145' y='68.742217' xlink:href='#g1-111'/>
<use x='321.053583' y='68.742217' xlink:href='#g1-100'/>
<use x='327.136276' y='68.742217' xlink:href='#g1-101'/>
<use x='332.561716' y='68.742217' xlink:href='#g1-65'/>
<use x='341.337063' y='68.742217' xlink:href='#g1-108'/>
<use x='345.086871' y='68.742217' xlink:href='#g1-108'/>
<use x='348.83668' y='68.742217' xlink:href='#g1-111'/>
<use x='354.464117' y='68.742217' xlink:href='#g1-99'/>
<use x='359.502106' y='68.742217' xlink:href='#g1-97'/>
<use x='365.64705' y='68.742217' xlink:href='#g1-116'/>
<use x='369.87421' y='68.742217' xlink:href='#g1-97'/>
<use x='376.019154' y='68.742217' xlink:href='#g1-98'/>
<use x='380.996259' y='68.742217' xlink:href='#g1-108'/>
<use x='384.746068' y='68.742217' xlink:href='#g1-101'/>
<use x='390.171508' y='68.742217' xlink:href='#g1-77'/>
<use x='402.745115' y='68.742217' xlink:href='#g1-101'/>
<use x='408.170555' y='68.742217' xlink:href='#g1-109'/>
<use x='418.409822' y='68.742217' xlink:href='#g1-111'/>
<use x='424.03726' y='68.742217' xlink:href='#g1-114'/>
<use x='429.637733' y='68.742217' xlink:href='#g1-121'/>
</g>
</svg>
<svg
version="1.1"
width="412.017841pt"
height="12.401574pt"
viewBox="-.299738 -.255124 412.017841 12.401574"
id="svg56"
sodipodi:docname="container-memory-high-best-effort.svg"
inkscape:version="1.3-beta (cedbd6c6ff, 2023-05-28)"
xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
xmlns:xlink="http://www.w3.org/1999/xlink"
xmlns="http://www.w3.org/2000/svg"
xmlns:svg="http://www.w3.org/2000/svg">
<sodipodi:namedview
id="namedview56"
pagecolor="#ffffff"
bordercolor="#000000"
borderopacity="0.25"
inkscape:showpageshadow="2"
inkscape:pageopacity="0.0"
inkscape:pagecheckerboard="0"
inkscape:deskcolor="#d1d1d1"
inkscape:document-units="pt"
showgrid="false"
inkscape:zoom="2.8287609"
inkscape:cx="203.97623"
inkscape:cy="-13.610199"
inkscape:window-width="1920"
inkscape:window-height="1137"
inkscape:window-x="-8"
inkscape:window-y="-8"
inkscape:window-maximized="1"
inkscape:current-layer="page1" />
<defs
id="defs1">
<path
id="g2-61"
d="M8.069738-3.873474C8.237111-3.873474 8.452304-3.873474 8.452304-4.088667C8.452304-4.315816 8.249066-4.315816 8.069738-4.315816H1.028144C.860772-4.315816 .645579-4.315816 .645579-4.100623C.645579-3.873474 .848817-3.873474 1.028144-3.873474H8.069738ZM8.069738-1.649813C8.237111-1.649813 8.452304-1.649813 8.452304-1.865006C8.452304-2.092154 8.249066-2.092154 8.069738-2.092154H1.028144C.860772-2.092154 .645579-2.092154 .645579-1.876961C.645579-1.649813 .848817-1.649813 1.028144-1.649813H8.069738Z" />
<path
id="g0-3"
d="M3.287671-5.104857C3.299626-5.272229 3.299626-5.559153 2.988792-5.559153C2.797509-5.559153 2.642092-5.403736 2.677958-5.248319V-5.092902L2.84533-3.239851L1.315068-4.351681C1.207472-4.411457 1.183562-4.435367 1.099875-4.435367C.932503-4.435367 .777086-4.267995 .777086-4.100623C.777086-3.90934 .896638-3.861519 1.016189-3.801743L2.713823-2.988792L1.06401-2.187796C.872727-2.092154 .777086-2.044334 .777086-1.865006S.932503-1.530262 1.099875-1.530262C1.183562-1.530262 1.207472-1.530262 1.506351-1.75741L2.84533-2.725778L2.666002-.71731C2.666002-.466252 2.881196-.406476 2.976837-.406476C3.120299-.406476 3.299626-.490162 3.299626-.71731L3.120299-2.725778L4.65056-1.613948C4.758157-1.554172 4.782067-1.530262 4.865753-1.530262C5.033126-1.530262 5.188543-1.697634 5.188543-1.865006C5.188543-2.044334 5.080946-2.10411 4.937484-2.175841C4.220174-2.534496 4.196264-2.534496 3.251806-2.976837L4.901619-3.777833C5.092902-3.873474 5.188543-3.921295 5.188543-4.100623S5.033126-4.435367 4.865753-4.435367C4.782067-4.435367 4.758157-4.435367 4.459278-4.208219L3.120299-3.239851L3.287671-5.104857Z" />
<path
id="g1-58"
d="M2.199751-.573848C2.199751-.920548 1.912827-1.159651 1.625903-1.159651C1.279203-1.159651 1.0401-.872727 1.0401-.585803C1.0401-.239103 1.327024 0 1.613948 0C1.960648 0 2.199751-.286924 2.199751-.573848Z" />
<path
id="g1-65"
d="M2.032379-1.327024C1.613948-.621669 1.207472-.382565 .633624-.3467C.502117-.334745 .406476-.334745 .406476-.119552C.406476-.047821 .466252 0 .549938 0C.765131 0 1.303113-.02391 1.518306-.02391C1.865006-.02391 2.247572 0 2.582316 0C2.654047 0 2.797509 0 2.797509-.227148C2.797509-.334745 2.701868-.3467 2.630137-.3467C2.355168-.37061 2.12802-.466252 2.12802-.753176C2.12802-.920548 2.199751-1.052055 2.355168-1.315068L3.263761-2.82142H6.312329C6.324284-2.713823 6.324284-2.618182 6.336239-2.510585C6.372105-2.199751 6.515567-.956413 6.515567-.729265C6.515567-.37061 5.905853-.3467 5.71457-.3467C5.583064-.3467 5.451557-.3467 5.451557-.131507C5.451557 0 5.559153 0 5.630884 0C5.834122 0 6.073225-.02391 6.276463-.02391H6.957908C7.687173-.02391 8.2132 0 8.225156 0C8.308842 0 8.440349 0 8.440349-.227148C8.440349-.3467 8.332752-.3467 8.153425-.3467C7.49589-.3467 7.483935-.454296 7.44807-.812951L6.718804-8.272976C6.694894-8.51208 6.647073-8.53599 6.515567-8.53599C6.396015-8.53599 6.324284-8.51208 6.216687-8.332752L2.032379-1.327024ZM3.466999-3.16812L5.869988-7.185056L6.276463-3.16812H3.466999Z" />
<path
id="g1-70"
d="M3.550685-3.897385H4.698381C5.606974-3.897385 5.678705-3.694147 5.678705-3.347447C5.678705-3.19203 5.654795-3.024658 5.595019-2.761644C5.571108-2.713823 5.559153-2.654047 5.559153-2.630137C5.559153-2.546451 5.606974-2.49863 5.69066-2.49863C5.786301-2.49863 5.798257-2.546451 5.846077-2.737733L6.539477-5.523288C6.539477-5.571108 6.503611-5.642839 6.419925-5.642839C6.312329-5.642839 6.300374-5.595019 6.252553-5.391781C6.001494-4.495143 5.762391-4.244085 4.722291-4.244085H3.634371L4.411457-7.340473C4.519054-7.758904 4.542964-7.79477 5.033126-7.79477H6.635118C8.129514-7.79477 8.344707-7.352428 8.344707-6.503611C8.344707-6.43188 8.344707-6.168867 8.308842-5.858032C8.296887-5.810212 8.272976-5.654795 8.272976-5.606974C8.272976-5.511333 8.332752-5.475467 8.404483-5.475467C8.488169-5.475467 8.53599-5.523288 8.5599-5.738481L8.810959-7.830635C8.810959-7.866501 8.834869-7.986052 8.834869-8.009963C8.834869-8.141469 8.727273-8.141469 8.51208-8.141469H2.84533C2.618182-8.141469 2.49863-8.141469 2.49863-7.926276C2.49863-7.79477 2.582316-7.79477 2.785554-7.79477C3.526775-7.79477 3.526775-7.711083 3.526775-7.579577C3.526775-7.519801 3.514819-7.47198 3.478954-7.340473L1.865006-.884682C1.75741-.466252 1.733499-.3467 .896638-.3467C.669489-.3467 .549938-.3467 .549938-.131507C.549938 0 .657534 0 .729265 0C.956413 0 1.195517-.02391 1.422665-.02391H2.976837C3.239851-.02391 3.526775 0 3.789788 0C3.897385 0 4.040847 0 4.040847-.215193C4.040847-.3467 3.969116-.3467 3.706102-.3467C2.761644-.3467 2.737733-.430386 2.737733-.609714C2.737733-.669489 2.761644-.765131 2.785554-.848817L3.550685-3.897385Z" />
<path
id="g1-77"
d="M10.855293-7.292653C10.962889-7.699128 10.9868-7.81868 11.835616-7.81868C12.062765-7.81868 12.170361-7.81868 12.170361-8.045828C12.170361-8.16538 12.086675-8.16538 11.859527-8.16538H10.424907C10.126027-8.16538 10.114072-8.153425 9.982565-7.962142L5.618929-1.06401L4.722291-7.902366C4.686426-8.16538 4.674471-8.16538 4.363636-8.16538H2.881196C2.654047-8.16538 2.546451-8.16538 2.546451-7.938232C2.546451-7.81868 2.654047-7.81868 2.833375-7.81868C3.56264-7.81868 3.56264-7.723039 3.56264-7.591532C3.56264-7.567621 3.56264-7.49589 3.514819-7.316563L1.984558-1.219427C1.841096-.645579 1.566127-.382565 .765131-.3467C.729265-.3467 .585803-.334745 .585803-.131507C.585803 0 .6934 0 .74122 0C.980324 0 1.590037-.02391 1.829141-.02391H2.402989C2.570361-.02391 2.773599 0 2.940971 0C3.024658 0 3.156164 0 3.156164-.227148C3.156164-.334745 3.036613-.3467 2.988792-.3467C2.594271-.358655 2.211706-.430386 2.211706-.860772C2.211706-.980324 2.211706-.992279 2.259527-1.159651L3.90934-7.746949H3.921295L4.913574-.32279C4.94944-.035866 4.961395 0 5.068991 0C5.200498 0 5.260274-.095641 5.32005-.203238L10.126027-7.806725H10.137983L8.404483-.884682C8.296887-.466252 8.272976-.3467 7.436115-.3467C7.208966-.3467 7.089415-.3467 7.089415-.131507C7.089415 0 7.197011 0 7.268742 0C7.47198 0 7.711083-.02391 7.914321-.02391H9.325031C9.528269-.02391 9.779328 0 9.982565 0C10.078207 0 10.209714 0 10.209714-.227148C10.209714-.3467 10.102117-.3467 9.92279-.3467C9.193524-.3467 9.193524-.442341 9.193524-.561893C9.193524-.573848 9.193524-.657534 9.217435-.753176L10.855293-7.292653Z" />
<path
id="g1-78"
d="M8.846824-6.910087C8.978331-7.424159 9.169614-7.782814 10.078207-7.81868C10.114072-7.81868 10.257534-7.830635 10.257534-8.033873C10.257534-8.16538 10.149938-8.16538 10.102117-8.16538C9.863014-8.16538 9.2533-8.141469 9.014197-8.141469H8.440349C8.272976-8.141469 8.057783-8.16538 7.890411-8.16538C7.81868-8.16538 7.675218-8.16538 7.675218-7.938232C7.675218-7.81868 7.770859-7.81868 7.854545-7.81868C8.571856-7.79477 8.619676-7.519801 8.619676-7.304608C8.619676-7.197011 8.607721-7.161146 8.571856-6.993773L7.220922-1.601993L4.662516-7.962142C4.578829-8.153425 4.566874-8.16538 4.303861-8.16538H2.84533C2.606227-8.16538 2.49863-8.16538 2.49863-7.938232C2.49863-7.81868 2.582316-7.81868 2.809465-7.81868C2.86924-7.81868 3.574595-7.81868 3.574595-7.711083C3.574595-7.687173 3.550685-7.591532 3.53873-7.555666L1.948692-1.219427C1.80523-.633624 1.518306-.382565 .729265-.3467C.669489-.3467 .549938-.334745 .549938-.119552C.549938 0 .669489 0 .705355 0C.944458 0 1.554172-.02391 1.793275-.02391H2.367123C2.534496-.02391 2.737733 0 2.905106 0C2.988792 0 3.120299 0 3.120299-.227148C3.120299-.334745 3.000747-.3467 2.952927-.3467C2.558406-.358655 2.175841-.430386 2.175841-.860772C2.175841-.956413 2.199751-1.06401 2.223661-1.159651L3.837609-7.555666C3.90934-7.436115 3.90934-7.412204 3.957161-7.304608L6.802491-.215193C6.862267-.071731 6.886177 0 6.993773 0C7.113325 0 7.12528-.035866 7.173101-.239103L8.846824-6.910087Z" />
<path
id="g1-84"
d="M4.985305-7.292653C5.057036-7.579577 5.080946-7.687173 5.260274-7.734994C5.355915-7.758904 5.750436-7.758904 6.001494-7.758904C7.197011-7.758904 7.758904-7.711083 7.758904-6.77858C7.758904-6.599253 7.711083-6.144956 7.639352-5.702615L7.627397-5.559153C7.627397-5.511333 7.675218-5.439601 7.746949-5.439601C7.866501-5.439601 7.866501-5.499377 7.902366-5.69066L8.249066-7.806725C8.272976-7.914321 8.272976-7.938232 8.272976-7.974097C8.272976-8.105604 8.201245-8.105604 7.962142-8.105604H1.422665C1.147696-8.105604 1.135741-8.093649 1.06401-7.878456L.334745-5.726526C.32279-5.702615 .286924-5.571108 .286924-5.559153C.286924-5.499377 .334745-5.439601 .406476-5.439601C.502117-5.439601 .526027-5.487422 .573848-5.642839C1.075965-7.089415 1.327024-7.758904 2.917061-7.758904H3.718057C4.004981-7.758904 4.124533-7.758904 4.124533-7.627397C4.124533-7.591532 4.124533-7.567621 4.064757-7.352428L2.462765-.932503C2.343213-.466252 2.319303-.3467 1.052055-.3467C.753176-.3467 .669489-.3467 .669489-.119552C.669489 0 .800996 0 .860772 0C1.159651 0 1.470486-.02391 1.769365-.02391H3.634371C3.93325-.02391 4.25604 0 4.554919 0C4.686426 0 4.805978 0 4.805978-.227148C4.805978-.3467 4.722291-.3467 4.411457-.3467C3.335492-.3467 3.335492-.454296 3.335492-.633624C3.335492-.645579 3.335492-.729265 3.383313-.920548L4.985305-7.292653Z" />
<path
id="g1-97"
d="M3.598506-1.422665C3.53873-1.219427 3.53873-1.195517 3.371357-.968369C3.108344-.633624 2.582316-.119552 2.020423-.119552C1.530262-.119552 1.255293-.561893 1.255293-1.267248C1.255293-1.924782 1.625903-3.263761 1.853051-3.765878C2.259527-4.60274 2.82142-5.033126 3.287671-5.033126C4.076712-5.033126 4.23213-4.052802 4.23213-3.957161C4.23213-3.945205 4.196264-3.789788 4.184309-3.765878L3.598506-1.422665ZM4.363636-4.483188C4.23213-4.794022 3.90934-5.272229 3.287671-5.272229C1.936737-5.272229 .478207-3.526775 .478207-1.75741C.478207-.573848 1.171606 .119552 1.984558 .119552C2.642092 .119552 3.203985-.394521 3.53873-.789041C3.658281-.083686 4.220174 .119552 4.578829 .119552S5.224408-.095641 5.439601-.526027C5.630884-.932503 5.798257-1.661768 5.798257-1.709589C5.798257-1.769365 5.750436-1.817186 5.678705-1.817186C5.571108-1.817186 5.559153-1.75741 5.511333-1.578082C5.332005-.872727 5.104857-.119552 4.614695-.119552C4.267995-.119552 4.244085-.430386 4.244085-.669489C4.244085-.944458 4.27995-1.075965 4.387547-1.542217C4.471233-1.841096 4.531009-2.10411 4.62665-2.450809C5.068991-4.244085 5.176588-4.674471 5.176588-4.746202C5.176588-4.913574 5.045081-5.045081 4.865753-5.045081C4.483188-5.045081 4.387547-4.62665 4.363636-4.483188Z" />
<path
id="g1-98"
d="M2.761644-7.998007C2.773599-8.045828 2.797509-8.117559 2.797509-8.177335C2.797509-8.296887 2.677958-8.296887 2.654047-8.296887C2.642092-8.296887 2.211706-8.261021 1.996513-8.237111C1.793275-8.225156 1.613948-8.201245 1.398755-8.18929C1.111831-8.16538 1.028144-8.153425 1.028144-7.938232C1.028144-7.81868 1.147696-7.81868 1.267248-7.81868C1.876961-7.81868 1.876961-7.711083 1.876961-7.591532C1.876961-7.507846 1.78132-7.161146 1.733499-6.945953L1.446575-5.798257C1.327024-5.32005 .645579-2.606227 .597758-2.391034C.537983-2.092154 .537983-1.888917 .537983-1.733499C.537983-.514072 1.219427 .119552 1.996513 .119552C3.383313 .119552 4.817933-1.661768 4.817933-3.395268C4.817933-4.495143 4.196264-5.272229 3.299626-5.272229C2.677958-5.272229 2.116065-4.758157 1.888917-4.519054L2.761644-7.998007ZM2.008468-.119552C1.625903-.119552 1.207472-.406476 1.207472-1.338979C1.207472-1.733499 1.243337-1.960648 1.458531-2.797509C1.494396-2.952927 1.685679-3.718057 1.733499-3.873474C1.75741-3.969116 2.462765-5.033126 3.275716-5.033126C3.801743-5.033126 4.040847-4.507098 4.040847-3.88543C4.040847-3.311582 3.706102-1.960648 3.407223-1.338979C3.108344-.6934 2.558406-.119552 2.008468-.119552Z" />
<path
id="g1-99"
d="M4.674471-4.495143C4.447323-4.495143 4.339726-4.495143 4.172354-4.351681C4.100623-4.291905 3.969116-4.112578 3.969116-3.921295C3.969116-3.682192 4.148443-3.53873 4.375592-3.53873C4.662516-3.53873 4.985305-3.777833 4.985305-4.25604C4.985305-4.829888 4.435367-5.272229 3.610461-5.272229C2.044334-5.272229 .478207-3.56264 .478207-1.865006C.478207-.824907 1.123786 .119552 2.343213 .119552C3.969116 .119552 4.99726-1.147696 4.99726-1.303113C4.99726-1.374844 4.925529-1.43462 4.877709-1.43462C4.841843-1.43462 4.829888-1.422665 4.722291-1.315068C3.957161-.298879 2.82142-.119552 2.367123-.119552C1.542217-.119552 1.279203-.836862 1.279203-1.43462C1.279203-1.853051 1.482441-3.012702 1.912827-3.825654C2.223661-4.387547 2.86924-5.033126 3.622416-5.033126C3.777833-5.033126 4.435367-5.009215 4.674471-4.495143Z" />
<path
id="g1-100"
d="M6.01345-7.998007C6.025405-8.045828 6.049315-8.117559 6.049315-8.177335C6.049315-8.296887 5.929763-8.296887 5.905853-8.296887C5.893898-8.296887 5.308095-8.249066 5.248319-8.237111C5.045081-8.225156 4.865753-8.201245 4.65056-8.18929C4.351681-8.16538 4.267995-8.153425 4.267995-7.938232C4.267995-7.81868 4.363636-7.81868 4.531009-7.81868C5.116812-7.81868 5.128767-7.711083 5.128767-7.591532C5.128767-7.519801 5.104857-7.424159 5.092902-7.388294L4.363636-4.483188C4.23213-4.794022 3.90934-5.272229 3.287671-5.272229C1.936737-5.272229 .478207-3.526775 .478207-1.75741C.478207-.573848 1.171606 .119552 1.984558 .119552C2.642092 .119552 3.203985-.394521 3.53873-.789041C3.658281-.083686 4.220174 .119552 4.578829 .119552S5.224408-.095641 5.439601-.526027C5.630884-.932503 5.798257-1.661768 5.798257-1.709589C5.798257-1.769365 5.750436-1.817186 5.678705-1.817186C5.571108-1.817186 5.559153-1.75741 5.511333-1.578082C5.332005-.872727 5.104857-.119552 4.614695-.119552C4.267995-.119552 4.244085-.430386 4.244085-.669489C4.244085-.71731 4.244085-.968369 4.327771-1.303113L6.01345-7.998007ZM3.598506-1.422665C3.53873-1.219427 3.53873-1.195517 3.371357-.968369C3.108344-.633624 2.582316-.119552 2.020423-.119552C1.530262-.119552 1.255293-.561893 1.255293-1.267248C1.255293-1.924782 1.625903-3.263761 1.853051-3.765878C2.259527-4.60274 2.82142-5.033126 3.287671-5.033126C4.076712-5.033126 4.23213-4.052802 4.23213-3.957161C4.23213-3.945205 4.196264-3.789788 4.184309-3.765878L3.598506-1.422665Z" />
<path
id="g1-101"
d="M2.139975-2.773599C2.462765-2.773599 3.275716-2.797509 3.849564-3.012702C4.758157-3.359402 4.841843-4.052802 4.841843-4.267995C4.841843-4.794022 4.387547-5.272229 3.598506-5.272229C2.343213-5.272229 .537983-4.136488 .537983-2.008468C.537983-.753176 1.255293 .119552 2.343213 .119552C3.969116 .119552 4.99726-1.147696 4.99726-1.303113C4.99726-1.374844 4.925529-1.43462 4.877709-1.43462C4.841843-1.43462 4.829888-1.422665 4.722291-1.315068C3.957161-.298879 2.82142-.119552 2.367123-.119552C1.685679-.119552 1.327024-.657534 1.327024-1.542217C1.327024-1.709589 1.327024-2.008468 1.506351-2.773599H2.139975ZM1.566127-3.012702C2.080199-4.853798 3.21594-5.033126 3.598506-5.033126C4.124533-5.033126 4.483188-4.722291 4.483188-4.267995C4.483188-3.012702 2.570361-3.012702 2.068244-3.012702H1.566127Z" />
<path
id="g1-103"
d="M4.040847-1.518306C3.993026-1.327024 3.969116-1.279203 3.813699-1.099875C3.323537-.466252 2.82142-.239103 2.450809-.239103C2.056289-.239103 1.685679-.549938 1.685679-1.374844C1.685679-2.008468 2.044334-3.347447 2.307347-3.88543C2.654047-4.554919 3.19203-5.033126 3.694147-5.033126C4.483188-5.033126 4.638605-4.052802 4.638605-3.981071L4.60274-3.813699L4.040847-1.518306ZM4.782067-4.483188C4.62665-4.829888 4.291905-5.272229 3.694147-5.272229C2.391034-5.272229 .908593-3.634371 .908593-1.853051C.908593-.609714 1.661768 0 2.426899 0C3.060523 0 3.622416-.502117 3.837609-.74122L3.574595 .334745C3.407223 .992279 3.335492 1.291158 2.905106 1.709589C2.414944 2.199751 1.960648 2.199751 1.697634 2.199751C1.338979 2.199751 1.0401 2.175841 .74122 2.080199C1.123786 1.972603 1.219427 1.637858 1.219427 1.506351C1.219427 1.315068 1.075965 1.123786 .812951 1.123786C.526027 1.123786 .215193 1.362889 .215193 1.75741C.215193 2.247572 .705355 2.438854 1.721544 2.438854C3.263761 2.438854 4.064757 1.446575 4.220174 .800996L5.547198-4.554919C5.583064-4.698381 5.583064-4.722291 5.583064-4.746202C5.583064-4.913574 5.451557-5.045081 5.272229-5.045081C4.985305-5.045081 4.817933-4.805978 4.782067-4.483188Z" />
<path
id="g1-104"
d="M3.359402-7.998007C3.371357-8.045828 3.395268-8.117559 3.395268-8.177335C3.395268-8.296887 3.275716-8.296887 3.251806-8.296887C3.239851-8.296887 2.654047-8.249066 2.594271-8.237111C2.391034-8.225156 2.211706-8.201245 1.996513-8.18929C1.697634-8.16538 1.613948-8.153425 1.613948-7.938232C1.613948-7.81868 1.709589-7.81868 1.876961-7.81868C2.462765-7.81868 2.47472-7.711083 2.47472-7.591532C2.47472-7.519801 2.450809-7.424159 2.438854-7.388294L.705355-.466252C.657534-.286924 .657534-.263014 .657534-.191283C.657534 .071731 .860772 .119552 .980324 .119552C1.183562 .119552 1.338979-.035866 1.398755-.167372L1.936737-2.331258C1.996513-2.594271 2.068244-2.84533 2.12802-3.108344C2.259527-3.610461 2.259527-3.622416 2.486675-3.969116S3.251806-5.033126 4.172354-5.033126C4.65056-5.033126 4.817933-4.674471 4.817933-4.196264C4.817933-3.526775 4.351681-2.223661 4.088667-1.506351C3.981071-1.219427 3.921295-1.06401 3.921295-.848817C3.921295-.310834 4.291905 .119552 4.865753 .119552C5.977584 .119552 6.396015-1.637858 6.396015-1.709589C6.396015-1.769365 6.348194-1.817186 6.276463-1.817186C6.168867-1.817186 6.156912-1.78132 6.097136-1.578082C5.822167-.621669 5.379826-.119552 4.901619-.119552C4.782067-.119552 4.590785-.131507 4.590785-.514072C4.590785-.824907 4.734247-1.207472 4.782067-1.338979C4.99726-1.912827 5.535243-3.323537 5.535243-4.016936C5.535243-4.734247 5.116812-5.272229 4.208219-5.272229C3.526775-5.272229 2.929016-4.94944 2.438854-4.327771L3.359402-7.998007Z" />
<path
id="g1-105"
d="M3.383313-1.709589C3.383313-1.769365 3.335492-1.817186 3.263761-1.817186C3.156164-1.817186 3.144209-1.78132 3.084433-1.578082C2.773599-.490162 2.283437-.119552 1.888917-.119552C1.745455-.119552 1.578082-.155417 1.578082-.514072C1.578082-.836862 1.721544-1.195517 1.853051-1.554172L2.689913-3.777833C2.725778-3.873474 2.809465-4.088667 2.809465-4.315816C2.809465-4.817933 2.450809-5.272229 1.865006-5.272229C.765131-5.272229 .32279-3.53873 .32279-3.443088C.32279-3.395268 .37061-3.335492 .454296-3.335492C.561893-3.335492 .573848-3.383313 .621669-3.550685C.908593-4.554919 1.362889-5.033126 1.829141-5.033126C1.936737-5.033126 2.139975-5.021171 2.139975-4.638605C2.139975-4.327771 1.984558-3.93325 1.888917-3.670237L1.052055-1.446575C.980324-1.255293 .908593-1.06401 .908593-.848817C.908593-.310834 1.279203 .119552 1.853051 .119552C2.952927 .119552 3.383313-1.625903 3.383313-1.709589ZM3.287671-7.460025C3.287671-7.639352 3.144209-7.854545 2.881196-7.854545C2.606227-7.854545 2.295392-7.591532 2.295392-7.280697C2.295392-6.981818 2.546451-6.886177 2.689913-6.886177C3.012702-6.886177 3.287671-7.197011 3.287671-7.460025Z" />
<path
id="g1-108"
d="M3.036613-7.998007C3.048568-8.045828 3.072478-8.117559 3.072478-8.177335C3.072478-8.296887 2.952927-8.296887 2.929016-8.296887C2.917061-8.296887 2.486675-8.261021 2.271482-8.237111C2.068244-8.225156 1.888917-8.201245 1.673724-8.18929C1.3868-8.16538 1.303113-8.153425 1.303113-7.938232C1.303113-7.81868 1.422665-7.81868 1.542217-7.81868C2.15193-7.81868 2.15193-7.711083 2.15193-7.591532C2.15193-7.543711 2.15193-7.519801 2.092154-7.304608L.609714-1.374844C.573848-1.243337 .549938-1.147696 .549938-.956413C.549938-.358655 .992279 .119552 1.601993 .119552C1.996513 .119552 2.259527-.143462 2.450809-.514072C2.654047-.908593 2.82142-1.661768 2.82142-1.709589C2.82142-1.769365 2.773599-1.817186 2.701868-1.817186C2.594271-1.817186 2.582316-1.75741 2.534496-1.578082C2.319303-.753176 2.10411-.119552 1.625903-.119552C1.267248-.119552 1.267248-.502117 1.267248-.669489C1.267248-.71731 1.267248-.968369 1.350934-1.303113L3.036613-7.998007Z" />
<path
id="g1-109"
d="M2.462765-3.502864C2.486675-3.574595 2.785554-4.172354 3.227895-4.554919C3.53873-4.841843 3.945205-5.033126 4.411457-5.033126C4.889664-5.033126 5.057036-4.674471 5.057036-4.196264C5.057036-4.124533 5.057036-3.88543 4.913574-3.323537L4.614695-2.092154C4.519054-1.733499 4.291905-.848817 4.267995-.71731C4.220174-.537983 4.148443-.227148 4.148443-.179328C4.148443-.011955 4.27995 .119552 4.459278 .119552C4.817933 .119552 4.877709-.155417 4.985305-.585803L5.702615-3.443088C5.726526-3.53873 6.348194-5.033126 7.663263-5.033126C8.141469-5.033126 8.308842-4.674471 8.308842-4.196264C8.308842-3.526775 7.84259-2.223661 7.579577-1.506351C7.47198-1.219427 7.412204-1.06401 7.412204-.848817C7.412204-.310834 7.782814 .119552 8.356663 .119552C9.468493 .119552 9.886924-1.637858 9.886924-1.709589C9.886924-1.769365 9.839103-1.817186 9.767372-1.817186C9.659776-1.817186 9.647821-1.78132 9.588045-1.578082C9.313076-.621669 8.870735-.119552 8.392528-.119552C8.272976-.119552 8.081694-.131507 8.081694-.514072C8.081694-.824907 8.225156-1.207472 8.272976-1.338979C8.488169-1.912827 9.026152-3.323537 9.026152-4.016936C9.026152-4.734247 8.607721-5.272229 7.699128-5.272229C6.898132-5.272229 6.252553-4.817933 5.774346-4.112578C5.738481-4.758157 5.34396-5.272229 4.447323-5.272229C3.383313-5.272229 2.82142-4.519054 2.606227-4.220174C2.570361-4.901619 2.080199-5.272229 1.554172-5.272229C1.207472-5.272229 .932503-5.104857 .705355-4.65056C.490162-4.220174 .32279-3.490909 .32279-3.443088S.37061-3.335492 .454296-3.335492C.549938-3.335492 .561893-3.347447 .633624-3.622416C.812951-4.327771 1.0401-5.033126 1.518306-5.033126C1.793275-5.033126 1.888917-4.841843 1.888917-4.483188C1.888917-4.220174 1.769365-3.753923 1.685679-3.383313L1.350934-2.092154C1.303113-1.865006 1.171606-1.327024 1.111831-1.111831C1.028144-.800996 .896638-.239103 .896638-.179328C.896638-.011955 1.028144 .119552 1.207472 .119552C1.350934 .119552 1.518306 .047821 1.613948-.131507C1.637858-.191283 1.745455-.609714 1.80523-.848817L2.068244-1.924782L2.462765-3.502864Z" />
<path
id="g1-110"
d="M2.462765-3.502864C2.486675-3.574595 2.785554-4.172354 3.227895-4.554919C3.53873-4.841843 3.945205-5.033126 4.411457-5.033126C4.889664-5.033126 5.057036-4.674471 5.057036-4.196264C5.057036-3.514819 4.566874-2.15193 4.327771-1.506351C4.220174-1.219427 4.160399-1.06401 4.160399-.848817C4.160399-.310834 4.531009 .119552 5.104857 .119552C6.216687 .119552 6.635118-1.637858 6.635118-1.709589C6.635118-1.769365 6.587298-1.817186 6.515567-1.817186C6.40797-1.817186 6.396015-1.78132 6.336239-1.578082C6.06127-.597758 5.606974-.119552 5.140722-.119552C5.021171-.119552 4.829888-.131507 4.829888-.514072C4.829888-.812951 4.961395-1.171606 5.033126-1.338979C5.272229-1.996513 5.774346-3.335492 5.774346-4.016936C5.774346-4.734247 5.355915-5.272229 4.447323-5.272229C3.383313-5.272229 2.82142-4.519054 2.606227-4.220174C2.570361-4.901619 2.080199-5.272229 1.554172-5.272229C1.171606-5.272229 .908593-5.045081 .705355-4.638605C.490162-4.208219 .32279-3.490909 .32279-3.443088S.37061-3.335492 .454296-3.335492C.549938-3.335492 .561893-3.347447 .633624-3.622416C.824907-4.351681 1.0401-5.033126 1.518306-5.033126C1.793275-5.033126 1.888917-4.841843 1.888917-4.483188C1.888917-4.220174 1.769365-3.753923 1.685679-3.383313L1.350934-2.092154C1.303113-1.865006 1.171606-1.327024 1.111831-1.111831C1.028144-.800996 .896638-.239103 .896638-.179328C.896638-.011955 1.028144 .119552 1.207472 .119552C1.350934 .119552 1.518306 .047821 1.613948-.131507C1.637858-.191283 1.745455-.609714 1.80523-.848817L2.068244-1.924782L2.462765-3.502864Z" />
<path
id="g1-111"
d="M5.451557-3.287671C5.451557-4.423412 4.710336-5.272229 3.622416-5.272229C2.044334-5.272229 .490162-3.550685 .490162-1.865006C.490162-.729265 1.231382 .119552 2.319303 .119552C3.90934 .119552 5.451557-1.601993 5.451557-3.287671ZM2.331258-.119552C1.733499-.119552 1.291158-.597758 1.291158-1.43462C1.291158-1.984558 1.578082-3.203985 1.912827-3.801743C2.450809-4.722291 3.120299-5.033126 3.610461-5.033126C4.196264-5.033126 4.65056-4.554919 4.65056-3.718057C4.65056-3.239851 4.399502-1.960648 3.945205-1.231382C3.455044-.430386 2.797509-.119552 2.331258-.119552Z" />
<path
id="g1-114"
d="M4.65056-4.889664C4.27995-4.817933 4.088667-4.554919 4.088667-4.291905C4.088667-4.004981 4.315816-3.90934 4.483188-3.90934C4.817933-3.90934 5.092902-4.196264 5.092902-4.554919C5.092902-4.937484 4.722291-5.272229 4.124533-5.272229C3.646326-5.272229 3.096389-5.057036 2.594271-4.327771C2.510585-4.961395 2.032379-5.272229 1.554172-5.272229C1.08792-5.272229 .848817-4.913574 .705355-4.65056C.502117-4.220174 .32279-3.502864 .32279-3.443088C.32279-3.395268 .37061-3.335492 .454296-3.335492C.549938-3.335492 .561893-3.347447 .633624-3.622416C.812951-4.339726 1.0401-5.033126 1.518306-5.033126C1.80523-5.033126 1.888917-4.829888 1.888917-4.483188C1.888917-4.220174 1.769365-3.753923 1.685679-3.383313L1.350934-2.092154C1.303113-1.865006 1.171606-1.327024 1.111831-1.111831C1.028144-.800996 .896638-.239103 .896638-.179328C.896638-.011955 1.028144 .119552 1.207472 .119552C1.338979 .119552 1.566127 .035866 1.637858-.203238C1.673724-.298879 2.116065-2.10411 2.187796-2.379078C2.247572-2.642092 2.319303-2.893151 2.379078-3.156164C2.426899-3.323537 2.47472-3.514819 2.510585-3.670237C2.546451-3.777833 2.86924-4.363636 3.16812-4.62665C3.311582-4.758157 3.622416-5.033126 4.112578-5.033126C4.303861-5.033126 4.495143-4.99726 4.65056-4.889664Z" />
<path
id="g1-116"
d="M2.402989-4.805978H3.502864C3.730012-4.805978 3.849564-4.805978 3.849564-5.021171C3.849564-5.152677 3.777833-5.152677 3.53873-5.152677H2.486675L2.929016-6.898132C2.976837-7.065504 2.976837-7.089415 2.976837-7.173101C2.976837-7.364384 2.82142-7.47198 2.666002-7.47198C2.570361-7.47198 2.295392-7.436115 2.199751-7.053549L1.733499-5.152677H.609714C.37061-5.152677 .263014-5.152677 .263014-4.925529C.263014-4.805978 .3467-4.805978 .573848-4.805978H1.637858L.848817-1.649813C.753176-1.231382 .71731-1.111831 .71731-.956413C.71731-.394521 1.111831 .119552 1.78132 .119552C2.988792 .119552 3.634371-1.625903 3.634371-1.709589C3.634371-1.78132 3.58655-1.817186 3.514819-1.817186C3.490909-1.817186 3.443088-1.817186 3.419178-1.769365C3.407223-1.75741 3.395268-1.745455 3.311582-1.554172C3.060523-.956413 2.510585-.119552 1.817186-.119552C1.458531-.119552 1.43462-.418431 1.43462-.681445C1.43462-.6934 1.43462-.920548 1.470486-1.06401L2.402989-4.805978Z" />
<path
id="g1-121"
d="M3.144209 1.338979C2.82142 1.793275 2.355168 2.199751 1.769365 2.199751C1.625903 2.199751 1.052055 2.175841 .872727 1.625903C.908593 1.637858 .968369 1.637858 .992279 1.637858C1.350934 1.637858 1.590037 1.327024 1.590037 1.052055S1.362889 .681445 1.183562 .681445C.992279 .681445 .573848 .824907 .573848 1.41071C.573848 2.020423 1.08792 2.438854 1.769365 2.438854C2.964882 2.438854 4.172354 1.338979 4.507098 .011955L5.678705-4.65056C5.69066-4.710336 5.71457-4.782067 5.71457-4.853798C5.71457-5.033126 5.571108-5.152677 5.391781-5.152677C5.284184-5.152677 5.033126-5.104857 4.937484-4.746202L4.052802-1.231382C3.993026-1.016189 3.993026-.992279 3.897385-.860772C3.658281-.526027 3.263761-.119552 2.689913-.119552C2.020423-.119552 1.960648-.777086 1.960648-1.099875C1.960648-1.78132 2.283437-2.701868 2.606227-3.56264C2.737733-3.90934 2.809465-4.076712 2.809465-4.315816C2.809465-4.817933 2.450809-5.272229 1.865006-5.272229C.765131-5.272229 .32279-3.53873 .32279-3.443088C.32279-3.395268 .37061-3.335492 .454296-3.335492C.561893-3.335492 .573848-3.383313 .621669-3.550685C.908593-4.554919 1.362889-5.033126 1.829141-5.033126C1.936737-5.033126 2.139975-5.033126 2.139975-4.638605C2.139975-4.327771 2.008468-3.981071 1.829141-3.526775C1.243337-1.960648 1.243337-1.566127 1.243337-1.279203C1.243337-.143462 2.056289 .119552 2.654047 .119552C3.000747 .119552 3.431133 .011955 3.849564-.430386L3.861519-.418431C3.682192 .286924 3.56264 .753176 3.144209 1.338979Z" />
</defs>
<g
id="page1"
transform="matrix(1.13 0 0 1.13 -80.23 -68.28816)">
<use
x="70.734745"
y="68.742217"
xlink:href="#g1-109"
id="use1" />
<use
x="80.974012"
y="68.742217"
xlink:href="#g1-101"
id="use2" />
<use
x="86.399452"
y="68.742217"
xlink:href="#g1-109"
id="use3" />
<use
x="96.638719"
y="68.742217"
xlink:href="#g1-111"
id="use4" />
<use
x="102.266156"
y="68.742217"
xlink:href="#g1-114"
id="use5" />
<use
x="107.86663"
y="68.742217"
xlink:href="#g1-121"
id="use6" />
<use
x="114.003281"
y="68.742217"
xlink:href="#g1-58"
id="use7" />
<use
x="117.254943"
y="68.742217"
xlink:href="#g1-104"
id="use8" />
<use
x="123.993498"
y="68.742217"
xlink:href="#g1-105"
id="use9" />
<use
x="127.98693"
y="68.742217"
xlink:href="#g1-103"
id="use10" />
<use
x="134.021186"
y="68.742217"
xlink:href="#g1-104"
id="use11" />
<use
x="144.08057"
y="68.742217"
xlink:href="#g2-61"
id="use12" />
<use
x="156.506051"
y="68.742217"
xlink:href="#g1-77"
id="use13" />
<use
x="169.079658"
y="68.742217"
xlink:href="#g1-101"
id="use14" />
<use
x="174.505098"
y="68.742217"
xlink:href="#g1-109"
id="use15" />
<use
x="184.744365"
y="68.742217"
xlink:href="#g1-111"
id="use16" />
<use
x="190.371803"
y="68.742217"
xlink:href="#g1-114"
id="use17" />
<use
x="195.972276"
y="68.742217"
xlink:href="#g1-121"
id="use18" />
<use
x="202.108928"
y="68.742217"
xlink:href="#g1-84"
id="use19" />
<use
x="210.595764"
y="68.742217"
xlink:href="#g1-104"
id="use20" />
<use
x="217.334319"
y="68.742217"
xlink:href="#g1-114"
id="use21" />
<use
x="222.934792"
y="68.742217"
xlink:href="#g1-111"
id="use22" />
<use
x="228.56223"
y="68.742217"
xlink:href="#g1-116"
id="use23" />
<use
x="232.789389"
y="68.742217"
xlink:href="#g1-116"
id="use24" />
<use
x="237.01656"
y="68.742218"
xlink:href="#g1-105"
id="use25"
transform="translate(3.9823008)" />
<use
x="241.00998"
y="68.742218"
xlink:href="#g1-108"
id="use26"
transform="translate(-3.9823008)" />
<use
x="244.75979"
y="68.742217"
xlink:href="#g1-110"
id="use27" />
<use
x="251.747395"
y="68.742217"
xlink:href="#g1-103"
id="use28" />
<use
x="257.781652"
y="68.742217"
xlink:href="#g1-70"
id="use29" />
<use
x="266.985267"
y="68.742217"
xlink:href="#g1-97"
id="use30" />
<use
x="273.130211"
y="68.742217"
xlink:href="#g1-99"
id="use31" />
<use
x="278.1682"
y="68.742217"
xlink:href="#g1-116"
id="use32" />
<use
x="282.395359"
y="68.742217"
xlink:href="#g1-111"
id="use33" />
<use
x="288.022797"
y="68.742217"
xlink:href="#g1-114"
id="use34" />
<use
x="296.224599"
y="68.742217"
xlink:href="#g0-3"
id="use35" />
<use
x="304.803535"
y="68.742217"
xlink:href="#g1-78"
id="use36" />
<use
x="315.426145"
y="68.742217"
xlink:href="#g1-111"
id="use37" />
<use
x="321.053583"
y="68.742217"
xlink:href="#g1-100"
id="use38" />
<use
x="327.136276"
y="68.742217"
xlink:href="#g1-101"
id="use39" />
<use
x="332.561716"
y="68.742217"
xlink:href="#g1-65"
id="use40" />
<use
x="341.337063"
y="68.742217"
xlink:href="#g1-108"
id="use41" />
<use
x="345.086871"
y="68.742217"
xlink:href="#g1-108"
id="use42" />
<use
x="348.83668"
y="68.742217"
xlink:href="#g1-111"
id="use43" />
<use
x="354.464117"
y="68.742217"
xlink:href="#g1-99"
id="use44" />
<use
x="359.502106"
y="68.742217"
xlink:href="#g1-97"
id="use45" />
<use
x="365.64705"
y="68.742217"
xlink:href="#g1-116"
id="use46" />
<use
x="369.87421"
y="68.742217"
xlink:href="#g1-97"
id="use47" />
<use
x="376.019154"
y="68.742217"
xlink:href="#g1-98"
id="use48" />
<use
x="380.996259"
y="68.742217"
xlink:href="#g1-108"
id="use49" />
<use
x="384.746068"
y="68.742217"
xlink:href="#g1-101"
id="use50" />
<use
x="390.171508"
y="68.742217"
xlink:href="#g1-77"
id="use51" />
<use
x="402.745115"
y="68.742217"
xlink:href="#g1-101"
id="use52" />
<use
x="408.170555"
y="68.742217"
xlink:href="#g1-109"
id="use53" />
<use
x="418.409822"
y="68.742217"
xlink:href="#g1-111"
id="use54" />
<use
x="424.03726"
y="68.742217"
xlink:href="#g1-114"
id="use55" />
<use
x="429.637733"
y="68.742217"
xlink:href="#g1-121"
id="use56" />
</g>
</svg>

Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 30 KiB

After

Width:  |  Height:  |  Size: 34 KiB

1296
content/en/blog/_posts/2023-05-05-memory-qos-cgroups-v2/container-memory-high-limit.svg
View File

File diff suppressed because it is too large Load Diff
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 43 KiB

After

Width:  |  Height:  |  Size: 53 KiB

1152
content/en/blog/_posts/2023-05-05-memory-qos-cgroups-v2/container-memory-high-no-limits.svg
View File

File diff suppressed because it is too large Load Diff
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 45 KiB

After

Width:  |  Height:  |  Size: 54 KiB

1445
content/en/blog/_posts/2023-05-05-memory-qos-cgroups-v2/container-memory-high.svg
View File

File diff suppressed because it is too large Load Diff
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 48 KiB

After

Width:  |  Height:  |  Size: 59 KiB

27
content/en/blog/_posts/2023-05-05-memory-qos-cgroups-v2/index.md
View File

@ -128,18 +128,14 @@ enforces the limit to prevent the container from using more than the configured
resource limit. If a process in a container tries to consume more than the
specified limit, kernel terminates a process(es) with an Out of Memory (OOM) error.
```formula
memory.max = pod.spec.containers[i].resources.limits[memory]
```
{{< figure src="/blog/2023/05/05/qos-memory-resources/container-memory-max.svg" title="memory.max maps to limits.memory" alt="memory.max maps to limits.memory" >}}
`memory.min` is mapped to `requests.memory`, which results in reservation of memory resources
that should never be reclaimed by the kernel. This is how Memory QoS ensures the availability of
memory for Kubernetes pods. If there's no unprotected reclaimable memory available, the OOM
killer is invoked to make more memory available.
```formula
memory.min = pod.spec.containers[i].resources.requests[memory]
```
{{< figure src="/blog/2023/05/05/qos-memory-resources/container-memory-min.svg" title="memory.min maps to requests.memory" alt="memory.min maps to requests.memory" >}}
For memory protection, in addition to the original way of limiting memory usage, Memory QoS
throttles workload approaching its memory limit, ensuring that the system is not overwhelmed
@ -149,10 +145,7 @@ the KubeletConfiguration when you enable MemoryQoS feature. It is set to 0.9 by
`requests.memory` and `limits.memory` as in the formula below, and rounding down the
value to the nearest page size:
```formula
memory.high = pod.spec.containers[i].resources.requests[memory] + MemoryThrottlingFactor *
{(pod.spec.containers[i].resources.limits[memory] or NodeAllocatableMemory) - pod.spec.containers[i].resources.requests[memory]}
```
{{< figure src="/blog/2023/05/05/qos-memory-resources/container-memory-high.svg" title="memory.high formula" alt="memory.high formula" >}}
{{< note >}}
If a container has no memory limits specified, `limits.memory` is substituted for node allocatable memory.
@ -256,26 +249,18 @@ as per QOS classes:
* When requests.memory and limits.memory are set, the formula is used as-is:
```formula
memory.high = pod.spec.containers[i].resources.requests[memory] + MemoryThrottlingFactor *
{(pod.spec.containers[i].resources.limits[memory]) - pod.spec.containers[i].resources.requests[memory]}
```
{{< figure src="/blog/2023/05/05/qos-memory-resources/container-memory-high-limit.svg" title="memory.high when requests and limits are set" alt="memory.high when requests and limits are set" >}}
* When requests.memory is set and limits.memory is not set, limits.memory is substituted
for node allocatable memory in the formula:
```formula
memory.high = pod.spec.containers[i].resources.requests[memory] + MemoryThrottlingFactor *
{(NodeAllocatableMemory) - pod.spec.containers[i].resources.requests[memory]}
```
{{< figure src="/blog/2023/05/05/qos-memory-resources/container-memory-high-no-limits.svg" title="memory.high when requests and limits are not set" alt="memory.high when requests and limits are not set" >}}
1. **BestEffort** by their QoS definition do not require any memory or CPU limits or requests.
For this case, kubernetes sets requests.memory = 0 and substitute limits.memory for node allocatable
memory in the formula:
```formula
memory.high = MemoryThrottlingFactor * NodeAllocatableMemory
```
{{< figure src="/blog/2023/05/05/qos-memory-resources/container-memory-high-best-effort.svg" title="memory.high for BestEffort Pod" alt="memory.high for BestEffort Pod" >}}
**Summary**: Only Pods in Burstable and BestEffort QoS classes will set `memory.high`.
Guaranteed QoS pods do not set `memory.high` as their memory is guaranteed.

8
content/en/blog/_posts/2023-10-12-bootstrap-an-air-gapped-cluster-with-kubeadm/index.md
View File

@ -22,7 +22,7 @@ A real air-gapped network can take some effort to set up, so for this post, I wi
### Local topology
This VM will have its network connectivity disabled but in a way that doesn't shut down the VM's virtual NIC. Instead, its network will be downed by injecting a default route to a dummy interface, making anything internet-hosted unreachable. However, the VM still has a connected route to the bridge interface on the host, which means that network connectivity to the host is still working. This posture means that data can be transferred from the host/laptop to the VM via scp, even with the default route on the VM black-holing all traffic that isn't destined for the local bridge subnet. This type of transfer is analogous to carrying data across the air gap and will be used throughout this post.
This VM will have its network connectivity disabled but in a way that doesn't shut down the VM's virtual NIC. Instead, its network will be downed by injecting a default route to a dummy interface, making anything internet-hosted unreachable. However, the VM still has a connected route to the bridge interface on the host, which means that network connectivity to the host is still working. This posture means that data can be transferred from the host/laptop to the VM via `scp`, even with the default route on the VM black-holing all traffic that isn't destined for the local bridge subnet. This type of transfer is analogous to carrying data across the air gap and will be used throughout this post.
Other details about the lab setup:
@ -35,7 +35,7 @@ While this single VM lab is a simplified example, the below diagram more approxi
{{< figure src="example_production_topology.svg" alt="Example production topology which shows 3 control plane Kubernetes nodes and 'n' worker nodes along with a Docker registry in an air-gapped environment. Additionally shows two workstations, one on each side of the air gap and an IT admin which physically carries the artifacts across." >}}
Note, there is still intentional isolation between the envirnment and the internet. There are also some things that are not shown in order to keep the diagram simple, for example malware scanning on the secure side of the air gap.
Note, there is still intentional isolation between the environment and the internet. There are also some things that are not shown in order to keep the diagram simple, for example malware scanning on the secure side of the air gap.
Back to the single VM lab environment.
@ -144,7 +144,7 @@ reboot
On the laptop/host machine, download all of the artifacts enumerated in the previous section. Since the air gapped VM is running Fedora 37, all of the dependencies shown in this part are for Fedora 37. Note, this procedure will only work on AArch64 or AMD64 CPU architectures as they are the most popular and widely available.. You can execute this procedure anywhere you have write permissions; your home directory is a perfectly suitable choice.
Note, operating system packages for the Kubernetes artifacts that need to be carried across can now be found at [pkgs.k8s.io](https://kubernetes.io/blog/2023/08/15/pkgs-k8s-io-introduction/). This blog post will use a combination of Fedora repositories and GitHub in order to download all of the required artifacts. When you’re doing this on your own cluster, you should decide whether to use the official Kubernetes packages, or the official packages from your operating system distribution - both are valid choices.
Note, operating system packages for the Kubernetes artifacts that need to be carried across can now be found at [pkgs.k8s.io](/blog/2023/08/15/pkgs-k8s-io-introduction/). This blog post will use a combination of Fedora repositories and GitHub in order to download all of the required artifacts. When you’re doing this on your own cluster, you should decide whether to use the official Kubernetes packages, or the official packages from your operating system distribution - both are valid choices.
@ -612,7 +612,7 @@ export ZARF_VERSION=v0.28.3
curl -LO "https://github.com/defenseunicorns/zarf/releases/download/${ZARF_VERSION}/zarf_${ZARF_VERSION}_Linux_${K8s_ARCH}"
```
Zarf needs to bootstrap itself into a Kubernetes cluster through the use of an init package. That also needs to be transported across the air gap so let's download it onto the host/laptop:
```bash
```bash
curl -LO "https://github.com/defenseunicorns/zarf/releases/download/${ZARF_VERSION}/zarf-init-${K8s_ARCH}-${ZARF_VERSION}.tar.zst"
```
The way that Zarf is declarative is through the use of a zarf.yaml file. Here is the zarf.yaml file that will be used for this Podinfo installation. Write it to whatever directory you you have write access to on your host/laptop; your home directory is fine:

114
content/en/blog/_posts/2023-10-20-kcs-shanghai/index.md Normal file
View File

@ -0,0 +1,114 @@
---
layout: blog
title: "A Quick Recap of 2023 China Kubernetes Contributor Summit"
slug: kcs-shanghai
date: 2023-10-20
canonicalUrl: https://www.kubernetes.dev/blog/2023/10/20/kcs-shanghai/
---
**Author:** Paco Xu and Michael Yao (DaoCloud)
On September 26, 2023, the first day of
[KubeCon + CloudNativeCon + Open Source Summit China 2023](https://www.lfasiallc.com/kubecon-cloudnativecon-open-source-summit-china/),
nearly 50 contributors gathered in Shanghai for the Kubernetes Contributor Summit.
{{< figure src="/blog/2023/10/20/kcs-shanghai/kcs04.jpeg" alt="All participants in the 2023 Kubernetes Contributor Summit" caption="All participants in the 2023 Kubernetes Contributor Summit" >}}
This marked the first in-person offline gathering held in China after three years of the pandemic.
## A joyful meetup
The event began with welcome speeches from [Kevin Wang](https://github.com/kevin-wangzefeng) from Huawei Cloud,
one of the co-chairs of KubeCon, and [Puja](https://github.com/puja108) from Giant Swarm.
Following the opening remarks, the contributors introduced themselves briefly. Most attendees were from China,
while some contributors had made the journey from Europe and the United States specifically for the conference.
Technical experts from companies such as Microsoft, Intel, Huawei, as well as emerging forces like DaoCloud,
were present. Laughter and cheerful voices filled the room, regardless of whether English was spoken with
European or American accents or if conversations were carried out in authentic Chinese language. This created
an atmosphere of comfort, joy, respect, and anticipation. Past contributions brought everyone closer, and
mutual recognition and accomplishments made this offline gathering possible.
{{< figure src="/blog/2023/10/20/kcs-shanghai/kcs06.jpeg" alt="Face to face meeting in Shanghai" caption="Face to face meeting in Shanghai" >}}
The attending contributors were no longer just GitHub IDs; they transformed into vivid faces.
From sitting together and capturing group photos to attempting to identify "Who is who,"
a loosely connected collective emerged. This team structure, although loosely knit and free-spirited,
was established to pursue shared dreams.
As the saying goes, "You reap what you sow." Each effort has been diligently documented within
the Kubernetes community contributions. Regardless of the passage of time, the community will
not erase those shining traces. Brilliance can be found in your PRs, issues, or comments.
It can also be seen in the smiling faces captured in meetup photos or heard through stories
passed down among contributors.
## Technical sharing and discussions
Next, there were three technical sharing sessions:
- [sig-multi-cluster](https://github.com/kubernetes/community/blob/master/sig-multicluster/README.md):
[Hongcai Ren](https://github.com/RainbowMango), a maintainer of Karmada, provided an introduction to
the responsibilities and roles of this SIG. Their focus is on designing, discussing, implementing,
and maintaining APIs, tools, and documentation related to multi-cluster management.
Cluster Federation, one of Karmada's core concepts, is also part of their work.
- [helmfile](https://github.com/helmfile/helmfile): [yxxhero](https://github.com/yxxhero)
from [GitLab](https://gitlab.cn/) presented how to deploy Kubernetes manifests declaratively,
customize configurations, and leverage the latest features of Helm, including Helmfile.
- [sig-scheduling](https://github.com/kubernetes/community/blob/master/sig-scheduling/README.md):
[william-wang](https://github.com/william-wang) from Huawei Cloud shared the recent updates and
future plans of SIG Scheduling. This SIG is responsible for designing, developing, and testing
components related to Pod scheduling.
{{< figure src="/blog/2023/10/20/kcs-shanghai/kcs03.jpeg" alt="A technical session about sig-multi-cluster" caption="A technical session about sig-multi-cluster" >}}
Following the sessions, a video featuring a call for contributors by [Sergey Kanzhelev](https://github.com/SergeyKanzhelev),
the SIG-Node Chair, was played. The purpose was to encourage more contributors to join the Kubernetes community,
with a special emphasis on the popular SIG-Node.
Lastly, Kevin hosted an Unconference collective discussion session covering topics such as
multi-cluster management, scheduling, elasticity, AI, and more. For detailed minutes of
the Unconference meeting, please refer to <https://docs.qq.com/doc/DY3pLWklzQkhjWHNT>.
## China's contributor statistics
The contributor summit took place in Shanghai, with 90% of the attendees being Chinese.
Within the Cloud Native Computing Foundation (CNCF) ecosystem, contributions from China have been steadily increasing. Currently:
- Chinese contributors account for 9% of the total.
- Contributions from China make up 11.7% of the overall volume.
- China ranks second globally in terms of contributions.
{{< note >}}
The data is from KubeCon keynotes by Chris Aniszczyk, CTO of Cloud Native Computing Foundation,
on September 26, 2023. This probably understates Chinese contributions. A lot of Chinese contributors
use VPNs and may not show up as being from China in the stats accurately.
{{< /note >}}
The Kubernetes Contributor Summit is an inclusive meetup that welcomes all community contributors, including:
- New Contributors
- Current Contributors
- docs
- code
- community management
- Subproject members
- Members of Special Interest Group (SIG) / Working Group (WG)
- Active Contributors
- Casual Contributors
## Acknowledgments
We would like to express our gratitude to the organizers of this event:
- [Kevin Wang](https://github.com/kevin-wangzefeng), the co-chair of KubeCon and the lead of the kubernetes contributor summit.
- [Paco Xu](https://github.com/pacoxu), who actively coordinated the venue, meals, invited contributors from both China and
international sources, and established WeChat groups to collect agenda topics. They also shared details of the event
before and after its occurrence through [pre and post announcements](https://github.com/kubernetes/community/issues/7510).
- [Mengjiao Liu](https://github.com/mengjiao-liu), who was responsible for organizing, coordinating,
and facilitating various matters related to the summit.
We extend our appreciation to all the contributors who attended the China Kubernetes Contributor Summit in Shanghai.
Your dedication and commitment to the Kubernetes community are invaluable.
Together, we continue to push the boundaries of cloud native technology and shape the future of this ecosystem.

BIN
content/en/blog/_posts/2023-10-20-kcs-shanghai/kcs03.jpeg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 2.5 MiB

BIN
content/en/blog/_posts/2023-10-20-kcs-shanghai/kcs04.jpeg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 3.9 MiB

BIN
content/en/blog/_posts/2023-10-20-kcs-shanghai/kcs06.jpeg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 3.1 MiB

105
content/en/blog/_posts/2023-10-23-pv-last-phase-transtition-time.md Normal file
View File

@ -0,0 +1,105 @@
---
layout: blog
title: PersistentVolume Last Phase Transition Time in Kubernetes
date: 2023-10-23
slug: persistent-volume-last-phase-transition-time
---
**Author:** Roman Bednář (Red Hat)
In the recent Kubernetes v1.28 release, we (SIG Storage) introduced a new alpha feature that aims to improve PersistentVolume (PV)
storage management and help cluster administrators gain better insights into the lifecycle of PVs.
With the addition of the `lastPhaseTransitionTime` field into the status of a PV,
cluster administrators are now able to track the last time a PV transitioned to a different
[phase](/docs/concepts/storage/persistent-volumes/#phase), allowing for more efficient
and informed resource management.
## Why do we need new PV field? {#why-new-field}
PersistentVolumes in Kubernetes play a crucial role in providing storage resources to workloads running in the cluster.
However, managing these PVs effectively can be challenging, especially when it comes
to determining the last time a PV transitioned between different phases, such as
`Pending`, `Bound` or `Released`.
Administrators often need to know when a PV was last used or transitioned to certain
phases; for instance, to implement retention policies, perform cleanup, or monitor storage health.
In the past, Kubernetes users have faced data loss issues when using the `Delete` retain policy and had to resort to the safer `Retain` policy.
When we planned the work to introduce the new `lastPhaseTransitionTime` field, we
wanted to provide a more generic solution that can be used for various use cases,
including manual cleanup based on the time a volume was last used or producing alerts based on phase transition times.
## How lastPhaseTransitionTime helps
Provided you've enabled the feature gate (see [How to use it](#how-to-use-it), the new `.status.lastPhaseTransitionTime` field of a PersistentVolume (PV)
is updated every time that PV transitions from one phase to another.
``
Whether it's transitioning from `Pending` to `Bound`, `Bound` to `Released`, or any other phase transition, the `lastPhaseTransitionTime` will be recorded.
For newly created PVs the phase will be set to `Pending` and the `lastPhaseTransitionTime` will be recorded as well.
This feature allows cluster administrators to:
1. Implement Retention Policies
With the `lastPhaseTransitionTime`, administrators can now track when a PV was last used or transitioned to the `Released` phase.
This information can be crucial for implementing retention policies to clean up resources that have been in the `Released` phase for a specific duration.
For example, it is now trivial to write a script or a policy that deletes all PVs that have been in the `Released` phase for a week.
2. Monitor Storage Health
By analyzing the phase transition times of PVs, administrators can monitor storage health more effectively.
For example, they can identify PVs that have been in the `Pending` phase for an unusually long time, which may indicate underlying issues with the storage provisioner.
## How to use it
The `lastPhaseTransitionTime` field is alpha starting from Kubernetes v1.28, so it requires
the `PersistentVolumeLastPhaseTransitionTime` feature gate to be enabled.
If you want to test the feature whilst it's alpha, you need to enable this feature gate on the `kube-controller-manager` and the `kube-apiserver`.
Use the `--feature-gates` command line argument:
```shell
--feature-gates="...,PersistentVolumeLastPhaseTransitionTime=true"
```
Keep in mind that the feature enablement does not have immediate effect; the new field will be populated whenever a PV is updated and transitions between phases.
Administrators can then access the new field through the PV status, which can be retrieved using standard Kubernetes API calls or through Kubernetes client libraries.
Here is an example of how to retrieve the `lastPhaseTransitionTime` for a specific PV using the `kubectl` command-line tool:
```shell
kubectl get pv <pv-name> -o jsonpath='{.status.lastPhaseTransitionTime}'
```
## Going forward
This feature was initially introduced as an alpha feature, behind a feature gate that is disabled by default.
During the alpha phase, we (Kubernetes SIG Storage) will collect feedback from the end user community and address any issues or improvements identified.
Once sufficient feedback has been received, or no complaints are received the feature can move to beta.
The beta phase will allow us to further validate the implementation and ensure its stability.
At least two Kubernetes releases will happen between the release where this field graduates
to beta and the release that graduates the field to general availability (GA). That means that
the earliest release where this field could be generally available is Kubernetes 1.32,
likely to be scheduled for early 2025.
## Getting involved
We always welcome new contributors so if you would like to get involved you can
join our [Kubernetes Storage Special-Interest-Group](https://github.com/kubernetes/community/tree/master/sig-storage) (SIG).
If you would like to share feedback, you can do so on our
[public Slack channel](https://app.slack.com/client/T09NY5SBT/C09QZFCE5).
If you're not already part of that Slack workspace, you can visit https://slack.k8s.io/ for an invitation.
Special thanks to all the contributors that provided great reviews, shared valuable insight and helped implement this feature (alphabetical order):
- Han Kang ([logicalhan](https://github.com/logicalhan))
- Jan Šafránek ([jsafrane](https://github.com/jsafrane))
- Jordan Liggitt ([liggitt](https://github.com/liggitt))
- Kiki ([carlory](https://github.com/carlory))
- Michelle Au ([msau42](https://github.com/msau42))
- Tim Bannister ([sftim](https://github.com/sftim))
- Wojciech Tyczynski ([wojtek-t](https://github.com/wojtek-t))
- Xing Yang ([xing-yang](https://github.com/xing-yang))

223
content/en/blog/_posts/2023-10-24-kubernetes-1.28-release-interview.md Normal file
View File

@ -0,0 +1,223 @@
---
layout: blog
title: "Plants, process and parties: the Kubernetes 1.28 release interview"
date: 2023-10-24
---
**Author**: Craig Box
Since 2018, one of my favourite contributions to the Kubernetes community has been to [share the story of each release](https://www.google.com/search?q=%22release+interview%22+site%3Akubernetes.io%2Fblog). Many of these stories were told on behalf of a past employer; by popular demand, I've brought them back, now under my own name. If you were a fan of the old show, I would be delighted if you would [subscribe](https://craigbox.substack.com/about).
Back in August, [we welcomed the release of Kubernetes 1.28](/blog/2023/08/15/kubernetes-v1-28-release/). That release was led by [Grace Nguyen](https://twitter.com/gracenng), a CS student at the University of Waterloo. Grace joined me for the traditional release interview, and while you can read her story below, [I encourage you to listen to it if you can](https://craigbox.substack.com/p/the-kubernetes-128-release-interview).
*This transcript has been lightly edited and condensed for clarity.*
---
**You're a student at the University of Waterloo, so I want to spend the first two minutes of this interview talking about the Greater Kitchener-Waterloo region. It's August, so this is one of the four months of the year when there's no snow visible on the ground?**<br>
Well, it's not that bad. I think the East Coast has it kind of good. I grew up in Calgary, but I do love summer here in Waterloo. We have a [petting zoo](https://goo.gl/maps/W1nM7LjNZPv) close to our university campus, so I go and see the llamas sometimes.
**Is that a new thing?**<br>
I'm not sure, it seems like it's been around five-ish years, the Waterloo Park?
**I lived there in 2007, for a couple of years, just to set the scene for why we're talking about this. I think they were building a lot of the park then. I do remember, of course, that [Kitchener holds the second largest Oktoberfest in the world](https://www.oktoberfest.ca/). Is that something you've had a chance to check out?**<br>
I have not. I actually didn't know that was a fact.
**The local civic organization is going to have to do a bit more work, I feel. Do you like ribs?**<br>
I have mixed feelings about ribs. It's kind of a hit or miss situation for me so far.
**Again, that might be something that's changed over the last few years. The Ribfests used to have a lot of trophies with little pigs on top of them, but I feel that the shifting dining habits of the world might mean they have to offer some vegan or vegetarian options, to please the modern palette.**<br>
[LAUGHS] For sure. Do you recommend the Oktoberfest here? Have you been?
**I went a couple of times. It was a lot of fun.**<br>
Okay.
**It's basically just drinking. I would have recommended it back then; I'm not sure it would be quite what I'd be doing today.**<br>
All right, good to know.
**The Ribfest, however, I would go back just for that.**<br>
Oh, ok.
**And the great thing about Ribfests as a concept is that they have one in every little town. [The Kitchener Ribfest](https://kitchenerribandbeerfest.com/), I looked it up, it's in July; you've just missed that. But, you could go to the [Waterloo Ribfest](https://northernheatribseries.ca/waterloo/) in September.**<br>
Oh, it is in September? They have their own Ribfest?
**They do. I think Guelph has one, and Cambridge has one. That's the advantage of the region — there are lots of little cities. Kitchener and Waterloo are two cities that grew into each other — they do call them the Twin Cities. I hear that they finally built the light rail link between the two of them?**<br>
It is fantastic, and makes the city so much more walkable.
**Yes, you can go from one mall to the other. That's Canada for you.**<br>
Well, Uptown is really nice. I quite like it. It's quite cozy.
**Do you ever cross the border over into Kitchener? Or only when you've lost a bet?**<br>
Yeah, not a lot. Only for farmer's market, I say.
**It's worthwhile. There's a lot of good food there, I remember.**<br>
Yeah. Quite lovely.
**Now we've got all that out of the way, let's travel back in time a little bit. You mentioned there that you went to high school in Calgary?**<br>
I did. I had not been to Ontario before I went to university. Calgary was frankly too cold and not walkable enough for me.
**I basically say the same thing about Waterloo and that's why I moved to England.**<br>
Fascinating. Gets better.
**How did you get into tech?**<br>
I took a computer science class in high school. I was one of maybe only three women in the class, and I kind of stuck with it since.
**Was the gender distribution part of your thought process at the time?**<br>
Yeah, I think I was drawn to it partially because I didn't see a lot of people who looked like me in the class.
**You followed it through to university. What is it that you're studying?**<br>
I am studying computer engineering, so a lot of hardware stuff.
**You're involved in the [UW Cybersecurity Club](https://www.facebook.com/groups/uwcyber/). What can you tell me about that without having to kill me?**<br>
Oh, we are very nice and friendly people! I told myself I'm going to have a nice and chill summer and then I got chosen to lead the release and also ended up running the Waterloo Cybersecurity Club. The club kind of died out during the pandemic, because we weren't on campus, but we have so many smart and amazing people who are in cybersecurity, so it's great to get them together and I learned so many things.
**Is that like the modern equivalent of the [LAN party](https://en.wikipedia.org/wiki/LAN_party)? You're all getting into a dark room and trying to hack the Gibson?**<br>
[LAUGHS] Well, you'll have to explain to me again what a LAN party is. Do you bring your own PC?
**You used to. Back in the day it was incomprehensible that you could communicate with a different person in a different place at a fast enough speed, so you had to physically sit next to somebody and plug a cable in between you.**<br>
Okay, well kind of the same, I guess. We bring our own laptop and we go to CTF competitions together.
**They didn't have laptops back in the days of LAN parties. You'd bring a giant 19-inch square monitor, and everything. It was a badge of honor what you could carry.**<br>
Okay. Can't relate, but good to know. [LAUGHS]
**One of the more unique aspects of UW is its [co-op system](https://uwaterloo.ca/future-students/co-op). Tell us a little bit about that?**<br>
As part of my degree, I am required to do minimum five and maximum six co-ops. I've done all six of them. Two of them were in Kubernetes and that's how I got started.
**A co-op is a placement, as opposed to something you do on campus?**<br>
Right, so co-op is basically an internship. My first one was at the Canada Revenue Agency. We didn't have wifi and I had my own cubicle, which is interesting. They don't do that anymore, they have open office space. But my second was at Ericsson, where I learned about Kubernetes. It was during the pandemic. KubeCon offered virtual attendance for students and I signed up and I poked around and I have been around since.
**What was it like going through university during the COVID years? What did that mean in terms of the fact you would previously have traveled to these internships? Did you do them all from home?**<br>
I'm not totally sure what I missed out on. For sure, a lot of relationship building, but also that we do have to move a lot as part of the co-op experience. Last fall I was in San Francisco, I was in Palo Alto earlier this year. A lot of that dynamic has already been the case.
**Definitely different weather systems, Palo Alto versus Waterloo.**<br>
Oh, for sure. Yes, yes. Really glad I was there over the winter.
**The first snow would fall in Ontario about the end of October and it would pile up over the next few months. There were still piles that hadn't melted by June. That's why I say, there were only four months of the year, July through September, where there was no snow on the ground.**<br>
That's true. Didn't catch any snow in Palo Alto, and honestly, that's great. [CHUCKLES]
**Thank you, global warming, I guess.**<br>
Oh no! [LAUGHS]
**Tell me about the co-op term that you did working with Kubernetes at Ericsson?**<br>
This was such a long time ago, but we were trying to build some sort of pipeline to deploy testing. It was running inside a cluster, and I learned Helm charts and all that good stuff. And then, for the co-op after that, I worked at a Canadian startup in FinTech. It was 24/7 Kubernetes, [building their secret injection system, using ArgoCD to automatically pull secrets from 1Password](https://medium.com/@nng.grace/automated-kubernetes-secret-injection-with-1password-secret-automation-and-hashicorp-vault-8db826c50c1d).
**How did that lead you on to involvement with the release team?**<br>
It was over the pandemic, so I didn't have a lot to do, I went to the conference, saw so many cool talks. One that really stuck out to me was [a Kubernetes hacking talk by Tabitha Sable and V Korbes](https://www.youtube.com/watch?v=-4W3ChRVeLI). I thought it was the most amazing thing and it was so cool. One of my friends was on the release team at the time, and she showed me what she does. I applied and thankfully got in. I didn't have any open source experience. It was fully like one of those things where someone took a chance on me.
**How would you characterize the experience that you've had to date? You have had involvement with pretty much every release since then.**<br>
Yeah, I think it was a really formative experience, and the community has been such a big part of it.
**You started as an enhancement shadow with Kubernetes 1.22, eventually moving up to enhancements lead, then you moved on to be the release lead shadow. Obviously, you are the lead for 1.28, but for 1.27 you did something a bit different. What was that, and why did you do it?**<br>
For 1.25 and 1.26, I was release lead shadow, so I had an understanding of what that role was like. I wanted to shadow another team, and at that time I thought CI Signal was a big black box to me. I joined the team, but I also had capacity for other things, I joined as a branch manager associate as well.
**What is the difference between that role and the traditional release team roles we think about?**<br>
Yeah, that's a great question. So the branch management role is a more constant role. They don't necessarily get swapped out every release. You shadow as an associate, so you do things like cut releases, distribute them, update distros, things like that. It's a really important role, and the folks that are in there are more technical. So if you have been on the release team for a long time and are looking for more permanent role, I recommend looking into that.
**Congratulations again on [the release of 1.28 today](/blog/2023/08/15/kubernetes-v1-28-release/).**<br>
Yeah, thank you.
**What is the best new feature in Kubernetes 1.28, and why is it [sidecar container support](/blog/2023/08/25/native-sidecar-containers/)?**<br>
Great question. I am as excited as you. In 1.28, we have a new feature in alpha, which is sidecar container support. We introduced a new field called restartPolicy for init containers, that allows the containers to live throughout the life cycle of the pod and not block the pod from terminating. Craig, you know a lot about this, but there are so many use cases for this. It is a very common pattern. You use it for logging, monitoring, metrics; also configs and secrets as well.
**And the service mesh!**<br>
And the service mesh.
**Very popular. I will say that the Sidecar pattern was called out very early on, in [a blog post Brendan Burns wrote](/blog/2015/06/the-distributed-system-toolkit-patterns/), talking about how you can achieve some of the things you just mentioned. Support for it in Kubernetes has been— it's been a while, shall we say. I've been doing these interviews since 2018, and September 2019 was when [I first had a conversation with a release manager](/blog/2019/12/06/when-youre-in-the-release-team-youre-family-the-kubernetes-1.16-release-interview/) who felt they had to apologize for Sidecar containers not shipping in that release.**<br>
Well, here we are!
**Thank you for not letting the side down.**<br>
[LAUGHS]
**There are a bunch of other features that are going to GA in 1.28. Tell me about what's new with [kubectl events](https://github.com/kubernetes/enhancements/issues/1440)?**<br>
It got a new CLI and now it is separate from kubectl get. I think that changes in the CLI are always a little bit more apparent because they are user-facing.
**Are there a lot of other user-facing changes, or are most of the things in the release very much behind the scenes?**<br>
I would say it's a good mix of both; it depends on what you're interested in.
**I am interested, of course, in [non-graceful node shutdown support](https://github.com/kubernetes/enhancements/issues/2268). What can you tell us about that?**<br>
Right, so for situations where you have a hardware failure or a broken OS, we have added additional support for a better graceful shutdown.
**If someone trips over the power cord at your LAN party and your cluster goes offline as a result?**<br>
Right, exactly. More availability! That's always good.
**And if it's not someone tripping over your power cord, it's probably DNS that broke your cluster. What's changed in terms of DNS configuration?**<br>
Oh, we introduced [a new feature gate to allow more DNS search path](https://github.com/kubernetes/enhancements/issues/2595).
**Is that all there is to it?**<br>
That's pretty much it. [LAUGHING] Yeah, you can have more and longer DNS search path.
**It can never be long enough. Just search everything! If .com doesn't work, try .net and try .io after that.**<br>
Surely.
**Those are a few of the big features that are moving to stable. Obviously, over the course of the last few releases, features come in, moving from Alpha to Beta and so on. New features coming in today might not be available to people for a while. As you mentioned, there are feature gates that you can enable to allow people to have access to these. What are some of the newest features that have been introduced that are in Alpha, that are particularly interesting to you personally?**<br>
I have two. The first one is [`kubectl delete --interactive`](https://github.com/kubernetes/enhancements/issues/3895). I'm always nervous when I delete something, you know, it's going to be a typo or it's going to be on the wrong tab. So we have an `--interactive` flag for that now.
**So you can get feedback on what you're about to delete before you do it?**<br>
Right; confirmation is good!
**You mentioned two there, what was the second one?**<br>
Right; this one is close to my heart. It is a SIG Release KEP, [publishing on community infrastructure](https://github.com/kubernetes/enhancements/issues/1731). I'm not sure if you know, but as part of my branch management associate role in 1.27, I had the opportunity to cut a few releases. It takes up to 12 hours sometimes. And now, we are hoping that the process only includes release managers, so we don't have to call up the folks at Google and, you know, lengthen that process anymore.
**Is 12 hours the expected length for software of this size, or is there work in place to try and bring that down?**<br>
There's so much work in place to bring that down. I think 12 hours is on the shorter end of it. Unfortunately, we have had a situation where we have to, you know, switch the release manager because it's just so late at night for them.
**They've fallen asleep halfway through?**<br>
Exactly, yeah. 6 to 12 hours, I think, is our status quo.
**The theme for this release is "[Planternetes](/blog/2023/08/15/kubernetes-v1-28-release/#release-theme-and-logo)". That's going to need some explanation, I feel.**<br>
Okay. I had full creative control over this. It is summer in the northern hemisphere, and I am a big house plant fanatic. It's always a little sad when I have to move cities for co-op and can't take my plants with me.
**Is that a border control thing? They don't let you take them over the border?**<br>
It's not even that; they're just so clunky and fragile. It's usually not worth the effort. But I think our community is very much like a garden. We have very critical roles in the ecosystem and we all have to work together.
**Will you be posting seeds out to contributors and growing something together all around the world?**<br>
That would be so cool if we had merch, like a little card with seeds embedded in it. I don't think we have the budget for that though. [LAUGHS]
**You say that. There are people who are inspired in many different areas. I love talking to the release managers and hearing the things that they're interested in. You should think about taking some seeds off one of your plants, and just spreading them around the world. People can take pictures, and tag you in them on Instagram.**<br>
That's cool. You know how we have a SIG Beard? We can have a SIG Plant.
**You worked for a long time with the release lead for 1.27. Xander Grzywinski. One of the benefits of having [done my interview with him in writing](https://craigbox.substack.com/p/kubernetes-and-chill) and not as a podcast is I didn't have to try and butcher pronouncing his surname. Can you help me out here?**<br>
I unfortunately cannot. I don't want to butcher it either!
**Anyway, Xander told me that he suspected that in this release you would have to deal with some very last-minute PRs, as is tradition. Was that the case?**<br>
I vividly remember the last minute PRs from last release because I was trying to cut the releases, as part of the branch management team. Thankfully, that was not the case this release. We have had other challenges, of course.
**Can you tell me some of those challenges?**<br>
I think improvement on documentation is always a big part. The KEP process can be very daunting to new contributors. How do you get people to review your KEPs? How do you opt in? All that stuff. We're improving documentations for that.
**As someone who has been through a lot of releases, I've been feeling, like you've said, that the last minute nature has slowed down a little. The process is perhaps improving. Do you see that, or do you think there's still a long way to go for the leads to improve it?**<br>
I think we've come very far. When I started in 1.22, we were using spreadsheets to track a hundred enhancements. It was a monster; I was terrified to touch it. Now, we're on GitHub boards. As a result of that, we are actually merging the bug triage and CI Signal team in 1.29.
**What's the impact of that?**<br>
The bug triage team is now using the GitHub board to track issues, which is much more efficient. We are able to merge the two teams together.
**I have heard a rumor that GitHub boards are powered by spreadsheets underneath.**<br>
Honestly, even if that's true, the fact that it's on the same platform and it has better version control is just magical.
**At this time, the next release lead has not yet been announced, but tradition dictates that you write down your feelings, best wishes and instructions to them in an envelope, which you'll leave in their desk drawer. What are you going to put inside that envelope?**<br>
Our 1.28 release lead is fantastic and they're so capable of handling the release—
**That's you, isn't it?**<br>
1.29? [LAUGHS] No, I'm too tired. I need to catch up on my sleep. My advice for them? It's going to be okay. It's all going to be okay. I was going to echo Leo's and Cici's words, to overcommunicate, but I think that has been said enough times already.
**You've communicated enough. Stop! No more communication!**<br>
Yeah, no more communication. [LAUGHS] It's going to be okay. And honestly, shout out to my emeritus advisor, Leo, for reminding me that. Sometimes there are a lot of fires and it can be overwhelming, but it will be okay.
**As we've alluded to a little bit throughout our conversation, there are a lot of people in the Kubernetes community who, for want of a better term, have had "a lot of experience" at running these systems. Then there are, of course, a lot of people who are just at the beginning of their careers; like yourself, at university. How do you see the difference between how those groups interact? Is there one team throughout, or what do you think that each can learn from the other?**<br>
I think the diversity of the team is one of its strengths and I really enjoy it. I learn so much from folks who have been doing this for 20 years or folks who are new to the industry like I am.
**I know the CNCF goes to a lot of effort to enable new people to take part. Is there anything that you can say about how people might get involved?**<br>
Firstly, I think SIG Release has started a wonderful tradition, or system, of [helping new folks join the release team as a shadow](https://github.com/kubernetes/sig-release/blob/master/release-team/shadows.md), and helping them grow into bigger positions, like leads. I think other SIGs are also following that template as well. But a big part of me joining and sticking with the community has been the ability to go to conferences. As I said, my first conference was KubeCon, when I was not involved in the community at all. And so a big shout-out to the CNCF and the companies that sponsor the Dan Kohn and the speaker scholarships. They have been the sole reason that I was able to attend KubeCon, meet people, and feel the power of the community.
**Last year's KubeCon in North America was in Detroit?**<br>
Detroit, [I was there, yeah](https://medium.com/@nng.grace/kubecon-in-the-motor-city-4e23e0446751).
**That's quite a long drive?**<br>
I was in SF, so I flew over.
**You live right next door! If only you'd been in Waterloo.**<br>
Yeah, but who knows? Maybe I'll do a road trip from Waterloo to Chicago this year.
---
_[Grace Nguyen](https://twitter.com/GraceNNG) is a student at the University of Waterloo, and was the release team lead for Kubernetes 1.28. Subscribe to [Let's Get To The News](https://craigbox.substack.com/about#§follow-the-podcast), or search for it wherever you get your podcasts._

1
content/en/docs/concepts/architecture/_index.md
View File

@ -5,3 +5,4 @@ description: >
The architectural concepts behind Kubernetes.
---
{{< figure src="/images/docs/kubernetes-cluster-architecture.svg" alt="Components of Kubernetes" caption="Kubernetes cluster architecture" class="diagram-large" >}}

4
content/en/docs/concepts/architecture/cgroups.md
View File

@ -104,8 +104,8 @@ updated to newer versions that support cgroup v2. For example:
DaemonSet for monitoring pods and containers, update it to v0.43.0 or later.
* If you deploy Java applications, prefer to use versions which fully support cgroup v2:
* [OpenJDK / HotSpot](https://bugs.openjdk.org/browse/JDK-8230305): jdk8u372, 11.0.16, 15 and later
* [IBM Semeru Runtimes](https://www.eclipse.org/openj9/docs/version0.33/#control-groups-v2-support): jdk8u345-b01, 11.0.16.0, 17.0.4.0, 18.0.2.0 and later
* [IBM Java](https://www.ibm.com/docs/en/sdk-java-technology/8?topic=new-service-refresh-7#whatsnew_sr7__fp15): 8.0.7.15 and later
* [IBM Semeru Runtimes](https://www.ibm.com/support/pages/apar/IJ46681): 8.0.382.0, 11.0.20.0, 17.0.8.0, and later
* [IBM Java](https://www.ibm.com/support/pages/apar/IJ46681): 8.0.8.6 and later
* If you are using the [uber-go/automaxprocs](https://github.com/uber-go/automaxprocs) package, make sure
the version you use is v1.5.1 or higher.

2
content/en/docs/concepts/cluster-administration/addons.md
View File

@ -37,7 +37,7 @@ installation instructions. The list does not try to be exhaustive.
network policies on L3-L7 using an identity-based security model that is
decoupled from network addressing. Cilium can act as a replacement for
kube-proxy; it also offers additional, opt-in observability and security features.
Cilium is a [CNCF project at the Incubation level](https://www.cncf.io/projects/cilium/).
Cilium is a [CNCF project at the Graduated level](https://www.cncf.io/projects/cilium/).
* [CNI-Genie](https://github.com/cni-genie/CNI-Genie) enables Kubernetes to seamlessly
connect to a choice of CNI plugins, such as Calico, Canal, Flannel, or Weave.
CNI-Genie is a [CNCF project at the Sandbox level](https://www.cncf.io/projects/cni-genie/).

67
content/en/docs/concepts/cluster-administration/flow-control.md
View File

@ -488,6 +488,8 @@ exports additional metrics. Monitoring these can help you determine whether your
configuration is inappropriately throttling important traffic, or find
poorly-behaved workloads that may be harming system health.
#### Maturity level BETA
* `apiserver_flowcontrol_rejected_requests_total` is a counter vector
(cumulative since server start) of requests that were rejected,
broken down by the labels `flow_schema` (indicating the one that
@ -509,6 +511,37 @@ poorly-behaved workloads that may be harming system health.
vector (cumulative since server start) of requests that began
executing, broken down by `flow_schema` and `priority_level`.
* `apiserver_flowcontrol_current_inqueue_requests` is a gauge vector
holding the instantaneous number of queued (not executing) requests,
broken down by `priority_level` and `flow_schema`.
* `apiserver_flowcontrol_current_executing_requests` is a gauge vector
holding the instantaneous number of executing (not waiting in a
queue) requests, broken down by `priority_level` and `flow_schema`.
* `apiserver_flowcontrol_current_executing_seats` is a gauge vector
holding the instantaneous number of occupied seats, broken down by
`priority_level` and `flow_schema`.
* `apiserver_flowcontrol_request_wait_duration_seconds` is a histogram
vector of how long requests spent queued, broken down by the labels
`flow_schema`, `priority_level`, and `execute`. The `execute` label
indicates whether the request has started executing.
{{< note >}}
Since each FlowSchema always assigns requests to a single
PriorityLevelConfiguration, you can add the histograms for all the
FlowSchemas for one priority level to get the effective histogram for
requests assigned to that priority level.
{{< /note >}}
* `apiserver_flowcontrol_nominal_limit_seats` is a gauge vector
holding each priority level's nominal concurrency limit, computed
from the API server's total concurrency limit and the priority
level's configured nominal concurrency shares.
#### Maturity level ALPHA
* `apiserver_current_inqueue_requests` is a gauge vector of recent
high water marks of the number of queued requests, grouped by a
label named `request_kind` whose value is `mutating` or `readOnly`.
@ -518,6 +551,10 @@ poorly-behaved workloads that may be harming system health.
last window's high water mark of number of requests actively being
served.
* `apiserver_current_inqueue_seats` is a gauge vector of the sum over
queued requests of the largest number of seats each will occupy,
grouped by labels named `flow_schema` and `priority_level`.
* `apiserver_flowcontrol_read_vs_write_current_requests` is a
histogram vector of observations, made at the end of every
nanosecond, of the number of requests broken down by the labels
@ -528,14 +565,6 @@ poorly-behaved workloads that may be harming system health.
number of requests (queue volume limit for waiting and concurrency
limit for executing).
* `apiserver_flowcontrol_current_inqueue_requests` is a gauge vector
holding the instantaneous number of queued (not executing) requests,
broken down by `priority_level` and `flow_schema`.
* `apiserver_flowcontrol_current_executing_requests` is a gauge vector
holding the instantaneous number of executing (not waiting in a
queue) requests, broken down by `priority_level` and `flow_schema`.
* `apiserver_flowcontrol_request_concurrency_in_use` is a gauge vector
holding the instantaneous number of occupied seats, broken down by
`priority_level` and `flow_schema`.
@ -584,11 +613,6 @@ poorly-behaved workloads that may be harming system health.
was always equal to `apiserver_flowcontrol_current_limit_seats`
(which did not exist as a distinct metric).
* `apiserver_flowcontrol_nominal_limit_seats` is a gauge vector
holding each priority level's nominal concurrency limit, computed
from the API server's total concurrency limit and the priority
level's configured nominal concurrency shares.
* `apiserver_flowcontrol_lower_limit_seats` is a gauge vector holding
the lower bound on each priority level's dynamic concurrency limit.
@ -631,18 +655,6 @@ poorly-behaved workloads that may be harming system health.
holding, for each priority level, the dynamic concurrency limit
derived in the last adjustment.
* `apiserver_flowcontrol_request_wait_duration_seconds` is a histogram
vector of how long requests spent queued, broken down by the labels
`flow_schema`, `priority_level`, and `execute`. The `execute` label
indicates whether the request has started executing.
{{< note >}}
Since each FlowSchema always assigns requests to a single
PriorityLevelConfiguration, you can add the histograms for all the
FlowSchemas for one priority level to get the effective histogram for
requests assigned to that priority level.
{{< /note >}}
* `apiserver_flowcontrol_request_execution_seconds` is a histogram
vector of how long requests took to actually execute, broken down by
`flow_schema` and `priority_level`.
@ -661,6 +673,11 @@ poorly-behaved workloads that may be harming system health.
to a request being dispatched but did not, due to lack of available
concurrency, broken down by `flow_schema` and `priority_level`.
* `apiserver_flowcontrol_epoch_advance_total` is a counter vector of
the number of attempts to jump a priority level's progress meter
backward to avoid numeric overflow, grouped by `priority_level` and
`success`.
## Good practices for using API Priority and Fairness
When a given priority level exceeds its permitted concurrency, requests can

7
content/en/docs/concepts/cluster-administration/system-logs.md
View File

@ -17,6 +17,13 @@ scheduler decisions).
<!-- body -->
{{< warning >}}
In contrast to the command line flags described here, the *log
output* itself does *not* fall under the Kubernetes API stability guarantees:
individual log entries and their formatting may change from one release
to the next!
{{< /warning >}}
## Klog
klog is the Kubernetes logging library. [klog](https://github.com/kubernetes/klog)

188
content/en/docs/concepts/configuration/secret.md
View File

@ -6,8 +6,8 @@ content_type: concept
feature:
title: Secret and configuration management
description: >
Deploy and update secrets and application configuration without rebuilding your image
and without exposing secrets in your stack configuration.
Deploy and update Secrets and application configuration without rebuilding your image
and without exposing Secrets in your stack configuration.
weight: 30
---
@ -24,7 +24,7 @@ Because Secrets can be created independently of the Pods that use them, there
is less risk of the Secret (and its data) being exposed during the workflow of
creating, viewing, and editing Pods. Kubernetes, and applications that run in
your cluster, can also take additional precautions with Secrets, such as avoiding
writing secret data to nonvolatile storage.
writing sensitive data to nonvolatile storage.
Secrets are similar to {{< glossary_tooltip text="ConfigMaps" term_id="configmap" >}}
but are specifically intended to hold confidential data.
@ -68,7 +68,7 @@ help automate node registration.
### Use case: dotfiles in a secret volume
You can make your data "hidden" by defining a key that begins with a dot.
This key represents a dotfile or "hidden" file. For example, when the following secret
This key represents a dotfile or "hidden" file. For example, when the following Secret
is mounted into a volume, `secret-volume`, the volume will contain a single file,
called `.secret-file`, and the `dotfile-test-container` will have this file
present at the path `/etc/secret-volume/.secret-file`.
@ -78,35 +78,7 @@ Files beginning with dot characters are hidden from the output of `ls -l`;
you must use `ls -la` to see them when listing directory contents.
{{< /note >}}
```yaml
apiVersion: v1
kind: Secret
metadata:
name: dotfile-secret
data:
.secret-file: dmFsdWUtMg0KDQo=
---
apiVersion: v1
kind: Pod
metadata:
name: secret-dotfiles-pod
spec:
volumes:
- name: secret-volume
secret:
secretName: dotfile-secret
containers:
- name: dotfile-test-container
image: registry.k8s.io/busybox
command:
- ls
- "-l"
- "/etc/secret-volume"
volumeMounts:
- name: secret-volume
readOnly: true
mountPath: "/etc/secret-volume"
```
{{% code language="yaml" file="secret/dotfile-secret.yaml" %}}
### Use case: Secret visible to one container in a Pod
@ -135,8 +107,8 @@ Here are some of your options:
[ServiceAccount](/docs/reference/access-authn-authz/authentication/#service-account-tokens)
and its tokens to identify your client.
- There are third-party tools that you can run, either within or outside your cluster,
that provide secrets management. For example, a service that Pods access over HTTPS,
that reveals a secret if the client correctly authenticates (for example, with a ServiceAccount
that manage sensitive data. For example, a service that Pods access over HTTPS,
that reveals a Secret if the client correctly authenticates (for example, with a ServiceAccount
token).
- For authentication, you can implement a custom signer for X.509 certificates, and use
[CertificateSigningRequests](/docs/reference/access-authn-authz/certificate-signing-requests/)
@ -251,18 +223,7 @@ fills in some other fields such as the `kubernetes.io/service-account.uid` annot
The following example configuration declares a ServiceAccount token Secret:
```yaml
apiVersion: v1
kind: Secret
metadata:
name: secret-sa-sample
annotations:
kubernetes.io/service-account.name: "sa-name"
type: kubernetes.io/service-account-token
data:
# You can include additional key value pairs as you do with Opaque Secrets
extra: YmFyCg==
```
{{% code language="yaml" file="secret/serviceaccount-token-secret.yaml" %}}
After creating the Secret, wait for Kubernetes to populate the `token` key in the `data` field.
@ -290,16 +251,7 @@ you must use one of the following `type` values for that Secret:
Below is an example for a `kubernetes.io/dockercfg` type of Secret:
```yaml
apiVersion: v1
kind: Secret
metadata:
name: secret-dockercfg
type: kubernetes.io/dockercfg
data:
.dockercfg: |
"<base64 encoded ~/.dockercfg file>"
```
{{% code language="yaml" file="secret/dockercfg-secret.yaml" %}}
{{< note >}}
If you do not want to perform the base64 encoding, you can choose to use the
@ -369,16 +321,11 @@ Secret manifest.
The following manifest is an example of a basic authentication Secret:
```yaml
apiVersion: v1
kind: Secret
metadata:
name: secret-basic-auth
type: kubernetes.io/basic-auth
stringData:
username: admin # required field for kubernetes.io/basic-auth
password: t0p-Secret # required field for kubernetes.io/basic-auth
```
{{% code language="yaml" file="secret/basicauth-secret.yaml" %}}
{{< note >}}
The `stringData` field for a Secret does not work well with server-side apply.
{{< /note >}}
The basic authentication Secret type is provided only for convenience.
You can create an `Opaque` type for credentials used for basic authentication.
@ -397,17 +344,7 @@ as the SSH credential to use.
The following manifest is an example of a Secret used for SSH public/private
key authentication:
```yaml
apiVersion: v1
kind: Secret
metadata:
name: secret-ssh-auth
type: kubernetes.io/ssh-auth
data:
# the data is abbreviated in this example
ssh-privatekey: |
MIIEpQIBAAKCAQEAulqb/Y ...
```
{{% code language="yaml" file="secret/ssh-auth-secret.yaml" %}}
The SSH authentication Secret type is provided only for convenience.
You can create an `Opaque` type for credentials used for SSH authentication.
@ -440,21 +377,7 @@ the base64 encoded certificate and private key. For details, see
The following YAML contains an example config for a TLS Secret:
```yaml
apiVersion: v1
kind: Secret
metadata:
name: secret-tls
type: kubernetes.io/tls
stringData:
# the data is abbreviated in this example
tls.crt: |
--------BEGIN CERTIFICATE-----
MIIC2DCCAcCgAwIBAgIBATANBgkqh ...
tls.key: |
-----BEGIN RSA PRIVATE KEY-----
MIIEpgIBAAKCAQEA7yn3bRHQ5FHMQ ...
```
{{% code language="yaml" file="secret/tls-auth-secret.yaml" %}}
The TLS Secret type is provided only for convenience.
You can create an `Opaque` type for credentials used for TLS authentication.
@ -486,26 +409,12 @@ string of the token ID.
As a Kubernetes manifest, a bootstrap token Secret might look like the
following:
```yaml
apiVersion: v1
kind: Secret
metadata:
name: bootstrap-token-5emitj
namespace: kube-system
type: bootstrap.kubernetes.io/token
data:
auth-extra-groups: c3lzdGVtOmJvb3RzdHJhcHBlcnM6a3ViZWFkbTpkZWZhdWx0LW5vZGUtdG9rZW4=
expiration: MjAyMC0wOS0xM1QwNDozOToxMFo=
token-id: NWVtaXRq
token-secret: a3E0Z2lodnN6emduMXAwcg==
usage-bootstrap-authentication: dHJ1ZQ==
usage-bootstrap-signing: dHJ1ZQ==
```
{{% code language="yaml" file="secret/bootstrap-token-secret-base64.yaml" %}}
A bootstrap token Secret has the following keys specified under `data`:
- `token-id`: A random 6 character string as the token identifier. Required.
- `token-secret`: A random 16 character string as the actual token secret. Required.
- `token-secret`: A random 16 character string as the actual token Secret. Required.
- `description`: A human-readable string that describes what the token is
used for. Optional.
- `expiration`: An absolute UTC time using [RFC3339](https://datatracker.ietf.org/doc/html/rfc3339) specifying when the token
@ -518,26 +427,11 @@ A bootstrap token Secret has the following keys specified under `data`:
You can alternatively provide the values in the `stringData` field of the Secret
without base64 encoding them:
```yaml
apiVersion: v1
kind: Secret
metadata:
# Note how the Secret is named
name: bootstrap-token-5emitj
# A bootstrap token Secret usually resides in the kube-system namespace
namespace: kube-system
type: bootstrap.kubernetes.io/token
stringData:
auth-extra-groups: "system:bootstrappers:kubeadm:default-node-token"
expiration: "2020-09-13T04:39:10Z"
# This token ID is used in the name
token-id: "5emitj"
token-secret: "kq4gihvszzgn1p0r"
# This token can be used for authentication
usage-bootstrap-authentication: "true"
# and it can be used for signing
usage-bootstrap-signing: "true"
```
{{% code language="yaml" file="secret/bootstrap-token-secret-literal.yaml" %}}
{{< note >}}
The `stringData` field for a Secret does not work well with server-side apply.
{{< /note >}}
## Working with Secrets
@ -568,9 +462,9 @@ precedence.
#### Size limit {#restriction-data-size}
Individual secrets are limited to 1MiB in size. This is to discourage creation
of very large secrets that could exhaust the API server and kubelet memory.
However, creation of many smaller secrets could also exhaust memory. You can
Individual Secrets are limited to 1MiB in size. This is to discourage creation
of very large Secrets that could exhaust the API server and kubelet memory.
However, creation of many smaller Secrets could also exhaust memory. You can
use a [resource quota](/docs/concepts/policy/resource-quotas/) to limit the
number of Secrets (or other resources) in a namespace.
@ -613,25 +507,7 @@ When you reference a Secret in a Pod, you can mark the Secret as _optional_,
such as in the following example. If an optional Secret doesn't exist,
Kubernetes ignores it.
```yaml
apiVersion: v1
kind: Pod
metadata:
name: mypod
spec:
containers:
- name: mypod
image: redis
volumeMounts:
- name: foo
mountPath: "/etc/foo"
readOnly: true
volumes:
- name: foo
secret:
secretName: mysecret
optional: true
```
{{% code language="yaml" file="secret/optional-secret.yaml" %}}
By default, Secrets are required. None of a Pod's containers will start until
all non-optional Secrets are available.
@ -708,17 +584,17 @@ LASTSEEN FIRSTSEEN COUNT NAME KIND SUBOBJECT
0s 0s 1 dapi-test-pod Pod Warning InvalidEnvironmentVariableNames kubelet, 127.0.0.1 Keys [1badkey, 2alsobad] from the EnvFrom secret default/mysecret were skipped since they are considered invalid environment variable names.
```
### Container image pull secrets {#using-imagepullsecrets}
### Container image pull Secrets {#using-imagepullsecrets}
If you want to fetch container images from a private repository, you need a way for
the kubelet on each node to authenticate to that repository. You can configure
_image pull secrets_ to make this possible. These secrets are configured at the Pod
_image pull Secrets_ to make this possible. These Secrets are configured at the Pod
level.
#### Using imagePullSecrets
The `imagePullSecrets` field is a list of references to secrets in the same namespace.
You can use an `imagePullSecrets` to pass a secret that contains a Docker (or other) image registry
The `imagePullSecrets` field is a list of references to Secrets in the same namespace.
You can use an `imagePullSecrets` to pass a Secret that contains a Docker (or other) image registry
password to the kubelet. The kubelet uses this information to pull a private image on behalf of your Pod.
See the [PodSpec API](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#podspec-v1-core)
for more information about the `imagePullSecrets` field.
@ -787,7 +663,7 @@ Secrets it expects to interact with, other apps within the same namespace can
render those assumptions invalid.
A Secret is only sent to a node if a Pod on that node requires it.
For mounting secrets into Pods, the kubelet stores a copy of the data into a `tmpfs`
For mounting Secrets into Pods, the kubelet stores a copy of the data into a `tmpfs`
so that the confidential data is not written to durable storage.
Once the Pod that depends on the Secret is deleted, the kubelet deletes its local copy
of the confidential data from the Secret.

42
content/en/docs/concepts/containers/images.md
View File

@ -265,38 +265,26 @@ See [Configure a kubelet image credential provider](/docs/tasks/administer-clust
The interpretation of `config.json` varies between the original Docker
implementation and the Kubernetes interpretation. In Docker, the `auths` keys
can only specify root URLs, whereas Kubernetes allows glob URLs as well as
prefix-matched paths. This means that a `config.json` like this is valid:
prefix-matched paths. The only limitation is that glob patterns (`*`) have to
include the dot (`.`) for each subdomain. The amount of matched subdomains has
to be equal to the amount of glob patterns (`*.`), for example:
- `*.kubernetes.io` will *not* match `kubernetes.io`, but `abc.kubernetes.io`
- `*.*.kubernetes.io` will *not* match `abc.kubernetes.io`, but `abc.def.kubernetes.io`
- `prefix.*.io` will match `prefix.kubernetes.io`
- `*-good.kubernetes.io` will match `prefix-good.kubernetes.io`
This means that a `config.json` like this is valid:
```json
{
"auths": {
"*my-registry.io/images": {
"auth": "…"
}
"my-registry.io/images": { "auth": "…" },
"*.my-registry.io/images": { "auth": "…" }
}
}
```
The root URL (`*my-registry.io`) is matched by using the following syntax:
```
pattern:
{ term }
term:
'*' matches any sequence of non-Separator characters
'?' matches any single non-Separator character
'[' [ '^' ] { character-range } ']'
character class (must be non-empty)
c matches character c (c != '*', '?', '\\', '[')
'\\' c matches character c
character-range:
c matches character c (c != '\\', '-', ']')
'\\' c matches character c
lo '-' hi matches character c for lo <= c <= hi
```
Image pull operations would now pass the credentials to the CRI container
runtime for every valid pattern. For example the following container image names
would match successfully:
@ -305,10 +293,14 @@ would match successfully:
- `my-registry.io/images/my-image`
- `my-registry.io/images/another-image`
- `sub.my-registry.io/images/my-image`
But not:
- `a.sub.my-registry.io/images/my-image`
- `a.b.sub.my-registry.io/images/my-image`
The kubelet performs image pulls sequentially for every found credential. This
means, that multiple entries in `config.json` are possible, too:
means, that multiple entries in `config.json` for different paths are possible, too:
```json
{

4
content/en/docs/concepts/extend-kubernetes/compute-storage-net/network-plugins.md
View File

@ -172,3 +172,7 @@ metadata:
## {{% heading "whatsnext" %}}
- Learn more about [Cluster Networking](/docs/concepts/cluster-administration/networking/)
- Learn more about [Network Policies](/docs/concepts/services-networking/network-policies/)
- Learn about the [Troubleshooting CNI plugin-related errors](/docs/tasks/administer-cluster/migrating-from-dockershim/troubleshooting-cni-plugin-related-errors/)

2
content/en/docs/concepts/extend-kubernetes/operator.md
View File

@ -129,7 +129,7 @@ operator.
* Read the {{< glossary_tooltip text="CNCF" term_id="cncf" >}}
[Operator White Paper](https://github.com/cncf/tag-app-delivery/blob/eece8f7307f2970f46f100f51932db106db46968/operator-wg/whitepaper/Operator-WhitePaper_v1-0.md).
[Operator White Paper](https://github.com/cncf/tag-app-delivery/blob/163962c4b1cd70d085107fc579e3e04c2e14d59c/operator-wg/whitepaper/Operator-WhitePaper_v1-0.md).
* Learn more about [Custom Resources](/docs/concepts/extend-kubernetes/api-extension/custom-resources/)
* Find ready-made operators on [OperatorHub.io](https://operatorhub.io/) to suit your use case
* [Publish](https://operatorhub.io/) your operator for other people to use

8
content/en/docs/concepts/overview/_index.md
View File

@ -129,6 +129,14 @@ Kubernetes provides you with:
Kubernetes lets you store and manage sensitive information, such as passwords, OAuth tokens,
and SSH keys. You can deploy and update secrets and application configuration without
rebuilding your container images, and without exposing secrets in your stack configuration.
* **Batch execution**
In addition to services, Kubernetes can manage your batch and CI workloads, replacing containers that fail, if desired.
* **Horizontal scaling**
Scale your application up and down with a simple command, with a UI, or automatically based on CPU usage.
* **IPv4/IPv6 dual-stack**
Allocation of IPv4 and IPv6 addresses to Pods and Services
* **Designed for extensibility**
Add features to your Kubernetes cluster without changing upstream source code.
## What Kubernetes is not

12
content/en/docs/concepts/policy/resource-quotas.md
View File

@ -465,7 +465,7 @@ from getting scheduled in a failure domain.
Using this scope operators can prevent certain namespaces (`foo-ns` in the example below)
from having pods that use cross-namespace pod affinity by creating a resource quota object in
that namespace with `CrossNamespaceAffinity` scope and hard limit of 0:
that namespace with `CrossNamespacePodAffinity` scope and hard limit of 0:
```yaml
apiVersion: v1
@ -478,11 +478,12 @@ spec:
pods: "0"
scopeSelector:
matchExpressions:
- scopeName: CrossNamespaceAffinity
- scopeName: CrossNamespacePodAffinity
operator: Exists
```
If operators want to disallow using `namespaces` and `namespaceSelector` by default, and
only allow it for specific namespaces, they could configure `CrossNamespaceAffinity`
only allow it for specific namespaces, they could configure `CrossNamespacePodAffinity`
as a limited resource by setting the kube-apiserver flag --admission-control-config-file
to the path of the following configuration file:
@ -497,12 +498,13 @@ plugins:
limitedResources:
- resource: pods
matchScopes:
- scopeName: CrossNamespaceAffinity
- scopeName: CrossNamespacePodAffinity
operator: Exists
```
With the above configuration, pods can use `namespaces` and `namespaceSelector` in pod affinity only
if the namespace where they are created have a resource quota object with
`CrossNamespaceAffinity` scope and a hard limit greater than or equal to the number of pods using those fields.
`CrossNamespacePodAffinity` scope and a hard limit greater than or equal to the number of pods using those fields.
## Requests compared to Limits {#requests-vs-limits}

35
content/en/docs/concepts/scheduling-eviction/assign-pod-node.md
View File

@ -35,8 +35,10 @@ specific Pods:
## Node labels {#built-in-node-labels}
Like many other Kubernetes objects, nodes have
[labels](/docs/concepts/overview/working-with-objects/labels/). You can [attach labels manually](/docs/tasks/configure-pod-container/assign-pods-nodes/#add-a-label-to-a-node).
Kubernetes also populates a [standard set of labels](/docs/reference/node/node-labels/) on all nodes in a cluster.
[labels](/docs/concepts/overview/working-with-objects/labels/). You can
[attach labels manually](/docs/tasks/configure-pod-container/assign-pods-nodes/#add-a-label-to-a-node).
Kubernetes also populates a [standard set of labels](/docs/reference/node/node-labels/)
on all nodes in a cluster.
{{<note>}}
The value of these labels is cloud provider specific and is not guaranteed to be reliable.
@ -303,17 +305,23 @@ Pod affinity rule uses the "hard"
`requiredDuringSchedulingIgnoredDuringExecution`, while the anti-affinity rule
uses the "soft" `preferredDuringSchedulingIgnoredDuringExecution`.
The affinity rule says that the scheduler can only schedule a Pod onto a node if
the node is in the same zone as one or more existing Pods with the label
`security=S1`. More precisely, the scheduler must place the Pod on a node that has the
`topology.kubernetes.io/zone=V` label, as long as there is at least one node in
that zone that currently has one or more Pods with the Pod label `security=S1`.
The affinity rule specifies that the scheduler is allowed to place the example Pod
on a node only if that node belongs to a specific [zone](/docs/concepts/scheduling-eviction/topology-spread-constraints/topology-spread-constraints/)
where other Pods have been labeled with `security=S1`.
For instance, if we have a cluster with a designated zone, let's call it "Zone V,"
consisting of nodes labeled with `topology.kubernetes.io/zone=V`, the scheduler can
assign the Pod to any node within Zone V, as long as there is at least one Pod within
Zone V already labeled with `security=S1`. Conversely, if there are no Pods with `security=S1`
labels in Zone V, the scheduler will not assign the example Pod to any node in that zone.
The anti-affinity rule says that the scheduler should try to avoid scheduling
the Pod onto a node that is in the same zone as one or more Pods with the label
`security=S2`. More precisely, the scheduler should try to avoid placing the Pod on a node that has the
`topology.kubernetes.io/zone=R` label if there are other nodes in the
same zone currently running Pods with the `Security=S2` Pod label.
The anti-affinity rule specifies that the scheduler should try to avoid scheduling the Pod
on a node if that node belongs to a specific [zone](/docs/concepts/scheduling-eviction/topology-spread-constraints/topology-spread-constraints/)
where other Pods have been labeled with `security=S2`.
For instance, if we have a cluster with a designated zone, let's call it "Zone R,"
consisting of nodes labeled with `topology.kubernetes.io/zone=R`, the scheduler should avoid
assigning the Pod to any node within Zone R, as long as there is at least one Pod within
Zone R already labeled with `security=S2`. Conversely, the anti-affinity rule does not impact
scheduling into Zone R if there are no Pods with `security=S2` labels.
To get yourself more familiar with the examples of Pod affinity and anti-affinity,
refer to the [design proposal](https://git.k8s.io/design-proposals-archive/scheduling/podaffinity.md).
@ -327,7 +335,8 @@ to learn more about how these work.
In principle, the `topologyKey` can be any allowed label key with the following
exceptions for performance and security reasons:
- For Pod affinity and anti-affinity, an empty `topologyKey` field is not allowed in both `requiredDuringSchedulingIgnoredDuringExecution`
- For Pod affinity and anti-affinity, an empty `topologyKey` field is not allowed in both
`requiredDuringSchedulingIgnoredDuringExecution`
and `preferredDuringSchedulingIgnoredDuringExecution`.
- For `requiredDuringSchedulingIgnoredDuringExecution` Pod anti-affinity rules,
the admission controller `LimitPodHardAntiAffinityTopology` limits

12
content/en/docs/concepts/scheduling-eviction/node-pressure-eviction.md
View File

@ -105,13 +105,11 @@ does not support other configurations.
Some kubelet garbage collection features are deprecated in favor of eviction:
| Existing Flag | New Flag | Rationale |
| ------------- | -------- | --------- |
| `--image-gc-high-threshold` | `--eviction-hard` or `--eviction-soft` | existing eviction signals can trigger image garbage collection |
| `--image-gc-low-threshold` | `--eviction-minimum-reclaim` | eviction reclaims achieve the same behavior |
| `--maximum-dead-containers` | - | deprecated once old logs are stored outside of container's context |
| `--maximum-dead-containers-per-container` | - | deprecated once old logs are stored outside of container's context |
| `--minimum-container-ttl-duration` | - | deprecated once old logs are stored outside of container's context |
| Existing Flag | Rationale |
| ------------- | --------- |
| `--maximum-dead-containers` | deprecated once old logs are stored outside of container's context |
| `--maximum-dead-containers-per-container` | deprecated once old logs are stored outside of container's context |
| `--minimum-container-ttl-duration` | deprecated once old logs are stored outside of container's context |
### Eviction thresholds

2
content/en/docs/concepts/scheduling-eviction/pod-scheduling-readiness.md
View File

@ -6,7 +6,7 @@ weight: 40
<!-- overview -->
{{< feature-state for_k8s_version="v1.26" state="alpha" >}}
{{< feature-state for_k8s_version="v1.27" state="beta" >}}
Pods were considered ready for scheduling once created. Kubernetes scheduler
does its due diligence to find nodes to place all pending Pods. However, in a

35
content/en/docs/concepts/scheduling-eviction/taint-and-toleration.md
View File

@ -85,9 +85,27 @@ An empty `effect` matches all effects with key `key1`.
{{< /note >}}
The above example used `effect` of `NoSchedule`. Alternatively, you can use `effect` of `PreferNoSchedule`.
This is a "preference" or "soft" version of `NoSchedule` -- the system will *try* to avoid placing a
pod that does not tolerate the taint on the node, but it is not required. The third kind of `effect` is
`NoExecute`, described later.
The allowed values for the `effect` field are:
`NoExecute`
: This affects pods that are already running on the node as follows:
* Pods that do not tolerate the taint are evicted immediately
* Pods that tolerate the taint without specifying `tolerationSeconds` in
their toleration specification remain bound forever
* Pods that tolerate the taint with a specified `tolerationSeconds` remain
bound for the specified amount of time. After that time elapses, the node
lifecycle controller evicts the Pods from the node.
`NoSchedule`
: No new Pods will be scheduled on the tainted node unless they have a matching
toleration. Pods currently running on the node are **not** evicted.
`PreferNoSchedule`
: `PreferNoSchedule` is a "preference" or "soft" version of `NoSchedule`.
The control plane will *try* to avoid placing a Pod that does not tolerate
the taint on the node, but it is not guaranteed.
You can put multiple taints on the same node and multiple tolerations on the same pod.
The way Kubernetes processes multiple taints and tolerations is like a filter: start
@ -194,14 +212,7 @@ when there are node problems, which is described in the next section.
{{< feature-state for_k8s_version="v1.18" state="stable" >}}
The `NoExecute` taint effect, mentioned above, affects pods that are already
running on the node as follows
* pods that do not tolerate the taint are evicted immediately
* pods that tolerate the taint without specifying `tolerationSeconds` in
their toleration specification remain bound forever
* pods that tolerate the taint with a specified `tolerationSeconds` remain
bound for the specified amount of time
The node controller automatically taints a Node when certain conditions
are true. The following taints are built in:
@ -221,7 +232,9 @@ are true. The following taints are built in:
this node, the kubelet removes this taint.
In case a node is to be drained, the node controller or the kubelet adds relevant taints
with `NoExecute` effect. If the fault condition returns to normal the kubelet or node
with `NoExecute` effect. This effect is added by default for the
`node.kubernetes.io/not-ready` and `node.kubernetes.io/unreachable` taints.
If the fault condition returns to normal, the kubelet or node
controller can remove the relevant taint(s).
In some cases when the node is unreachable, the API server is unable to communicate

2
content/en/docs/concepts/services-networking/ingress-controllers.md
View File

@ -28,6 +28,7 @@ Kubernetes as a project supports and maintains [AWS](https://github.com/kubernet
{{% thirdparty-content %}}
* [AKS Application Gateway Ingress Controller](https://docs.microsoft.com/azure/application-gateway/tutorial-ingress-controller-add-on-existing?toc=https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Faks%2Ftoc.json&bc=https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fbread%2Ftoc.json) is an ingress controller that configures the [Azure Application Gateway](https://docs.microsoft.com/azure/application-gateway/overview).
* [Alibaba Cloud MSE Ingress](https://www.alibabacloud.com/help/en/mse/user-guide/overview-of-mse-ingress-gateways) is an ingress controller that configures the [Alibaba Cloud Native Gateway](https://www.alibabacloud.com/help/en/mse/product-overview/cloud-native-gateway-overview?spm=a2c63.p38356.0.0.20563003HJK9is), which is also the commercial version of [Higress](https://github.com/alibaba/higress).
* [Apache APISIX ingress controller](https://github.com/apache/apisix-ingress-controller) is an [Apache APISIX](https://github.com/apache/apisix)-based ingress controller.
* [Avi Kubernetes Operator](https://github.com/vmware/load-balancer-and-ingress-services-for-kubernetes) provides L4-L7 load-balancing using [VMware NSX Advanced Load Balancer](https://avinetworks.com/).
* [BFE Ingress Controller](https://github.com/bfenetworks/ingress-bfe) is a [BFE](https://www.bfe-networks.net)-based ingress controller.
@ -46,6 +47,7 @@ Kubernetes as a project supports and maintains [AWS](https://github.com/kubernet
which offers API gateway functionality.
* [HAProxy Ingress](https://haproxy-ingress.github.io/) is an ingress controller for
[HAProxy](https://www.haproxy.org/#desc).
* [Higress](https://github.com/alibaba/higress) is an [Envoy](https://www.envoyproxy.io) based API gateway that can run as an ingress controller.
* The [HAProxy Ingress Controller for Kubernetes](https://github.com/haproxytech/kubernetes-ingress#readme)
is also an ingress controller for [HAProxy](https://www.haproxy.org/#desc).
* [Istio Ingress](https://istio.io/latest/docs/tasks/traffic-management/ingress/kubernetes-ingress/)

6
content/en/docs/concepts/services-networking/ingress.md
View File

@ -84,7 +84,7 @@ is the [rewrite-target annotation](https://github.com/kubernetes/ingress-nginx/b
Different [Ingress controllers](/docs/concepts/services-networking/ingress-controllers) support different annotations.
Review the documentation for your choice of Ingress controller to learn which annotations are supported.
The Ingress [spec](https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#spec-and-status)
The [Ingress spec](/docs/reference/kubernetes-api/service-resources/ingress-v1/#IngressSpec)
has all the information needed to configure a load balancer or proxy server. Most importantly, it
contains a list of rules matched against all incoming requests. Ingress resource only supports rules
for directing HTTP(S) traffic.
@ -94,8 +94,8 @@ should be defined.
There are some ingress controllers, that work without the definition of a
default `IngressClass`. For example, the Ingress-NGINX controller can be
configured with a [flag](https://kubernetes.github.io/ingress-nginx/#what-is-the-flag-watch-ingress-without-class)
`--watch-ingress-without-class`. It is [recommended](https://kubernetes.github.io/ingress-nginx/#i-have-only-one-instance-of-the-ingresss-nginx-controller-in-my-cluster-what-should-i-do) though, to specify the
configured with a [flag](https://kubernetes.github.io/ingress-nginx/user-guide/k8s-122-migration/#what-is-the-flag-watch-ingress-without-class)
`--watch-ingress-without-class`. It is [recommended](https://kubernetes.github.io/ingress-nginx/user-guide/k8s-122-migration/#i-have-only-one-ingress-controller-in-my-cluster-what-should-i-do) though, to specify the
default `IngressClass` as shown [below](#default-ingress-class).
### Ingress rules

114
content/en/docs/concepts/services-networking/service.md
View File

@ -175,7 +175,6 @@ spec:
targetPort: http-web-svc
```
This works even if there is a mixture of Pods in the Service using a single
configured name, with the same network protocol available via different
port numbers. This offers a lot of flexibility for deploying and evolving
@ -269,7 +268,8 @@ as a destination.
{{< /note >}}
For an EndpointSlice that you create yourself, or in your own code,
you should also pick a value to use for the [`endpointslice.kubernetes.io/managed-by`](/docs/reference/labels-annotations-taints/#endpointslicekubernetesiomanaged-by) label.
you should also pick a value to use for the label
[`endpointslice.kubernetes.io/managed-by`](/docs/reference/labels-annotations-taints/#endpointslicekubernetesiomanaged-by).
If you create your own controller code to manage EndpointSlices, consider using a
value similar to `"my-domain.example/name-of-controller"`. If you are using a third
party tool, use the name of the tool in all-lowercase and change spaces and other
@ -283,7 +283,8 @@ managed by Kubernetes' own control plane.
#### Accessing a Service without a selector {#service-no-selector-access}
Accessing a Service without a selector works the same as if it had a selector.
In the [example](#services-without-selectors) for a Service without a selector, traffic is routed to one of the two endpoints defined in
In the [example](#services-without-selectors) for a Service without a selector,
traffic is routed to one of the two endpoints defined in
the EndpointSlice manifest: a TCP connection to 10.1.2.3 or 10.4.5.6, on port 9376.
{{< note >}}
@ -334,8 +335,7 @@ affects the legacy Endpoints API.
In that case, Kubernetes selects at most 1000 possible backend endpoints to store
into the Endpoints object, and sets an
{{< glossary_tooltip text="annotation" term_id="annotation" >}} on the
Endpoints:
{{< glossary_tooltip text="annotation" term_id="annotation" >}} on the Endpoints:
[`endpoints.kubernetes.io/over-capacity: truncated`](/docs/reference/labels-annotations-taints/#endpoints-kubernetes-io-over-capacity).
The control plane also removes that annotation if the number of backend Pods drops below 1000.
@ -349,7 +349,8 @@ The same API limit means that you cannot manually update an Endpoints to have mo
{{< feature-state for_k8s_version="v1.20" state="stable" >}}
The `appProtocol` field provides a way to specify an application protocol for
each Service port. This is used as a hint for implementations to offer richer behavior for protocols that they understand.
each Service port. This is used as a hint for implementations to offer
richer behavior for protocols that they understand.
The value of this field is mirrored by the corresponding
Endpoints and EndpointSlice objects.
@ -365,8 +366,6 @@ This field follows standard Kubernetes label syntax. Valid values are one of:
|----------|-------------|
| `kubernetes.io/h2c` | HTTP/2 over cleartext as described in [RFC 7540](https://www.rfc-editor.org/rfc/rfc7540) |
### Multi-port Services
For some Services, you need to expose more than one port.
@ -402,7 +401,6 @@ also start and end with an alphanumeric character.
For example, the names `123-abc` and `web` are valid, but `123_abc` and `-web` are not.
{{< /note >}}
## Service type {#publishing-services-service-types}
For some parts of your application (for example, frontends) you may want to expose a
@ -417,7 +415,8 @@ The available `type` values and their behaviors are:
: Exposes the Service on a cluster-internal IP. Choosing this value
makes the Service only reachable from within the cluster. This is the
default that is used if you don't explicitly specify a `type` for a Service.
You can expose the Service to the public internet using an [Ingress](/docs/concepts/services-networking/ingress/) or a
You can expose the Service to the public internet using an
[Ingress](/docs/concepts/services-networking/ingress/) or a
[Gateway](https://gateway-api.sigs.k8s.io/).
[`NodePort`](#type-nodeport)
@ -437,8 +436,9 @@ The available `type` values and their behaviors are:
No proxying of any kind is set up.
The `type` field in the Service API is designed as nested functionality - each level
adds to the previous. This is not strictly required on all cloud providers, but
the Kubernetes API design for Service requires it anyway.
adds to the previous. However there is an exception to this nested design. You can
define a `LoadBalancer` Service by
[disabling the load balancer `NodePort` allocation](/docs/concepts/services-networking/service/#load-balancer-nodeport-allocation).
### `type: ClusterIP` {#type-clusterip}
@ -508,11 +508,13 @@ spec:
selector:
app.kubernetes.io/name: MyApp
ports:
# By default and for convenience, the `targetPort` is set to the same value as the `port` field.
- port: 80
# By default and for convenience, the `targetPort` is set to
# the same value as the `port` field.
targetPort: 80
# Optional field
# By default and for convenience, the Kubernetes control plane will allocate a port from a range (default: 30000-32767)
# By default and for convenience, the Kubernetes control plane
# will allocate a port from a range (default: 30000-32767)
nodePort: 30007
```
@ -538,8 +540,7 @@ control plane).
If you want to specify particular IP address(es) to proxy the port, you can set the
`--nodeport-addresses` flag for kube-proxy or the equivalent `nodePortAddresses`
field of the
[kube-proxy configuration file](/docs/reference/config-api/kube-proxy-config.v1alpha1/)
field of the [kube-proxy configuration file](/docs/reference/config-api/kube-proxy-config.v1alpha1/)
to particular IP block(s).
This flag takes a comma-delimited list of IP blocks (e.g. `10.0.0.0/8`, `192.0.2.0/25`)
@ -553,7 +554,8 @@ This means that kube-proxy should consider all available network interfaces for
{{< note >}}
This Service is visible as `<NodeIP>:spec.ports[*].nodePort` and `.spec.clusterIP:spec.ports[*].port`.
If the `--nodeport-addresses` flag for kube-proxy or the equivalent field
in the kube-proxy configuration file is set, `<NodeIP>` would be a filtered node IP address (or possibly IP addresses).
in the kube-proxy configuration file is set, `<NodeIP>` would be a filtered
node IP address (or possibly IP addresses).
{{< /note >}}
### `type: LoadBalancer` {#loadbalancer}
@ -607,7 +609,8 @@ set is ignored.
{{< note >}}
The`.spec.loadBalancerIP` field for a Service was deprecated in Kubernetes v1.24.
This field was under-specified and its meaning varies across implementations. It also cannot support dual-stack networking. This field may be removed in a future API version.
This field was under-specified and its meaning varies across implementations.
It also cannot support dual-stack networking. This field may be removed in a future API version.
If you're integrating with a provider that supports specifying the load balancer IP address(es)
for a Service via a (provider specific) annotation, you should switch to doing that.
@ -703,117 +706,97 @@ depending on the cloud service provider you're using:
{{% tab name="Default" %}}
Select one of the tabs.
{{% /tab %}}
{{% tab name="GCP" %}}
```yaml
[...]
metadata:
name: my-service
annotations:
networking.gke.io/load-balancer-type: "Internal"
[...]
name: my-service
annotations:
networking.gke.io/load-balancer-type: "Internal"
```
{{% /tab %}}
{{% tab name="AWS" %}}
```yaml
[...]
metadata:
name: my-service
annotations:
service.beta.kubernetes.io/aws-load-balancer-internal: "true"
[...]
```
{{% /tab %}}
{{% tab name="Azure" %}}
```yaml
[...]
metadata:
name: my-service
annotations:
service.beta.kubernetes.io/azure-load-balancer-internal: "true"
[...]
name: my-service
annotations:
service.beta.kubernetes.io/azure-load-balancer-internal: "true"
```
{{% /tab %}}
{{% tab name="IBM Cloud" %}}
```yaml
[...]
metadata:
name: my-service
annotations:
service.kubernetes.io/ibm-load-balancer-cloud-provider-ip-type: "private"
[...]
name: my-service
annotations:
service.kubernetes.io/ibm-load-balancer-cloud-provider-ip-type: "private"
```
{{% /tab %}}
{{% tab name="OpenStack" %}}
```yaml
[...]
metadata:
name: my-service
annotations:
service.beta.kubernetes.io/openstack-internal-load-balancer: "true"
[...]
name: my-service
annotations:
service.beta.kubernetes.io/openstack-internal-load-balancer: "true"
```
{{% /tab %}}
{{% tab name="Baidu Cloud" %}}
```yaml
[...]
metadata:
name: my-service
annotations:
service.beta.kubernetes.io/cce-load-balancer-internal-vpc: "true"
[...]
name: my-service
annotations:
service.beta.kubernetes.io/cce-load-balancer-internal-vpc: "true"
```
{{% /tab %}}
{{% tab name="Tencent Cloud" %}}
```yaml
[...]
metadata:
annotations:
service.kubernetes.io/qcloud-loadbalancer-internal-subnetid: subnet-xxxxx
[...]
```
{{% /tab %}}
{{% tab name="Alibaba Cloud" %}}
```yaml
[...]
metadata:
annotations:
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-address-type: "intranet"
[...]
```
{{% /tab %}}
{{% tab name="OCI" %}}
```yaml
[...]
metadata:
name: my-service
annotations:
service.beta.kubernetes.io/oci-load-balancer-internal: true
[...]
name: my-service
annotations:
service.beta.kubernetes.io/oci-load-balancer-internal: true
```
{{% /tab %}}
{{< /tabs >}}
### `type: ExternalName` {#externalname}
Services of type ExternalName map a Service to a DNS name, not to a typical selector such as
`my-service` or `cassandra`. You specify these Services with the `spec.externalName` parameter.
@ -832,11 +815,14 @@ spec:
```
{{< note >}}
A Service of `type: ExternalName` accepts an IPv4 address string, but treats that string as a DNS name comprised of digits,
not as an IP address (the internet does not however allow such names in DNS). Services with external names that resemble IPv4
A Service of `type: ExternalName` accepts an IPv4 address string,
but treats that string as a DNS name comprised of digits,
not as an IP address (the internet does not however allow such names in DNS).
Services with external names that resemble IPv4
addresses are not resolved by DNS servers.
If you want to map a Service directly to a specific IP address, consider using [headless Services](#headless-services).
If you want to map a Service directly to a specific IP address, consider using
[headless Services](#headless-services).
{{< /note >}}
When looking up the host `my-service.prod.svc.cluster.local`, the cluster DNS Service
@ -902,7 +888,8 @@ finding a Service: environment variables and DNS.
When a Pod is run on a Node, the kubelet adds a set of environment variables
for each active Service. It adds `{SVCNAME}_SERVICE_HOST` and `{SVCNAME}_SERVICE_PORT` variables,
where the Service name is upper-cased and dashes are converted to underscores.
It also supports variables (see [makeLinkVariables](https://github.com/kubernetes/kubernetes/blob/dd2d12f6dc0e654c15d5db57a5f9f6ba61192726/pkg/kubelet/envvars/envvars.go#L72))
It also supports variables
(see [makeLinkVariables](https://github.com/kubernetes/kubernetes/blob/dd2d12f6dc0e654c15d5db57a5f9f6ba61192726/pkg/kubelet/envvars/envvars.go#L72))
that are compatible with Docker Engine's
"_[legacy container links](https://docs.docker.com/network/links/)_" feature.
@ -1034,7 +1021,9 @@ about the [Service API object](/docs/reference/generated/kubernetes-api/{{< para
## {{% heading "whatsnext" %}}
Learn more about Services and how they fit into Kubernetes:
* Follow the [Connecting Applications with Services](/docs/tutorials/services/connect-applications-service/) tutorial.
* Follow the [Connecting Applications with Services](/docs/tutorials/services/connect-applications-service/)
tutorial.
* Read about [Ingress](/docs/concepts/services-networking/ingress/), which
exposes HTTP and HTTPS routes from outside the cluster to Services within
your cluster.
@ -1042,6 +1031,7 @@ Learn more about Services and how they fit into Kubernetes:
Kubernetes that provides more flexibility than Ingress.
For more context, read the following:
* [Virtual IPs and Service Proxies](/docs/reference/networking/virtual-ips/)
* [EndpointSlices](/docs/concepts/services-networking/endpoint-slices/)
* [Service API reference](/docs/reference/kubernetes-api/service-resources/service-v1/)

3
content/en/docs/concepts/storage/ephemeral-volumes.md
View File

@ -47,8 +47,7 @@ different purposes:
[secret](/docs/concepts/storage/volumes/#secret): inject different
kinds of Kubernetes data into a Pod
- [CSI ephemeral volumes](#csi-ephemeral-volumes):
similar to the previous volume kinds, but provided by special
[CSI drivers](https://github.com/container-storage-interface/spec/blob/master/spec.md)
similar to the previous volume kinds, but provided by special {{< glossary_tooltip text="CSI" term_id="csi" >}} drivers
which specifically [support this feature](https://kubernetes-csi.github.io/docs/ephemeral-local-volumes.html)
- [generic ephemeral volumes](#generic-ephemeral-volumes), which
can be provided by all storage drivers that also support persistent volumes

5
content/en/docs/concepts/storage/volumes.md
View File

@ -245,9 +245,8 @@ The `emptyDir.medium` field controls where `emptyDir` volumes are stored. By
default `emptyDir` volumes are stored on whatever medium that backs the node
such as disk, SSD, or network storage, depending on your environment. If you set
the `emptyDir.medium` field to `"Memory"`, Kubernetes mounts a tmpfs (RAM-backed
filesystem) for you instead. While tmpfs is very fast, be aware that unlike
disks, tmpfs is cleared on node reboot and any files you write count against
your container's memory limit.
filesystem) for you instead. While tmpfs is very fast be aware that, unlike
disks, files you write count against the memory limit of the container that wrote them.
A size limit can be specified for the default medium, which limits the capacity

2
content/en/docs/concepts/storage/windows-storage.md
View File

@ -41,7 +41,7 @@ As a result, the following storage functionality is not supported on Windows nod
* Block device mapping
* Memory as the storage medium (for example, `emptyDir.medium` set to `Memory`)
* File system features like uid/gid; per-user Linux filesystem permissions
* Setting [secret permissions with DefaultMode](/docs/concepts/configuration/secret/#secret-files-permissions) (due to UID/GID dependency)
* Setting [secret permissions with DefaultMode](/docs/tasks/inject-data-application/distribute-credentials-secure/#set-posix-permissions-for-secret-keys) (due to UID/GID dependency)
* NFS based storage/volume support
* Expanding the mounted volume (resizefs)

99
content/en/docs/concepts/workloads/controllers/deployment.md
View File

@ -1197,6 +1197,105 @@ rolling update starts, such that the total number of old and new Pods does not e
Pods. Once old Pods have been killed, the new ReplicaSet can be scaled up further, ensuring that the
total number of Pods running at any time during the update is at most 130% of desired Pods.
Here are some Rolling Update Deployment examples that use the `maxUnavailable` and `maxSurge`:
{{< tabs name="tab_with_md" >}}
{{% tab name="Max Unavailable" %}}
```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
labels:
app: nginx
spec:
replicas: 3
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.14.2
ports:
- containerPort: 80
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 1
```
{{% /tab %}}
{{% tab name="Max Surge" %}}
```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
labels:
app: nginx
spec:
replicas: 3
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.14.2
ports:
- containerPort: 80
strategy:
type: RollingUpdate
rollingUpdate:
maxSurge: 1
```
{{% /tab %}}
{{% tab name="Hybrid" %}}
```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
labels:
app: nginx
spec:
replicas: 3
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.14.2
ports:
- containerPort: 80
strategy:
type: RollingUpdate
rollingUpdate:
maxSurge: 1
maxUnavailable: 1
```
{{% /tab %}}
{{< /tabs >}}
### Progress Deadline Seconds
`.spec.progressDeadlineSeconds` is an optional field that specifies the number of seconds you want

5
content/en/docs/contribute/generate-ref-docs/prerequisites-ref-docs.md
View File

@ -5,9 +5,9 @@
- You need to have these tools installed:
- [Python](https://www.python.org/downloads/) v3.7.x
- [Python](https://www.python.org/downloads/) v3.7.x+
- [Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git)
- [Golang](https://golang.org/doc/install) version 1.13+
- [Golang](https://go.dev/dl/) version 1.13+
- [Pip](https://pypi.org/project/pip/) used to install PyYAML
- [PyYAML](https://pyyaml.org/) v5.1.2
- [make](https://www.gnu.org/software/make/)
@ -19,4 +19,3 @@
- You need to know how to create a pull request to a GitHub repository.
This involves creating your own fork of the repository. For more
information, see [Work from a local clone](/docs/contribute/new-content/open-a-pr/#fork-the-repo).

78
content/en/docs/contribute/participate/issue-wrangler.md Normal file
View File

@ -0,0 +1,78 @@
---
title: Issue Wranglers
content_type: concept
weight: 20
---
<!-- overview -->
Alongside the [PR Wrangler](/docs/contribute/participate/pr-wranglers),formal approvers, and reviewers, members of SIG Docs take week long shifts [triaging and categorising issues](/docs/contribute/review/for-approvers.md/#triage-and-categorize-issues) for the repository.
<!-- body -->
## Duties
Each day in a week-long shift the Issue Wrangler will be responsible for:
- Triaging and tagging incoming issues daily. See [Triage and categorize issues](https://github.com/kubernetes/website/blob/main/content/en/docs/contribute/review/for-approvers.md/#triage-and-categorize-issues) for guidelines on how SIG Docs uses metadata.
- Keeping an eye on stale & rotten issues within the kubernetes/website repository.
- Maintenance of the [Issues board](https://github.com/orgs/kubernetes/projects/72/views/1).
### Requirements
- Must be an active member of the Kubernetes organization.
- A minimum of 15 [non-trivial](https://www.kubernetes.dev/docs/guide/pull-requests/#trivial-edits) contributions to Kubernetes (of which a certain amount should be directed towards kubernetes/website).
- Performing the role in an informal capacity already
### Helpful [Prow commands](https://prow.k8s.io/command-help) for wranglers
```
# reopen an issue
/reopen
# transfer issues that don't fit in k/website to another repository
/transfer[-issue]
# change the state of rotten issues
/remove-lifecycle rotten
# change the state of stale issues
/remove-lifecycle stale
# assign sig to an issue
/sig <sig_name>
# add specific area
/area <area_name>
# for beginner friendly issues
/good-first-issue
# issues that needs help
/help wanted
# tagging issue as support specific
/kind support
# to accept triaging for an issue
/triage accepted
# closing an issue we won't be working on and haven't fixed yet
/close not-planned
```
### When to close Issues
For an open source project to succeed, good issue management is crucial. But it is also critical to resolve issues in order to maintain the repository and communicate clearly with contributors and users.
Close issues when:
- A similar issue is reported more than once.You will first need to tag it as /triage duplicate; link it to the main issue & then close it. It is also advisable to direct the users to the original issue.
- It is very difficult to understand and address the issue presented by the author with the information provided.
However, encourage the user to provide more details or reopen the issue if they can reproduce it later.
- The same functionality is implemented elsewhere. One can close this issue and direct user to the appropriate place.
- The reported issue is not currently planned or aligned with the project's goals.
- If the issue appears to be spam and is clearly unrelated.
- If the issue is related to an external limitation or dependency and is beyond the control of the project.
To close an issue, leave a `/close` comment on the issue.

2
content/en/docs/reference/access-authn-authz/admission-controllers.md
View File

@ -24,7 +24,7 @@ Kubernetes API server prior to persistence of the object, but after the request
is authenticated and authorized.
Admission controllers may be _validating_, _mutating_, or both. Mutating
controllers may modify related objects to the requests they admit; validating controllers may not.
controllers may modify objects related to the requests they admit; validating controllers may not.
Admission controllers limit requests to create, delete, modify objects. Admission
controllers can also block custom verbs, such as a request connect to a Pod via

2
content/en/docs/reference/access-authn-authz/certificate-signing-requests.md
View File

@ -488,7 +488,7 @@ O is the group that this user will belong to. You can refer to
```shell
openssl genrsa -out myuser.key 2048
openssl req -new -key myuser.key -out myuser.csr
openssl req -new -key myuser.key -out myuser.csr -subj "/CN=myuser"
```
### Create a CertificateSigningRequest {#create-certificatessigningrequest}

152
content/en/docs/reference/access-authn-authz/kubelet-tls-bootstrapping.md
View File

@ -11,31 +11,35 @@ weight: 120
<!-- overview -->
In a Kubernetes cluster, the components on the worker nodes - kubelet and kube-proxy - need to communicate with Kubernetes control plane components, specifically kube-apiserver.
In order to ensure that communication is kept private, not interfered with, and ensure that each component of the cluster is talking to another trusted component, we strongly
In a Kubernetes cluster, the components on the worker nodes - kubelet and kube-proxy - need
to communicate with Kubernetes control plane components, specifically kube-apiserver.
In order to ensure that communication is kept private, not interfered with, and ensure that
each component of the cluster is talking to another trusted component, we strongly
recommend using client TLS certificates on nodes.
The normal process of bootstrapping these components, especially worker nodes that need certificates so they can communicate safely with kube-apiserver,
can be a challenging process as it is often outside of the scope of Kubernetes and requires significant additional work.
The normal process of bootstrapping these components, especially worker nodes that need certificates
so they can communicate safely with kube-apiserver, can be a challenging process as it is often outside
of the scope of Kubernetes and requires significant additional work.
This in turn, can make it challenging to initialize or scale a cluster.
In order to simplify the process, beginning in version 1.4, Kubernetes introduced a certificate request and signing API. The proposal can be
found [here](https://github.com/kubernetes/kubernetes/pull/20439).
In order to simplify the process, beginning in version 1.4, Kubernetes introduced a certificate request
and signing API. The proposal can be found [here](https://github.com/kubernetes/kubernetes/pull/20439).
This document describes the process of node initialization, how to set up TLS client certificate bootstrapping for
kubelets, and how it works.
<!-- body -->
## Initialization Process
## Initialization process
When a worker node starts up, the kubelet does the following:
1. Look for its `kubeconfig` file
2. Retrieve the URL of the API server and credentials, normally a TLS key and signed certificate from the `kubeconfig` file
3. Attempt to communicate with the API server using the credentials.
1. Retrieve the URL of the API server and credentials, normally a TLS key and signed certificate from the `kubeconfig` file
1. Attempt to communicate with the API server using the credentials.
Assuming that the kube-apiserver successfully validates the kubelet's credentials, it will treat the kubelet as a valid node, and begin to assign pods to it.
Assuming that the kube-apiserver successfully validates the kubelet's credentials,
it will treat the kubelet as a valid node, and begin to assign pods to it.
Note that the above process depends upon:
@ -45,35 +49,36 @@ Note that the above process depends upon:
All of the following are responsibilities of whoever sets up and manages the cluster:
1. Creating the CA key and certificate
2. Distributing the CA certificate to the control plane nodes, where kube-apiserver is running
3. Creating a key and certificate for each kubelet; strongly recommended to have a unique one, with a unique CN, for each kubelet
4. Signing the kubelet certificate using the CA key
5. Distributing the kubelet key and signed certificate to the specific node on which the kubelet is running
1. Distributing the CA certificate to the control plane nodes, where kube-apiserver is running
1. Creating a key and certificate for each kubelet; strongly recommended to have a unique one, with a unique CN, for each kubelet
1. Signing the kubelet certificate using the CA key
1. Distributing the kubelet key and signed certificate to the specific node on which the kubelet is running
The TLS Bootstrapping described in this document is intended to simplify, and partially or even completely automate, steps 3 onwards, as these are the most common when initializing or scaling
The TLS Bootstrapping described in this document is intended to simplify, and partially or even
completely automate, steps 3 onwards, as these are the most common when initializing or scaling
a cluster.
### Bootstrap Initialization
### Bootstrap initialization
In the bootstrap initialization process, the following occurs:
1. kubelet begins
2. kubelet sees that it does _not_ have a `kubeconfig` file
3. kubelet searches for and finds a `bootstrap-kubeconfig` file
4. kubelet reads its bootstrap file, retrieving the URL of the API server and a limited usage "token"
5. kubelet connects to the API server, authenticates using the token
6. kubelet now has limited credentials to create and retrieve a certificate signing request (CSR)
7. kubelet creates a CSR for itself with the signerName set to `kubernetes.io/kube-apiserver-client-kubelet`
8. CSR is approved in one of two ways:
1. kubelet sees that it does _not_ have a `kubeconfig` file
1. kubelet searches for and finds a `bootstrap-kubeconfig` file
1. kubelet reads its bootstrap file, retrieving the URL of the API server and a limited usage "token"
1. kubelet connects to the API server, authenticates using the token
1. kubelet now has limited credentials to create and retrieve a certificate signing request (CSR)
1. kubelet creates a CSR for itself with the signerName set to `kubernetes.io/kube-apiserver-client-kubelet`
1. CSR is approved in one of two ways:
* If configured, kube-controller-manager automatically approves the CSR
* If configured, an outside process, possibly a person, approves the CSR using the Kubernetes API or via `kubectl`
9. Certificate is created for the kubelet
10. Certificate is issued to the kubelet
11. kubelet retrieves the certificate
12. kubelet creates a proper `kubeconfig` with the key and signed certificate
13. kubelet begins normal operation
14. Optional: if configured, kubelet automatically requests renewal of the certificate when it is close to expiry
15. The renewed certificate is approved and issued, either automatically or manually, depending on configuration.
1. Certificate is created for the kubelet
1. Certificate is issued to the kubelet
1. kubelet retrieves the certificate
1. kubelet creates a proper `kubeconfig` with the key and signed certificate
1. kubelet begins normal operation
1. Optional: if configured, kubelet automatically requests renewal of the certificate when it is close to expiry
1. The renewed certificate is approved and issued, either automatically or manually, depending on configuration.
The rest of this document describes the necessary steps to configure TLS Bootstrapping, and its limitations.
@ -90,13 +95,16 @@ In addition, you need your Kubernetes Certificate Authority (CA).
## Certificate Authority
As without bootstrapping, you will need a Certificate Authority (CA) key and certificate. As without bootstrapping, these will be used
to sign the kubelet certificate. As before, it is your responsibility to distribute them to control plane nodes.
As without bootstrapping, you will need a Certificate Authority (CA) key and certificate.
As without bootstrapping, these will be used to sign the kubelet certificate. As before,
it is your responsibility to distribute them to control plane nodes.
For the purposes of this document, we will assume these have been distributed to control plane nodes at `/var/lib/kubernetes/ca.pem` (certificate) and `/var/lib/kubernetes/ca-key.pem` (key).
For the purposes of this document, we will assume these have been distributed to control
plane nodes at `/var/lib/kubernetes/ca.pem` (certificate) and `/var/lib/kubernetes/ca-key.pem` (key).
We will refer to these as "Kubernetes CA certificate and key".
All Kubernetes components that use these certificates - kubelet, kube-apiserver, kube-controller-manager - assume the key and certificate to be PEM-encoded.
All Kubernetes components that use these certificates - kubelet, kube-apiserver,
kube-controller-manager - assume the key and certificate to be PEM-encoded.
## kube-apiserver configuration
@ -116,24 +124,27 @@ containing the signing certificate, for example
### Initial bootstrap authentication
In order for the bootstrapping kubelet to connect to kube-apiserver and request a certificate, it must first authenticate to the server.
You can use any [authenticator](/docs/reference/access-authn-authz/authentication/) that can authenticate the kubelet.
In order for the bootstrapping kubelet to connect to kube-apiserver and request a certificate,
it must first authenticate to the server. You can use any
[authenticator](/docs/reference/access-authn-authz/authentication/) that can authenticate the kubelet.
While any authentication strategy can be used for the kubelet's initial
bootstrap credentials, the following two authenticators are recommended for ease
of provisioning.
1. [Bootstrap Tokens](#bootstrap-tokens)
2. [Token authentication file](#token-authentication-file)
1. [Token authentication file](#token-authentication-file)
Using bootstrap tokens is a simpler and more easily managed method to authenticate kubelets, and does not require any additional flags when starting kube-apiserver.
Using bootstrap tokens is a simpler and more easily managed method to authenticate kubelets,
and does not require any additional flags when starting kube-apiserver.
Whichever method you choose, the requirement is that the kubelet be able to authenticate as a user with the rights to:
1. create and retrieve CSRs
2. be automatically approved to request node client certificates, if automatic approval is enabled.
1. be automatically approved to request node client certificates, if automatic approval is enabled.
A kubelet authenticating using bootstrap tokens is authenticated as a user in the group `system:bootstrappers`, which is the standard method to use.
A kubelet authenticating using bootstrap tokens is authenticated as a user in the group
`system:bootstrappers`, which is the standard method to use.
As this feature matures, you
should ensure tokens are bound to a Role Based Access Control (RBAC) policy
@ -144,17 +155,20 @@ particular bootstrap group's access when you are done provisioning the nodes.
#### Bootstrap tokens
Bootstrap tokens are described in detail [here](/docs/reference/access-authn-authz/bootstrap-tokens/). These are tokens that are stored as secrets in the Kubernetes cluster,
and then issued to the individual kubelet. You can use a single token for an entire cluster, or issue one per worker node.
Bootstrap tokens are described in detail [here](/docs/reference/access-authn-authz/bootstrap-tokens/).
These are tokens that are stored as secrets in the Kubernetes cluster, and then issued to the individual kubelet.
You can use a single token for an entire cluster, or issue one per worker node.
The process is two-fold:
1. Create a Kubernetes secret with the token ID, secret and scope(s).
2. Issue the token to the kubelet
1. Issue the token to the kubelet
From the kubelet's perspective, one token is like another and has no special meaning.
From the kube-apiserver's perspective, however, the bootstrap token is special. Due to its `type`, `namespace` and `name`, kube-apiserver recognizes it as a special token,
and grants anyone authenticating with that token special bootstrap rights, notably treating them as a member of the `system:bootstrappers` group. This fulfills a basic requirement
From the kube-apiserver's perspective, however, the bootstrap token is special.
Due to its `type`, `namespace` and `name`, kube-apiserver recognizes it as a special token,
and grants anyone authenticating with that token special bootstrap rights, notably treating
them as a member of the `system:bootstrappers` group. This fulfills a basic requirement
for TLS bootstrapping.
The details for creating the secret are available [here](/docs/reference/access-authn-authz/bootstrap-tokens/).
@ -198,7 +212,8 @@ certificate signing request (CSR) as well as retrieve it when done.
Fortunately, Kubernetes ships with a `ClusterRole` with precisely these (and
only these) permissions, `system:node-bootstrapper`.
To do this, you only need to create a `ClusterRoleBinding` that binds the `system:bootstrappers` group to the cluster role `system:node-bootstrapper`.
To do this, you only need to create a `ClusterRoleBinding` that binds the `system:bootstrappers`
group to the cluster role `system:node-bootstrapper`.
```yaml
# enable bootstrapping nodes to create CSR
@ -237,9 +252,10 @@ In order for the controller-manager to sign certificates, it needs the following
As described earlier, you need to create a Kubernetes CA key and certificate, and distribute it to the control plane nodes.
These will be used by the controller-manager to sign the kubelet certificates.
Since these signed certificates will, in turn, be used by the kubelet to authenticate as a regular kubelet to kube-apiserver, it is important that the CA
provided to the controller-manager at this stage also be trusted by kube-apiserver for authentication. This is provided to kube-apiserver
with the flag `--client-ca-file=FILENAME` (for example, `--client-ca-file=/var/lib/kubernetes/ca.pem`), as described in the kube-apiserver configuration section.
Since these signed certificates will, in turn, be used by the kubelet to authenticate as a regular kubelet
to kube-apiserver, it is important that the CA provided to the controller-manager at this stage also be
trusted by kube-apiserver for authentication. This is provided to kube-apiserver with the flag `--client-ca-file=FILENAME`
(for example, `--client-ca-file=/var/lib/kubernetes/ca.pem`), as described in the kube-apiserver configuration section.
To provide the Kubernetes CA key and certificate to kube-controller-manager, use the following flags:
@ -266,10 +282,14 @@ RBAC permissions to the correct group.
There are two distinct sets of permissions:
* `nodeclient`: If a node is creating a new certificate for a node, then it does not have a certificate yet. It is authenticating using one of the tokens listed above, and thus is part of the group `system:bootstrappers`.
* `selfnodeclient`: If a node is renewing its certificate, then it already has a certificate (by definition), which it uses continuously to authenticate as part of the group `system:nodes`.
* `nodeclient`: If a node is creating a new certificate for a node, then it does not have a certificate yet.
It is authenticating using one of the tokens listed above, and thus is part of the group `system:bootstrappers`.
* `selfnodeclient`: If a node is renewing its certificate, then it already has a certificate (by definition),
which it uses continuously to authenticate as part of the group `system:nodes`.
To enable the kubelet to request and receive a new certificate, create a `ClusterRoleBinding` that binds the group in which the bootstrapping node is a member `system:bootstrappers` to the `ClusterRole` that grants it permission, `system:certificates.k8s.io:certificatesigningrequests:nodeclient`:
To enable the kubelet to request and receive a new certificate, create a `ClusterRoleBinding` that binds
the group in which the bootstrapping node is a member `system:bootstrappers` to the `ClusterRole` that
grants it permission, `system:certificates.k8s.io:certificatesigningrequests:nodeclient`:
```yaml
# Approve all CSRs for the group "system:bootstrappers"
@ -287,7 +307,8 @@ roleRef:
apiGroup: rbac.authorization.k8s.io
```
To enable the kubelet to renew its own client certificate, create a `ClusterRoleBinding` that binds the group in which the fully functioning node is a member `system:nodes` to the `ClusterRole` that
To enable the kubelet to renew its own client certificate, create a `ClusterRoleBinding` that binds
the group in which the fully functioning node is a member `system:nodes` to the `ClusterRole` that
grants it permission, `system:certificates.k8s.io:certificatesigningrequests:selfnodeclient`:
```yaml
@ -316,10 +337,10 @@ built-in approver doesn't explicitly deny CSRs. It only ignores unauthorized
requests. The controller also prunes expired certificates as part of garbage
collection.
## kubelet configuration
Finally, with the control plane nodes properly set up and all of the necessary authentication and authorization in place, we can configure the kubelet.
Finally, with the control plane nodes properly set up and all of the necessary
authentication and authorization in place, we can configure the kubelet.
The kubelet requires the following configuration to bootstrap:
@ -385,7 +406,7 @@ referencing the generated key and obtained certificate is written to the path
specified by `--kubeconfig`. The certificate and key file will be placed in the
directory specified by `--cert-dir`.
### Client and Serving Certificates
### Client and serving certificates
All of the above relate to kubelet _client_ certificates, specifically, the certificates a kubelet
uses to authenticate to kube-apiserver.
@ -402,7 +423,7 @@ be used as serving certificates, or `server auth`.
However, you _can_ enable its server certificate, at least partially, via certificate rotation.
### Certificate Rotation
### Certificate rotation
Kubernetes v1.8 and higher kubelet implements features for enabling
rotation of its client and/or serving certificates. Note, rotation of serving
@ -420,7 +441,7 @@ or pass the following command line argument to the kubelet (deprecated):
Enabling `RotateKubeletServerCertificate` causes the kubelet **both** to request a serving
certificate after bootstrapping its client credentials **and** to rotate that
certificate. To enable this behavior, use the field `serverTLSBootstrap` of
certificate. To enable this behavior, use the field `serverTLSBootstrap` of
the [kubelet configuration file](/docs/tasks/administer-cluster/kubelet-config-file/)
or pass the following command line argument to the kubelet (deprecated):
@ -430,8 +451,8 @@ or pass the following command line argument to the kubelet (deprecated):
{{< note >}}
The CSR approving controllers implemented in core Kubernetes do not
approve node _serving_ certificates for [security
reasons](https://github.com/kubernetes/community/pull/1982). To use
approve node _serving_ certificates for
[security reasons](https://github.com/kubernetes/community/pull/1982). To use
`RotateKubeletServerCertificate` operators need to run a custom approving
controller, or manually approve the serving certificate requests.
@ -439,9 +460,9 @@ A deployment-specific approval process for kubelet serving certificates should t
1. are requested by nodes (ensure the `spec.username` field is of the form
`system:node:<nodeName>` and `spec.groups` contains `system:nodes`)
2. request usages for a serving certificate (ensure `spec.usages` contains `server auth`,
1. request usages for a serving certificate (ensure `spec.usages` contains `server auth`,
optionally contains `digital signature` and `key encipherment`, and contains no other usages)
3. only have IP and DNS subjectAltNames that belong to the requesting node,
1. only have IP and DNS subjectAltNames that belong to the requesting node,
and have no URI and Email subjectAltNames (parse the x509 Certificate Signing Request
in `spec.request` to verify `subjectAltNames`)
@ -457,8 +478,11 @@ Like the kubelet, these other components also require a method of authenticating
You have several options for generating these credentials:
* The old way: Create and distribute certificates the same way you did for kubelet before TLS bootstrapping
* DaemonSet: Since the kubelet itself is loaded on each node, and is sufficient to start base services, you can run kube-proxy and other node-specific services not as a standalone process, but rather as a daemonset in the `kube-system` namespace. Since it will be in-cluster, you can give it a proper service account with appropriate permissions to perform its activities. This may be the simplest way to configure such services.
* DaemonSet: Since the kubelet itself is loaded on each node, and is sufficient to start base services,
you can run kube-proxy and other node-specific services not as a standalone process, but rather as a
daemonset in the `kube-system` namespace. Since it will be in-cluster, you can give it a proper service
account with appropriate permissions to perform its activities. This may be the simplest way to configure
such services.
## kubectl approval

7
content/en/docs/reference/command-line-tools-reference/feature-gates.md
View File

@ -185,7 +185,7 @@ For a reference to old feature gates that are removed, please refer to
| `SELinuxMountReadWriteOncePod` | `false` | Alpha | 1.25 | 1.26 |
| `SELinuxMountReadWriteOncePod` | `false` | Beta | 1.27 | 1.27 |
| `SELinuxMountReadWriteOncePod` | `true` | Beta | 1.28 | |
| `SchedulerQueueingHints` | `false` | Alpha | 1.28 | |
| `SchedulerQueueingHints` | `true` | Beta | 1.28 | |
| `SecurityContextDeny` | `false` | Alpha | 1.27 | |
| `SidecarContainers` | `false` | Alpha | 1.28 | |
| `SizeMemoryBackedVolumes` | `false` | Alpha | 1.20 | 1.21 |
@ -688,8 +688,11 @@ Each feature gate is designed for enabling/disabling a specific feature:
- `SELinuxMountReadWriteOncePod`: Speeds up container startup by allowing kubelet to mount volumes
for a Pod directly with the correct SELinux label instead of changing each file on the volumes
recursively. The initial implementation focused on ReadWriteOncePod volumes.
- `SchedulerQueueingHints`: Enables the scheduler's _queueing hints_ enhancement,
- `SchedulerQueueingHints`: Enables [the scheduler's _queueing hints_ enhancement](https://github.com/kubernetes/enhancements/blob/master/keps/sig-scheduling/4247-queueinghint/README.md),
which benefits to reduce the useless requeueing.
The scheduler retries scheduling pods if something changes in the cluster that could make the pod scheduled.
Queueing hints are internal signals that allow the scheduler to filter the changes in the cluster
that are relevant to the unscheduled pod, based on previous scheduling attempts.
- `SeccompDefault`: Enables the use of `RuntimeDefault` as the default seccomp profile
for all workloads.
The seccomp profile is specified in the `securityContext` of a Pod and/or a Container.

1
content/en/docs/reference/config-api/apiserver-admission.v1.md
View File

@ -11,7 +11,6 @@ auto_generated: true
- [AdmissionReview](#admission-k8s-io-v1-AdmissionReview)
## `AdmissionReview` {#admission-k8s-io-v1-AdmissionReview}

1
content/en/docs/reference/config-api/apiserver-audit.v1.md
View File

@ -14,7 +14,6 @@ auto_generated: true
- [Policy](#audit-k8s-io-v1-Policy)
- [PolicyList](#audit-k8s-io-v1-PolicyList)
## `Event` {#audit-k8s-io-v1-Event}

1
content/en/docs/reference/config-api/apiserver-config.v1.md
View File

@ -12,7 +12,6 @@ auto_generated: true
- [AdmissionConfiguration](#apiserver-config-k8s-io-v1-AdmissionConfiguration)
## `AdmissionConfiguration` {#apiserver-config-k8s-io-v1-AdmissionConfiguration}

84
content/en/docs/reference/config-api/apiserver-config.v1alpha1.md
View File

@ -15,6 +15,47 @@ auto_generated: true
- [TracingConfiguration](#apiserver-k8s-io-v1alpha1-TracingConfiguration)
## `TracingConfiguration` {#TracingConfiguration}
**Appears in:**
- [KubeletConfiguration](#kubelet-config-k8s-io-v1beta1-KubeletConfiguration)
- [TracingConfiguration](#apiserver-k8s-io-v1alpha1-TracingConfiguration)
<p>TracingConfiguration provides versioned configuration for OpenTelemetry tracing clients.</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>endpoint</code><br/>
<code>string</code>
</td>
<td>
<p>Endpoint of the collector this component will report traces to.
The connection is insecure, and does not currently support TLS.
Recommended is unset, and endpoint is the otlp grpc default, localhost:4317.</p>
</td>
</tr>
<tr><td><code>samplingRatePerMillion</code><br/>
<code>int32</code>
</td>
<td>
<p>SamplingRatePerMillion is the number of samples to collect per million spans.
Recommended is unset. If unset, sampler respects its parent span's sampling
rate, but otherwise never samples.</p>
</td>
</tr>
</tbody>
</table>
## `AdmissionConfiguration` {#apiserver-k8s-io-v1alpha1-AdmissionConfiguration}
@ -360,45 +401,4 @@ This does not use a unix:// prefix. (Eg: /etc/srv/kubernetes/konnectivity-server
</tr>
</tbody>
</table>
## `TracingConfiguration` {#TracingConfiguration}
**Appears in:**
- [KubeletConfiguration](#kubelet-config-k8s-io-v1beta1-KubeletConfiguration)
- [TracingConfiguration](#apiserver-k8s-io-v1alpha1-TracingConfiguration)
<p>TracingConfiguration provides versioned configuration for OpenTelemetry tracing clients.</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>endpoint</code><br/>
<code>string</code>
</td>
<td>
<p>Endpoint of the collector this component will report traces to.
The connection is insecure, and does not currently support TLS.
Recommended is unset, and endpoint is the otlp grpc default, localhost:4317.</p>
</td>
</tr>
<tr><td><code>samplingRatePerMillion</code><br/>
<code>int32</code>
</td>
<td>
<p>SamplingRatePerMillion is the number of samples to collect per million spans.
Recommended is unset. If unset, sampler respects its parent span's sampling
rate, but otherwise never samples.</p>
</td>
</tr>
</tbody>
</table>

88
content/en/docs/reference/config-api/apiserver-config.v1beta1.md
View File

@ -14,6 +14,49 @@ auto_generated: true
- [TracingConfiguration](#apiserver-k8s-io-v1beta1-TracingConfiguration)
## `TracingConfiguration` {#TracingConfiguration}
**Appears in:**
- [KubeletConfiguration](#kubelet-config-k8s-io-v1beta1-KubeletConfiguration)
- [TracingConfiguration](#apiserver-k8s-io-v1alpha1-TracingConfiguration)
- [TracingConfiguration](#apiserver-k8s-io-v1beta1-TracingConfiguration)
<p>TracingConfiguration provides versioned configuration for OpenTelemetry tracing clients.</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>endpoint</code><br/>
<code>string</code>
</td>
<td>
<p>Endpoint of the collector this component will report traces to.
The connection is insecure, and does not currently support TLS.
Recommended is unset, and endpoint is the otlp grpc default, localhost:4317.</p>
</td>
</tr>
<tr><td><code>samplingRatePerMillion</code><br/>
<code>int32</code>
</td>
<td>
<p>SamplingRatePerMillion is the number of samples to collect per million spans.
Recommended is unset. If unset, sampler respects its parent span's sampling
rate, but otherwise never samples.</p>
</td>
</tr>
</tbody>
</table>
## `EgressSelectorConfiguration` {#apiserver-k8s-io-v1beta1-EgressSelectorConfiguration}
@ -291,47 +334,4 @@ This does not use a unix:// prefix. (Eg: /etc/srv/kubernetes/konnectivity-server
</tr>
</tbody>
</table>
## `TracingConfiguration` {#TracingConfiguration}
**Appears in:**
- [KubeletConfiguration](#kubelet-config-k8s-io-v1beta1-KubeletConfiguration)
- [TracingConfiguration](#apiserver-k8s-io-v1alpha1-TracingConfiguration)
- [TracingConfiguration](#apiserver-k8s-io-v1beta1-TracingConfiguration)
<p>TracingConfiguration provides versioned configuration for OpenTelemetry tracing clients.</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>endpoint</code><br/>
<code>string</code>
</td>
<td>
<p>Endpoint of the collector this component will report traces to.
The connection is insecure, and does not currently support TLS.
Recommended is unset, and endpoint is the otlp grpc default, localhost:4317.</p>
</td>
</tr>
<tr><td><code>samplingRatePerMillion</code><br/>
<code>int32</code>
</td>
<td>
<p>SamplingRatePerMillion is the number of samples to collect per million spans.
Recommended is unset. If unset, sampler respects its parent span's sampling
rate, but otherwise never samples.</p>
</td>
</tr>
</tbody>
</table>

3
content/en/docs/reference/config-api/apiserver-encryption.v1.md
View File

@ -12,7 +12,6 @@ auto_generated: true
- [EncryptionConfiguration](#apiserver-config-k8s-io-v1-EncryptionConfiguration)
## `EncryptionConfiguration` {#apiserver-config-k8s-io-v1-EncryptionConfiguration}
@ -20,7 +19,7 @@ auto_generated: true
<p>EncryptionConfiguration stores the complete configuration for encryption providers.
It also allows the use of wildcards to specify the resources that should be encrypted.
Use '&ast;.&lt;group&gt;' to encrypt all resources within a group or '&ast;.&ast;' to encrypt all resources.
Use '&ast;&lt;group&gt;o encrypt all resources within a group or '&ast;.&ast;' to encrypt all resources.
'&ast;.' can be used to encrypt all resource in the core group. '&ast;.&ast;' will encrypt all
resources, even custom resources that are added after API server start.
Use of wildcards that overlap within the same resource list or across multiple

1
content/en/docs/reference/config-api/apiserver-eventratelimit.v1alpha1.md
View File

@ -11,7 +11,6 @@ auto_generated: true
- [Configuration](#eventratelimit-admission-k8s-io-v1alpha1-Configuration)
## `Configuration` {#eventratelimit-admission-k8s-io-v1alpha1-Configuration}

1
content/en/docs/reference/config-api/apiserver-webhookadmission.v1.md
View File

@ -12,7 +12,6 @@ auto_generated: true
- [WebhookAdmission](#apiserver-config-k8s-io-v1-WebhookAdmission)
## `WebhookAdmission` {#apiserver-config-k8s-io-v1-WebhookAdmission}

1
content/en/docs/reference/config-api/client-authentication.v1.md
View File

@ -11,7 +11,6 @@ auto_generated: true
- [ExecCredential](#client-authentication-k8s-io-v1-ExecCredential)
## `ExecCredential` {#client-authentication-k8s-io-v1-ExecCredential}

1
content/en/docs/reference/config-api/client-authentication.v1beta1.md
View File

@ -11,7 +11,6 @@ auto_generated: true
- [ExecCredential](#client-authentication-k8s-io-v1beta1-ExecCredential)
## `ExecCredential` {#client-authentication-k8s-io-v1beta1-ExecCredential}

1
content/en/docs/reference/config-api/imagepolicy.v1alpha1.md
View File

@ -11,7 +11,6 @@ auto_generated: true
- [ImageReview](#imagepolicy-k8s-io-v1alpha1-ImageReview)
## `ImageReview` {#imagepolicy-k8s-io-v1alpha1-ImageReview}

964
content/en/docs/reference/config-api/kube-controller-manager-config.v1alpha1.md
View File

@ -9,11 +9,491 @@ auto_generated: true
## Resource Types
- [KubeControllerManagerConfiguration](#kubecontrollermanager-config-k8s-io-v1alpha1-KubeControllerManagerConfiguration)
- [CloudControllerManagerConfiguration](#cloudcontrollermanager-config-k8s-io-v1alpha1-CloudControllerManagerConfiguration)
- [LeaderMigrationConfiguration](#controllermanager-config-k8s-io-v1alpha1-LeaderMigrationConfiguration)
- [KubeControllerManagerConfiguration](#kubecontrollermanager-config-k8s-io-v1alpha1-KubeControllerManagerConfiguration)
## `NodeControllerConfiguration` {#NodeControllerConfiguration}
**Appears in:**
- [CloudControllerManagerConfiguration](#cloudcontrollermanager-config-k8s-io-v1alpha1-CloudControllerManagerConfiguration)
<p>NodeControllerConfiguration contains elements describing NodeController.</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>ConcurrentNodeSyncs</code> <B>[Required]</B><br/>
<code>int32</code>
</td>
<td>
<p>ConcurrentNodeSyncs is the number of workers
concurrently synchronizing nodes</p>
</td>
</tr>
</tbody>
</table>
## `ServiceControllerConfiguration` {#ServiceControllerConfiguration}
**Appears in:**
- [CloudControllerManagerConfiguration](#cloudcontrollermanager-config-k8s-io-v1alpha1-CloudControllerManagerConfiguration)
- [KubeControllerManagerConfiguration](#kubecontrollermanager-config-k8s-io-v1alpha1-KubeControllerManagerConfiguration)
<p>ServiceControllerConfiguration contains elements describing ServiceController.</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>ConcurrentServiceSyncs</code> <B>[Required]</B><br/>
<code>int32</code>
</td>
<td>
<p>concurrentServiceSyncs is the number of services that are
allowed to sync concurrently. Larger number = more responsive service
management, but more CPU (and network) load.</p>
</td>
</tr>
</tbody>
</table>
## `CloudControllerManagerConfiguration` {#cloudcontrollermanager-config-k8s-io-v1alpha1-CloudControllerManagerConfiguration}
<p>CloudControllerManagerConfiguration contains elements describing cloud-controller manager.</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>apiVersion</code><br/>string</td><td><code>cloudcontrollermanager.config.k8s.io/v1alpha1</code></td></tr>
<tr><td><code>kind</code><br/>string</td><td><code>CloudControllerManagerConfiguration</code></td></tr>
<tr><td><code>Generic</code> <B>[Required]</B><br/>
<a href="#controllermanager-config-k8s-io-v1alpha1-GenericControllerManagerConfiguration"><code>GenericControllerManagerConfiguration</code></a>
</td>
<td>
<p>Generic holds configuration for a generic controller-manager</p>
</td>
</tr>
<tr><td><code>KubeCloudShared</code> <B>[Required]</B><br/>
<a href="#cloudcontrollermanager-config-k8s-io-v1alpha1-KubeCloudSharedConfiguration"><code>KubeCloudSharedConfiguration</code></a>
</td>
<td>
<p>KubeCloudSharedConfiguration holds configuration for shared related features
both in cloud controller manager and kube-controller manager.</p>
</td>
</tr>
<tr><td><code>NodeController</code> <B>[Required]</B><br/>
<a href="#NodeControllerConfiguration"><code>NodeControllerConfiguration</code></a>
</td>
<td>
<p>NodeController holds configuration for node controller
related features.</p>
</td>
</tr>
<tr><td><code>ServiceController</code> <B>[Required]</B><br/>
<a href="#ServiceControllerConfiguration"><code>ServiceControllerConfiguration</code></a>
</td>
<td>
<p>ServiceControllerConfiguration holds configuration for ServiceController
related features.</p>
</td>
</tr>
<tr><td><code>NodeStatusUpdateFrequency</code> <B>[Required]</B><br/>
<a href="https://pkg.go.dev/k8s.io/apimachinery/pkg/apis/meta/v1#Duration"><code>meta/v1.Duration</code></a>
</td>
<td>
<p>NodeStatusUpdateFrequency is the frequency at which the controller updates nodes' status</p>
</td>
</tr>
<tr><td><code>Webhook</code> <B>[Required]</B><br/>
<a href="#cloudcontrollermanager-config-k8s-io-v1alpha1-WebhookConfiguration"><code>WebhookConfiguration</code></a>
</td>
<td>
<p>Webhook is the configuration for cloud-controller-manager hosted webhooks</p>
</td>
</tr>
</tbody>
</table>
## `CloudProviderConfiguration` {#cloudcontrollermanager-config-k8s-io-v1alpha1-CloudProviderConfiguration}
**Appears in:**
- [KubeCloudSharedConfiguration](#cloudcontrollermanager-config-k8s-io-v1alpha1-KubeCloudSharedConfiguration)
<p>CloudProviderConfiguration contains basically elements about cloud provider.</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>Name</code> <B>[Required]</B><br/>
<code>string</code>
</td>
<td>
<p>Name is the provider for cloud services.</p>
</td>
</tr>
<tr><td><code>CloudConfigFile</code> <B>[Required]</B><br/>
<code>string</code>
</td>
<td>
<p>cloudConfigFile is the path to the cloud provider configuration file.</p>
</td>
</tr>
</tbody>
</table>
## `KubeCloudSharedConfiguration` {#cloudcontrollermanager-config-k8s-io-v1alpha1-KubeCloudSharedConfiguration}
**Appears in:**
- [CloudControllerManagerConfiguration](#cloudcontrollermanager-config-k8s-io-v1alpha1-CloudControllerManagerConfiguration)
- [KubeControllerManagerConfiguration](#kubecontrollermanager-config-k8s-io-v1alpha1-KubeControllerManagerConfiguration)
<p>KubeCloudSharedConfiguration contains elements shared by both kube-controller manager
and cloud-controller manager, but not genericconfig.</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>CloudProvider</code> <B>[Required]</B><br/>
<a href="#cloudcontrollermanager-config-k8s-io-v1alpha1-CloudProviderConfiguration"><code>CloudProviderConfiguration</code></a>
</td>
<td>
<p>CloudProviderConfiguration holds configuration for CloudProvider related features.</p>
</td>
</tr>
<tr><td><code>ExternalCloudVolumePlugin</code> <B>[Required]</B><br/>
<code>string</code>
</td>
<td>
<p>externalCloudVolumePlugin specifies the plugin to use when cloudProvider is &quot;external&quot;.
It is currently used by the in repo cloud providers to handle node and volume control in the KCM.</p>
</td>
</tr>
<tr><td><code>UseServiceAccountCredentials</code> <B>[Required]</B><br/>
<code>bool</code>
</td>
<td>
<p>useServiceAccountCredentials indicates whether controllers should be run with
individual service account credentials.</p>
</td>
</tr>
<tr><td><code>AllowUntaggedCloud</code> <B>[Required]</B><br/>
<code>bool</code>
</td>
<td>
<p>run with untagged cloud instances</p>
</td>
</tr>
<tr><td><code>RouteReconciliationPeriod</code> <B>[Required]</B><br/>
<a href="https://pkg.go.dev/k8s.io/apimachinery/pkg/apis/meta/v1#Duration"><code>meta/v1.Duration</code></a>
</td>
<td>
<p>routeReconciliationPeriod is the period for reconciling routes created for Nodes by cloud provider..</p>
</td>
</tr>
<tr><td><code>NodeMonitorPeriod</code> <B>[Required]</B><br/>
<a href="https://pkg.go.dev/k8s.io/apimachinery/pkg/apis/meta/v1#Duration"><code>meta/v1.Duration</code></a>
</td>
<td>
<p>nodeMonitorPeriod is the period for syncing NodeStatus in NodeController.</p>
</td>
</tr>
<tr><td><code>ClusterName</code> <B>[Required]</B><br/>
<code>string</code>
</td>
<td>
<p>clusterName is the instance prefix for the cluster.</p>
</td>
</tr>
<tr><td><code>ClusterCIDR</code> <B>[Required]</B><br/>
<code>string</code>
</td>
<td>
<p>clusterCIDR is CIDR Range for Pods in cluster.</p>
</td>
</tr>
<tr><td><code>AllocateNodeCIDRs</code> <B>[Required]</B><br/>
<code>bool</code>
</td>
<td>
<p>AllocateNodeCIDRs enables CIDRs for Pods to be allocated and, if
ConfigureCloudRoutes is true, to be set on the cloud provider.</p>
</td>
</tr>
<tr><td><code>CIDRAllocatorType</code> <B>[Required]</B><br/>
<code>string</code>
</td>
<td>
<p>CIDRAllocatorType determines what kind of pod CIDR allocator will be used.</p>
</td>
</tr>
<tr><td><code>ConfigureCloudRoutes</code> <B>[Required]</B><br/>
<code>bool</code>
</td>
<td>
<p>configureCloudRoutes enables CIDRs allocated with allocateNodeCIDRs
to be configured on the cloud provider.</p>
</td>
</tr>
<tr><td><code>NodeSyncPeriod</code> <B>[Required]</B><br/>
<a href="https://pkg.go.dev/k8s.io/apimachinery/pkg/apis/meta/v1#Duration"><code>meta/v1.Duration</code></a>
</td>
<td>
<p>nodeSyncPeriod is the period for syncing nodes from cloudprovider. Longer
periods will result in fewer calls to cloud provider, but may delay addition
of new nodes to cluster.</p>
</td>
</tr>
</tbody>
</table>
## `WebhookConfiguration` {#cloudcontrollermanager-config-k8s-io-v1alpha1-WebhookConfiguration}
**Appears in:**
- [CloudControllerManagerConfiguration](#cloudcontrollermanager-config-k8s-io-v1alpha1-CloudControllerManagerConfiguration)
<p>WebhookConfiguration contains configuration related to
cloud-controller-manager hosted webhooks</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>Webhooks</code> <B>[Required]</B><br/>
<code>[]string</code>
</td>
<td>
<p>Webhooks is the list of webhooks to enable or disable
'*' means &quot;all enabled by default webhooks&quot;
'foo' means &quot;enable 'foo'&quot;
'-foo' means &quot;disable 'foo'&quot;
first item for a particular name wins</p>
</td>
</tr>
</tbody>
</table>
## `LeaderMigrationConfiguration` {#controllermanager-config-k8s-io-v1alpha1-LeaderMigrationConfiguration}
**Appears in:**
- [GenericControllerManagerConfiguration](#controllermanager-config-k8s-io-v1alpha1-GenericControllerManagerConfiguration)
<p>LeaderMigrationConfiguration provides versioned configuration for all migrating leader locks.</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>apiVersion</code><br/>string</td><td><code>controllermanager.config.k8s.io/v1alpha1</code></td></tr>
<tr><td><code>kind</code><br/>string</td><td><code>LeaderMigrationConfiguration</code></td></tr>
<tr><td><code>leaderName</code> <B>[Required]</B><br/>
<code>string</code>
</td>
<td>
<p>LeaderName is the name of the leader election resource that protects the migration
E.g. 1-20-KCM-to-1-21-CCM</p>
</td>
</tr>
<tr><td><code>resourceLock</code> <B>[Required]</B><br/>
<code>string</code>
</td>
<td>
<p>ResourceLock indicates the resource object type that will be used to lock
Should be &quot;leases&quot; or &quot;endpoints&quot;</p>
</td>
</tr>
<tr><td><code>controllerLeaders</code> <B>[Required]</B><br/>
<a href="#controllermanager-config-k8s-io-v1alpha1-ControllerLeaderConfiguration"><code>[]ControllerLeaderConfiguration</code></a>
</td>
<td>
<p>ControllerLeaders contains a list of migrating leader lock configurations</p>
</td>
</tr>
</tbody>
</table>
## `ControllerLeaderConfiguration` {#controllermanager-config-k8s-io-v1alpha1-ControllerLeaderConfiguration}
**Appears in:**
- [LeaderMigrationConfiguration](#controllermanager-config-k8s-io-v1alpha1-LeaderMigrationConfiguration)
<p>ControllerLeaderConfiguration provides the configuration for a migrating leader lock.</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>name</code> <B>[Required]</B><br/>
<code>string</code>
</td>
<td>
<p>Name is the name of the controller being migrated
E.g. service-controller, route-controller, cloud-node-controller, etc</p>
</td>
</tr>
<tr><td><code>component</code> <B>[Required]</B><br/>
<code>string</code>
</td>
<td>
<p>Component is the name of the component in which the controller should be running.
E.g. kube-controller-manager, cloud-controller-manager, etc
Or '*' meaning the controller can be run under any component that participates in the migration</p>
</td>
</tr>
</tbody>
</table>
## `GenericControllerManagerConfiguration` {#controllermanager-config-k8s-io-v1alpha1-GenericControllerManagerConfiguration}
**Appears in:**
- [CloudControllerManagerConfiguration](#cloudcontrollermanager-config-k8s-io-v1alpha1-CloudControllerManagerConfiguration)
- [KubeControllerManagerConfiguration](#kubecontrollermanager-config-k8s-io-v1alpha1-KubeControllerManagerConfiguration)
<p>GenericControllerManagerConfiguration holds configuration for a generic controller-manager.</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>Port</code> <B>[Required]</B><br/>
<code>int32</code>
</td>
<td>
<p>port is the port that the controller-manager's http service runs on.</p>
</td>
</tr>
<tr><td><code>Address</code> <B>[Required]</B><br/>
<code>string</code>
</td>
<td>
<p>address is the IP address to serve on (set to 0.0.0.0 for all interfaces).</p>
</td>
</tr>
<tr><td><code>MinResyncPeriod</code> <B>[Required]</B><br/>
<a href="https://pkg.go.dev/k8s.io/apimachinery/pkg/apis/meta/v1#Duration"><code>meta/v1.Duration</code></a>
</td>
<td>
<p>minResyncPeriod is the resync period in reflectors; will be random between
minResyncPeriod and 2*minResyncPeriod.</p>
</td>
</tr>
<tr><td><code>ClientConnection</code> <B>[Required]</B><br/>
<a href="#ClientConnectionConfiguration"><code>ClientConnectionConfiguration</code></a>
</td>
<td>
<p>ClientConnection specifies the kubeconfig file and client connection
settings for the proxy server to use when communicating with the apiserver.</p>
</td>
</tr>
<tr><td><code>ControllerStartInterval</code> <B>[Required]</B><br/>
<a href="https://pkg.go.dev/k8s.io/apimachinery/pkg/apis/meta/v1#Duration"><code>meta/v1.Duration</code></a>
</td>
<td>
<p>How long to wait between starting controller managers</p>
</td>
</tr>
<tr><td><code>LeaderElection</code> <B>[Required]</B><br/>
<a href="#LeaderElectionConfiguration"><code>LeaderElectionConfiguration</code></a>
</td>
<td>
<p>leaderElection defines the configuration of leader election client.</p>
</td>
</tr>
<tr><td><code>Controllers</code> <B>[Required]</B><br/>
<code>[]string</code>
</td>
<td>
<p>Controllers is the list of controllers to enable or disable
'*' means &quot;all enabled by default controllers&quot;
'foo' means &quot;enable 'foo'&quot;
'-foo' means &quot;disable 'foo'&quot;
first item for a particular name wins</p>
</td>
</tr>
<tr><td><code>Debugging</code> <B>[Required]</B><br/>
<a href="#DebuggingConfiguration"><code>DebuggingConfiguration</code></a>
</td>
<td>
<p>DebuggingConfiguration holds configuration for Debugging related features.</p>
</td>
</tr>
<tr><td><code>LeaderMigrationEnabled</code> <B>[Required]</B><br/>
<code>bool</code>
</td>
<td>
<p>LeaderMigrationEnabled indicates whether Leader Migration should be enabled for the controller manager.</p>
</td>
</tr>
<tr><td><code>LeaderMigration</code> <B>[Required]</B><br/>
<a href="#controllermanager-config-k8s-io-v1alpha1-LeaderMigrationConfiguration"><code>LeaderMigrationConfiguration</code></a>
</td>
<td>
<p>LeaderMigration holds the configuration for Leader Migration.</p>
</td>
</tr>
</tbody>
</table>
## `KubeControllerManagerConfiguration` {#kubecontrollermanager-config-k8s-io-v1alpha1-KubeControllerManagerConfiguration}
@ -1466,486 +1946,4 @@ volume plugin should search for additional third party volume plugins</p>
</tr>
</tbody>
</table>
## `NodeControllerConfiguration` {#NodeControllerConfiguration}
**Appears in:**
- [CloudControllerManagerConfiguration](#cloudcontrollermanager-config-k8s-io-v1alpha1-CloudControllerManagerConfiguration)
<p>NodeControllerConfiguration contains elements describing NodeController.</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>ConcurrentNodeSyncs</code> <B>[Required]</B><br/>
<code>int32</code>
</td>
<td>
<p>ConcurrentNodeSyncs is the number of workers
concurrently synchronizing nodes</p>
</td>
</tr>
</tbody>
</table>
## `ServiceControllerConfiguration` {#ServiceControllerConfiguration}
**Appears in:**
- [CloudControllerManagerConfiguration](#cloudcontrollermanager-config-k8s-io-v1alpha1-CloudControllerManagerConfiguration)
- [KubeControllerManagerConfiguration](#kubecontrollermanager-config-k8s-io-v1alpha1-KubeControllerManagerConfiguration)
<p>ServiceControllerConfiguration contains elements describing ServiceController.</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>ConcurrentServiceSyncs</code> <B>[Required]</B><br/>
<code>int32</code>
</td>
<td>
<p>concurrentServiceSyncs is the number of services that are
allowed to sync concurrently. Larger number = more responsive service
management, but more CPU (and network) load.</p>
</td>
</tr>
</tbody>
</table>
## `CloudControllerManagerConfiguration` {#cloudcontrollermanager-config-k8s-io-v1alpha1-CloudControllerManagerConfiguration}
<p>CloudControllerManagerConfiguration contains elements describing cloud-controller manager.</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>apiVersion</code><br/>string</td><td><code>cloudcontrollermanager.config.k8s.io/v1alpha1</code></td></tr>
<tr><td><code>kind</code><br/>string</td><td><code>CloudControllerManagerConfiguration</code></td></tr>
<tr><td><code>Generic</code> <B>[Required]</B><br/>
<a href="#controllermanager-config-k8s-io-v1alpha1-GenericControllerManagerConfiguration"><code>GenericControllerManagerConfiguration</code></a>
</td>
<td>
<p>Generic holds configuration for a generic controller-manager</p>
</td>
</tr>
<tr><td><code>KubeCloudShared</code> <B>[Required]</B><br/>
<a href="#cloudcontrollermanager-config-k8s-io-v1alpha1-KubeCloudSharedConfiguration"><code>KubeCloudSharedConfiguration</code></a>
</td>
<td>
<p>KubeCloudSharedConfiguration holds configuration for shared related features
both in cloud controller manager and kube-controller manager.</p>
</td>
</tr>
<tr><td><code>NodeController</code> <B>[Required]</B><br/>
<a href="#NodeControllerConfiguration"><code>NodeControllerConfiguration</code></a>
</td>
<td>
<p>NodeController holds configuration for node controller
related features.</p>
</td>
</tr>
<tr><td><code>ServiceController</code> <B>[Required]</B><br/>
<a href="#ServiceControllerConfiguration"><code>ServiceControllerConfiguration</code></a>
</td>
<td>
<p>ServiceControllerConfiguration holds configuration for ServiceController
related features.</p>
</td>
</tr>
<tr><td><code>NodeStatusUpdateFrequency</code> <B>[Required]</B><br/>
<a href="https://pkg.go.dev/k8s.io/apimachinery/pkg/apis/meta/v1#Duration"><code>meta/v1.Duration</code></a>
</td>
<td>
<p>NodeStatusUpdateFrequency is the frequency at which the controller updates nodes' status</p>
</td>
</tr>
<tr><td><code>Webhook</code> <B>[Required]</B><br/>
<a href="#cloudcontrollermanager-config-k8s-io-v1alpha1-WebhookConfiguration"><code>WebhookConfiguration</code></a>
</td>
<td>
<p>Webhook is the configuration for cloud-controller-manager hosted webhooks</p>
</td>
</tr>
</tbody>
</table>
## `CloudProviderConfiguration` {#cloudcontrollermanager-config-k8s-io-v1alpha1-CloudProviderConfiguration}
**Appears in:**
- [KubeCloudSharedConfiguration](#cloudcontrollermanager-config-k8s-io-v1alpha1-KubeCloudSharedConfiguration)
<p>CloudProviderConfiguration contains basically elements about cloud provider.</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>Name</code> <B>[Required]</B><br/>
<code>string</code>
</td>
<td>
<p>Name is the provider for cloud services.</p>
</td>
</tr>
<tr><td><code>CloudConfigFile</code> <B>[Required]</B><br/>
<code>string</code>
</td>
<td>
<p>cloudConfigFile is the path to the cloud provider configuration file.</p>
</td>
</tr>
</tbody>
</table>
## `KubeCloudSharedConfiguration` {#cloudcontrollermanager-config-k8s-io-v1alpha1-KubeCloudSharedConfiguration}
**Appears in:**
- [CloudControllerManagerConfiguration](#cloudcontrollermanager-config-k8s-io-v1alpha1-CloudControllerManagerConfiguration)
- [KubeControllerManagerConfiguration](#kubecontrollermanager-config-k8s-io-v1alpha1-KubeControllerManagerConfiguration)
<p>KubeCloudSharedConfiguration contains elements shared by both kube-controller manager
and cloud-controller manager, but not genericconfig.</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>CloudProvider</code> <B>[Required]</B><br/>
<a href="#cloudcontrollermanager-config-k8s-io-v1alpha1-CloudProviderConfiguration"><code>CloudProviderConfiguration</code></a>
</td>
<td>
<p>CloudProviderConfiguration holds configuration for CloudProvider related features.</p>
</td>
</tr>
<tr><td><code>ExternalCloudVolumePlugin</code> <B>[Required]</B><br/>
<code>string</code>
</td>
<td>
<p>externalCloudVolumePlugin specifies the plugin to use when cloudProvider is &quot;external&quot;.
It is currently used by the in repo cloud providers to handle node and volume control in the KCM.</p>
</td>
</tr>
<tr><td><code>UseServiceAccountCredentials</code> <B>[Required]</B><br/>
<code>bool</code>
</td>
<td>
<p>useServiceAccountCredentials indicates whether controllers should be run with
individual service account credentials.</p>
</td>
</tr>
<tr><td><code>AllowUntaggedCloud</code> <B>[Required]</B><br/>
<code>bool</code>
</td>
<td>
<p>run with untagged cloud instances</p>
</td>
</tr>
<tr><td><code>RouteReconciliationPeriod</code> <B>[Required]</B><br/>
<a href="https://pkg.go.dev/k8s.io/apimachinery/pkg/apis/meta/v1#Duration"><code>meta/v1.Duration</code></a>
</td>
<td>
<p>routeReconciliationPeriod is the period for reconciling routes created for Nodes by cloud provider..</p>
</td>
</tr>
<tr><td><code>NodeMonitorPeriod</code> <B>[Required]</B><br/>
<a href="https://pkg.go.dev/k8s.io/apimachinery/pkg/apis/meta/v1#Duration"><code>meta/v1.Duration</code></a>
</td>
<td>
<p>nodeMonitorPeriod is the period for syncing NodeStatus in NodeController.</p>
</td>
</tr>
<tr><td><code>ClusterName</code> <B>[Required]</B><br/>
<code>string</code>
</td>
<td>
<p>clusterName is the instance prefix for the cluster.</p>
</td>
</tr>
<tr><td><code>ClusterCIDR</code> <B>[Required]</B><br/>
<code>string</code>
</td>
<td>
<p>clusterCIDR is CIDR Range for Pods in cluster.</p>
</td>
</tr>
<tr><td><code>AllocateNodeCIDRs</code> <B>[Required]</B><br/>
<code>bool</code>
</td>
<td>
<p>AllocateNodeCIDRs enables CIDRs for Pods to be allocated and, if
ConfigureCloudRoutes is true, to be set on the cloud provider.</p>
</td>
</tr>
<tr><td><code>CIDRAllocatorType</code> <B>[Required]</B><br/>
<code>string</code>
</td>
<td>
<p>CIDRAllocatorType determines what kind of pod CIDR allocator will be used.</p>
</td>
</tr>
<tr><td><code>ConfigureCloudRoutes</code> <B>[Required]</B><br/>
<code>bool</code>
</td>
<td>
<p>configureCloudRoutes enables CIDRs allocated with allocateNodeCIDRs
to be configured on the cloud provider.</p>
</td>
</tr>
<tr><td><code>NodeSyncPeriod</code> <B>[Required]</B><br/>
<a href="https://pkg.go.dev/k8s.io/apimachinery/pkg/apis/meta/v1#Duration"><code>meta/v1.Duration</code></a>
</td>
<td>
<p>nodeSyncPeriod is the period for syncing nodes from cloudprovider. Longer
periods will result in fewer calls to cloud provider, but may delay addition
of new nodes to cluster.</p>
</td>
</tr>
</tbody>
</table>
## `WebhookConfiguration` {#cloudcontrollermanager-config-k8s-io-v1alpha1-WebhookConfiguration}
**Appears in:**
- [CloudControllerManagerConfiguration](#cloudcontrollermanager-config-k8s-io-v1alpha1-CloudControllerManagerConfiguration)
<p>WebhookConfiguration contains configuration related to
cloud-controller-manager hosted webhooks</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>Webhooks</code> <B>[Required]</B><br/>
<code>[]string</code>
</td>
<td>
<p>Webhooks is the list of webhooks to enable or disable
'*' means &quot;all enabled by default webhooks&quot;
'foo' means &quot;enable 'foo'&quot;
'-foo' means &quot;disable 'foo'&quot;
first item for a particular name wins</p>
</td>
</tr>
</tbody>
</table>
## `LeaderMigrationConfiguration` {#controllermanager-config-k8s-io-v1alpha1-LeaderMigrationConfiguration}
**Appears in:**
- [GenericControllerManagerConfiguration](#controllermanager-config-k8s-io-v1alpha1-GenericControllerManagerConfiguration)
<p>LeaderMigrationConfiguration provides versioned configuration for all migrating leader locks.</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>apiVersion</code><br/>string</td><td><code>controllermanager.config.k8s.io/v1alpha1</code></td></tr>
<tr><td><code>kind</code><br/>string</td><td><code>LeaderMigrationConfiguration</code></td></tr>
<tr><td><code>leaderName</code> <B>[Required]</B><br/>
<code>string</code>
</td>
<td>
<p>LeaderName is the name of the leader election resource that protects the migration
E.g. 1-20-KCM-to-1-21-CCM</p>
</td>
</tr>
<tr><td><code>resourceLock</code> <B>[Required]</B><br/>
<code>string</code>
</td>
<td>
<p>ResourceLock indicates the resource object type that will be used to lock
Should be &quot;leases&quot; or &quot;endpoints&quot;</p>
</td>
</tr>
<tr><td><code>controllerLeaders</code> <B>[Required]</B><br/>
<a href="#controllermanager-config-k8s-io-v1alpha1-ControllerLeaderConfiguration"><code>[]ControllerLeaderConfiguration</code></a>
</td>
<td>
<p>ControllerLeaders contains a list of migrating leader lock configurations</p>
</td>
</tr>
</tbody>
</table>
## `ControllerLeaderConfiguration` {#controllermanager-config-k8s-io-v1alpha1-ControllerLeaderConfiguration}
**Appears in:**
- [LeaderMigrationConfiguration](#controllermanager-config-k8s-io-v1alpha1-LeaderMigrationConfiguration)
<p>ControllerLeaderConfiguration provides the configuration for a migrating leader lock.</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>name</code> <B>[Required]</B><br/>
<code>string</code>
</td>
<td>
<p>Name is the name of the controller being migrated
E.g. service-controller, route-controller, cloud-node-controller, etc</p>
</td>
</tr>
<tr><td><code>component</code> <B>[Required]</B><br/>
<code>string</code>
</td>
<td>
<p>Component is the name of the component in which the controller should be running.
E.g. kube-controller-manager, cloud-controller-manager, etc
Or '*' meaning the controller can be run under any component that participates in the migration</p>
</td>
</tr>
</tbody>
</table>
## `GenericControllerManagerConfiguration` {#controllermanager-config-k8s-io-v1alpha1-GenericControllerManagerConfiguration}
**Appears in:**
- [CloudControllerManagerConfiguration](#cloudcontrollermanager-config-k8s-io-v1alpha1-CloudControllerManagerConfiguration)
- [KubeControllerManagerConfiguration](#kubecontrollermanager-config-k8s-io-v1alpha1-KubeControllerManagerConfiguration)
<p>GenericControllerManagerConfiguration holds configuration for a generic controller-manager.</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>Port</code> <B>[Required]</B><br/>
<code>int32</code>
</td>
<td>
<p>port is the port that the controller-manager's http service runs on.</p>
</td>
</tr>
<tr><td><code>Address</code> <B>[Required]</B><br/>
<code>string</code>
</td>
<td>
<p>address is the IP address to serve on (set to 0.0.0.0 for all interfaces).</p>
</td>
</tr>
<tr><td><code>MinResyncPeriod</code> <B>[Required]</B><br/>
<a href="https://pkg.go.dev/k8s.io/apimachinery/pkg/apis/meta/v1#Duration"><code>meta/v1.Duration</code></a>
</td>
<td>
<p>minResyncPeriod is the resync period in reflectors; will be random between
minResyncPeriod and 2*minResyncPeriod.</p>
</td>
</tr>
<tr><td><code>ClientConnection</code> <B>[Required]</B><br/>
<a href="#ClientConnectionConfiguration"><code>ClientConnectionConfiguration</code></a>
</td>
<td>
<p>ClientConnection specifies the kubeconfig file and client connection
settings for the proxy server to use when communicating with the apiserver.</p>
</td>
</tr>
<tr><td><code>ControllerStartInterval</code> <B>[Required]</B><br/>
<a href="https://pkg.go.dev/k8s.io/apimachinery/pkg/apis/meta/v1#Duration"><code>meta/v1.Duration</code></a>
</td>
<td>
<p>How long to wait between starting controller managers</p>
</td>
</tr>
<tr><td><code>LeaderElection</code> <B>[Required]</B><br/>
<a href="#LeaderElectionConfiguration"><code>LeaderElectionConfiguration</code></a>
</td>
<td>
<p>leaderElection defines the configuration of leader election client.</p>
</td>
</tr>
<tr><td><code>Controllers</code> <B>[Required]</B><br/>
<code>[]string</code>
</td>
<td>
<p>Controllers is the list of controllers to enable or disable
'*' means &quot;all enabled by default controllers&quot;
'foo' means &quot;enable 'foo'&quot;
'-foo' means &quot;disable 'foo'&quot;
first item for a particular name wins</p>
</td>
</tr>
<tr><td><code>Debugging</code> <B>[Required]</B><br/>
<a href="#DebuggingConfiguration"><code>DebuggingConfiguration</code></a>
</td>
<td>
<p>DebuggingConfiguration holds configuration for Debugging related features.</p>
</td>
</tr>
<tr><td><code>LeaderMigrationEnabled</code> <B>[Required]</B><br/>
<code>bool</code>
</td>
<td>
<p>LeaderMigrationEnabled indicates whether Leader Migration should be enabled for the controller manager.</p>
</td>
</tr>
<tr><td><code>LeaderMigration</code> <B>[Required]</B><br/>
<a href="#controllermanager-config-k8s-io-v1alpha1-LeaderMigrationConfiguration"><code>LeaderMigrationConfiguration</code></a>
</td>
<td>
<p>LeaderMigration holds the configuration for Leader Migration.</p>
</td>
</tr>
</tbody>
</table>

6
content/en/docs/reference/config-api/kube-proxy-config.v1alpha1.md
View File

@ -12,6 +12,7 @@ auto_generated: true
- [KubeProxyConfiguration](#kubeproxy-config-k8s-io-v1alpha1-KubeProxyConfiguration)
## `ClientConnectionConfiguration` {#ClientConnectionConfiguration}
@ -80,10 +81,10 @@ client.</p>
**Appears in:**
- [KubeSchedulerConfiguration](#kubescheduler-config-k8s-io-v1-KubeSchedulerConfiguration)
- [KubeSchedulerConfiguration](#kubescheduler-config-k8s-io-v1beta3-KubeSchedulerConfiguration)
- [KubeSchedulerConfiguration](#kubescheduler-config-k8s-io-v1-KubeSchedulerConfiguration)
- [GenericControllerManagerConfiguration](#controllermanager-config-k8s-io-v1alpha1-GenericControllerManagerConfiguration)
@ -201,7 +202,6 @@ during leader election cycles.</p>
</tbody>
</table>
## `KubeProxyConfiguration` {#kubeproxy-config-k8s-io-v1alpha1-KubeProxyConfiguration}

6
content/en/docs/reference/config-api/kube-scheduler-config.v1.md
View File

@ -19,6 +19,7 @@ auto_generated: true
- [VolumeBindingArgs](#kubescheduler-config-k8s-io-v1-VolumeBindingArgs)
## `ClientConnectionConfiguration` {#ClientConnectionConfiguration}
@ -119,10 +120,10 @@ enableProfiling is true.</p>
**Appears in:**
- [KubeSchedulerConfiguration](#kubescheduler-config-k8s-io-v1-KubeSchedulerConfiguration)
- [KubeSchedulerConfiguration](#kubescheduler-config-k8s-io-v1beta3-KubeSchedulerConfiguration)
- [KubeSchedulerConfiguration](#kubescheduler-config-k8s-io-v1-KubeSchedulerConfiguration)
<p>LeaderElectionConfiguration defines the configuration of leader election
clients for components that can run with leader election enabled.</p>
@ -200,7 +201,6 @@ during leader election cycles.</p>
</tbody>
</table>
## `DefaultPreemptionArgs` {#kubescheduler-config-k8s-io-v1-DefaultPreemptionArgs}

354
content/en/docs/reference/config-api/kube-scheduler-config.v1beta3.md
View File

@ -19,6 +19,182 @@ auto_generated: true
- [VolumeBindingArgs](#kubescheduler-config-k8s-io-v1beta3-VolumeBindingArgs)
## `ClientConnectionConfiguration` {#ClientConnectionConfiguration}
**Appears in:**
- [KubeSchedulerConfiguration](#kubescheduler-config-k8s-io-v1beta3-KubeSchedulerConfiguration)
<p>ClientConnectionConfiguration contains details for constructing a client.</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>kubeconfig</code> <B>[Required]</B><br/>
<code>string</code>
</td>
<td>
<p>kubeconfig is the path to a KubeConfig file.</p>
</td>
</tr>
<tr><td><code>acceptContentTypes</code> <B>[Required]</B><br/>
<code>string</code>
</td>
<td>
<p>acceptContentTypes defines the Accept header sent by clients when connecting to a server, overriding the
default value of 'application/json'. This field will control all connections to the server used by a particular
client.</p>
</td>
</tr>
<tr><td><code>contentType</code> <B>[Required]</B><br/>
<code>string</code>
</td>
<td>
<p>contentType is the content type used when sending data to the server from this client.</p>
</td>
</tr>
<tr><td><code>qps</code> <B>[Required]</B><br/>
<code>float32</code>
</td>
<td>
<p>qps controls the number of queries per second allowed for this connection.</p>
</td>
</tr>
<tr><td><code>burst</code> <B>[Required]</B><br/>
<code>int32</code>
</td>
<td>
<p>burst allows extra queries to accumulate when a client is exceeding its rate.</p>
</td>
</tr>
</tbody>
</table>
## `DebuggingConfiguration` {#DebuggingConfiguration}
**Appears in:**
- [KubeSchedulerConfiguration](#kubescheduler-config-k8s-io-v1beta3-KubeSchedulerConfiguration)
<p>DebuggingConfiguration holds configuration for Debugging related features.</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>enableProfiling</code> <B>[Required]</B><br/>
<code>bool</code>
</td>
<td>
<p>enableProfiling enables profiling via web interface host:port/debug/pprof/</p>
</td>
</tr>
<tr><td><code>enableContentionProfiling</code> <B>[Required]</B><br/>
<code>bool</code>
</td>
<td>
<p>enableContentionProfiling enables block profiling, if
enableProfiling is true.</p>
</td>
</tr>
</tbody>
</table>
## `LeaderElectionConfiguration` {#LeaderElectionConfiguration}
**Appears in:**
- [KubeSchedulerConfiguration](#kubescheduler-config-k8s-io-v1beta3-KubeSchedulerConfiguration)
<p>LeaderElectionConfiguration defines the configuration of leader election
clients for components that can run with leader election enabled.</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>leaderElect</code> <B>[Required]</B><br/>
<code>bool</code>
</td>
<td>
<p>leaderElect enables a leader election client to gain leadership
before executing the main loop. Enable this when running replicated
components for high availability.</p>
</td>
</tr>
<tr><td><code>leaseDuration</code> <B>[Required]</B><br/>
<a href="https://pkg.go.dev/k8s.io/apimachinery/pkg/apis/meta/v1#Duration"><code>meta/v1.Duration</code></a>
</td>
<td>
<p>leaseDuration is the duration that non-leader candidates will wait
after observing a leadership renewal until attempting to acquire
leadership of a led but unrenewed leader slot. This is effectively the
maximum duration that a leader can be stopped before it is replaced
by another candidate. This is only applicable if leader election is
enabled.</p>
</td>
</tr>
<tr><td><code>renewDeadline</code> <B>[Required]</B><br/>
<a href="https://pkg.go.dev/k8s.io/apimachinery/pkg/apis/meta/v1#Duration"><code>meta/v1.Duration</code></a>
</td>
<td>
<p>renewDeadline is the interval between attempts by the acting master to
renew a leadership slot before it stops leading. This must be less
than or equal to the lease duration. This is only applicable if leader
election is enabled.</p>
</td>
</tr>
<tr><td><code>retryPeriod</code> <B>[Required]</B><br/>
<a href="https://pkg.go.dev/k8s.io/apimachinery/pkg/apis/meta/v1#Duration"><code>meta/v1.Duration</code></a>
</td>
<td>
<p>retryPeriod is the duration the clients should wait between attempting
acquisition and renewal of a leadership. This is only applicable if
leader election is enabled.</p>
</td>
</tr>
<tr><td><code>resourceLock</code> <B>[Required]</B><br/>
<code>string</code>
</td>
<td>
<p>resourceLock indicates the resource object type that will be used to lock
during leader election cycles.</p>
</td>
</tr>
<tr><td><code>resourceName</code> <B>[Required]</B><br/>
<code>string</code>
</td>
<td>
<p>resourceName indicates the name of resource object that will be used to lock
during leader election cycles.</p>
</td>
</tr>
<tr><td><code>resourceNamespace</code> <B>[Required]</B><br/>
<code>string</code>
</td>
<td>
<p>resourceName indicates the namespace of resource object that will be used to lock
during leader election cycles.</p>
</td>
</tr>
</tbody>
</table>
## `DefaultPreemptionArgs` {#kubescheduler-config-k8s-io-v1beta3-DefaultPreemptionArgs}
@ -1074,180 +1250,4 @@ Weight defaults to 1 if not specified or explicitly set to 0.</p>
</tr>
</tbody>
</table>
## `ClientConnectionConfiguration` {#ClientConnectionConfiguration}
**Appears in:**
- [KubeSchedulerConfiguration](#kubescheduler-config-k8s-io-v1beta3-KubeSchedulerConfiguration)
<p>ClientConnectionConfiguration contains details for constructing a client.</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>kubeconfig</code> <B>[Required]</B><br/>
<code>string</code>
</td>
<td>
<p>kubeconfig is the path to a KubeConfig file.</p>
</td>
</tr>
<tr><td><code>acceptContentTypes</code> <B>[Required]</B><br/>
<code>string</code>
</td>
<td>
<p>acceptContentTypes defines the Accept header sent by clients when connecting to a server, overriding the
default value of 'application/json'. This field will control all connections to the server used by a particular
client.</p>
</td>
</tr>
<tr><td><code>contentType</code> <B>[Required]</B><br/>
<code>string</code>
</td>
<td>
<p>contentType is the content type used when sending data to the server from this client.</p>
</td>
</tr>
<tr><td><code>qps</code> <B>[Required]</B><br/>
<code>float32</code>
</td>
<td>
<p>qps controls the number of queries per second allowed for this connection.</p>
</td>
</tr>
<tr><td><code>burst</code> <B>[Required]</B><br/>
<code>int32</code>
</td>
<td>
<p>burst allows extra queries to accumulate when a client is exceeding its rate.</p>
</td>
</tr>
</tbody>
</table>
## `DebuggingConfiguration` {#DebuggingConfiguration}
**Appears in:**
- [KubeSchedulerConfiguration](#kubescheduler-config-k8s-io-v1beta3-KubeSchedulerConfiguration)
<p>DebuggingConfiguration holds configuration for Debugging related features.</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>enableProfiling</code> <B>[Required]</B><br/>
<code>bool</code>
</td>
<td>
<p>enableProfiling enables profiling via web interface host:port/debug/pprof/</p>
</td>
</tr>
<tr><td><code>enableContentionProfiling</code> <B>[Required]</B><br/>
<code>bool</code>
</td>
<td>
<p>enableContentionProfiling enables block profiling, if
enableProfiling is true.</p>
</td>
</tr>
</tbody>
</table>
## `LeaderElectionConfiguration` {#LeaderElectionConfiguration}
**Appears in:**
- [KubeSchedulerConfiguration](#kubescheduler-config-k8s-io-v1beta3-KubeSchedulerConfiguration)
<p>LeaderElectionConfiguration defines the configuration of leader election
clients for components that can run with leader election enabled.</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>leaderElect</code> <B>[Required]</B><br/>
<code>bool</code>
</td>
<td>
<p>leaderElect enables a leader election client to gain leadership
before executing the main loop. Enable this when running replicated
components for high availability.</p>
</td>
</tr>
<tr><td><code>leaseDuration</code> <B>[Required]</B><br/>
<a href="https://pkg.go.dev/k8s.io/apimachinery/pkg/apis/meta/v1#Duration"><code>meta/v1.Duration</code></a>
</td>
<td>
<p>leaseDuration is the duration that non-leader candidates will wait
after observing a leadership renewal until attempting to acquire
leadership of a led but unrenewed leader slot. This is effectively the
maximum duration that a leader can be stopped before it is replaced
by another candidate. This is only applicable if leader election is
enabled.</p>
</td>
</tr>
<tr><td><code>renewDeadline</code> <B>[Required]</B><br/>
<a href="https://pkg.go.dev/k8s.io/apimachinery/pkg/apis/meta/v1#Duration"><code>meta/v1.Duration</code></a>
</td>
<td>
<p>renewDeadline is the interval between attempts by the acting master to
renew a leadership slot before it stops leading. This must be less
than or equal to the lease duration. This is only applicable if leader
election is enabled.</p>
</td>
</tr>
<tr><td><code>retryPeriod</code> <B>[Required]</B><br/>
<a href="https://pkg.go.dev/k8s.io/apimachinery/pkg/apis/meta/v1#Duration"><code>meta/v1.Duration</code></a>
</td>
<td>
<p>retryPeriod is the duration the clients should wait between attempting
acquisition and renewal of a leadership. This is only applicable if
leader election is enabled.</p>
</td>
</tr>
<tr><td><code>resourceLock</code> <B>[Required]</B><br/>
<code>string</code>
</td>
<td>
<p>resourceLock indicates the resource object type that will be used to lock
during leader election cycles.</p>
</td>
</tr>
<tr><td><code>resourceName</code> <B>[Required]</B><br/>
<code>string</code>
</td>
<td>
<p>resourceName indicates the name of resource object that will be used to lock
during leader election cycles.</p>
</td>
</tr>
<tr><td><code>resourceNamespace</code> <B>[Required]</B><br/>
<code>string</code>
</td>
<td>
<p>resourceName indicates the namespace of resource object that will be used to lock
during leader election cycles.</p>
</td>
</tr>
</tbody>
</table>

208
content/en/docs/reference/config-api/kubeadm-config.v1beta3.md
View File

@ -264,6 +264,109 @@ node only (e.g. the node ip).</p>
- [JoinConfiguration](#kubeadm-k8s-io-v1beta3-JoinConfiguration)
## `BootstrapToken` {#BootstrapToken}
**Appears in:**
- [InitConfiguration](#kubeadm-k8s-io-v1beta3-InitConfiguration)
<p>BootstrapToken describes one bootstrap token, stored as a Secret in the cluster</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>token</code> <B>[Required]</B><br/>
<a href="#BootstrapTokenString"><code>BootstrapTokenString</code></a>
</td>
<td>
<p><code>token</code> is used for establishing bidirectional trust between nodes and control-planes.
Used for joining nodes in the cluster.</p>
</td>
</tr>
<tr><td><code>description</code><br/>
<code>string</code>
</td>
<td>
<p><code>description</code> sets a human-friendly message why this token exists and what it's used
for, so other administrators can know its purpose.</p>
</td>
</tr>
<tr><td><code>ttl</code><br/>
<a href="https://pkg.go.dev/k8s.io/apimachinery/pkg/apis/meta/v1#Duration"><code>meta/v1.Duration</code></a>
</td>
<td>
<p><code>ttl</code> defines the time to live for this token. Defaults to <code>24h</code>.
<code>expires</code> and <code>ttl</code> are mutually exclusive.</p>
</td>
</tr>
<tr><td><code>expires</code><br/>
<a href="https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.28/#time-v1-meta"><code>meta/v1.Time</code></a>
</td>
<td>
<p><code>expires</code> specifies the timestamp when this token expires. Defaults to being set
dynamically at runtime based on the <code>ttl</code>. <code>expires</code> and <code>ttl</code> are mutually exclusive.</p>
</td>
</tr>
<tr><td><code>usages</code><br/>
<code>[]string</code>
</td>
<td>
<p><code>usages</code> describes the ways in which this token can be used. Can by default be used
for establishing bidirectional trust, but that can be changed here.</p>
</td>
</tr>
<tr><td><code>groups</code><br/>
<code>[]string</code>
</td>
<td>
<p><code>groups</code> specifies the extra groups that this token will authenticate as when/if
used for authentication</p>
</td>
</tr>
</tbody>
</table>
## `BootstrapTokenString` {#BootstrapTokenString}
**Appears in:**
- [BootstrapToken](#BootstrapToken)
<p>BootstrapTokenString is a token of the format <code>abcdef.abcdef0123456789</code> that is used
for both validation of the practically of the API server from a joining node's point
of view and as an authentication method for the node in the bootstrap phase of
&quot;kubeadm join&quot;. This token is and should be short-lived.</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>-</code> <B>[Required]</B><br/>
<code>string</code>
</td>
<td>
<span class="text-muted">No description provided.</span></td>
</tr>
<tr><td><code>-</code> <B>[Required]</B><br/>
<code>string</code>
</td>
<td>
<span class="text-muted">No description provided.</span></td>
</tr>
</tbody>
</table>
## `ClusterConfiguration` {#kubeadm-k8s-io-v1beta3-ClusterConfiguration}
@ -1237,107 +1340,4 @@ first alpha-numerically.</p>
</tr>
</tbody>
</table>
## `BootstrapToken` {#BootstrapToken}
**Appears in:**
- [InitConfiguration](#kubeadm-k8s-io-v1beta3-InitConfiguration)
<p>BootstrapToken describes one bootstrap token, stored as a Secret in the cluster</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>token</code> <B>[Required]</B><br/>
<a href="#BootstrapTokenString"><code>BootstrapTokenString</code></a>
</td>
<td>
<p><code>token</code> is used for establishing bidirectional trust between nodes and control-planes.
Used for joining nodes in the cluster.</p>
</td>
</tr>
<tr><td><code>description</code><br/>
<code>string</code>
</td>
<td>
<p><code>description</code> sets a human-friendly message why this token exists and what it's used
for, so other administrators can know its purpose.</p>
</td>
</tr>
<tr><td><code>ttl</code><br/>
<a href="https://pkg.go.dev/k8s.io/apimachinery/pkg/apis/meta/v1#Duration"><code>meta/v1.Duration</code></a>
</td>
<td>
<p><code>ttl</code> defines the time to live for this token. Defaults to <code>24h</code>.
<code>expires</code> and <code>ttl</code> are mutually exclusive.</p>
</td>
</tr>
<tr><td><code>expires</code><br/>
<a href="https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.28/#time-v1-meta"><code>meta/v1.Time</code></a>
</td>
<td>
<p><code>expires</code> specifies the timestamp when this token expires. Defaults to being set
dynamically at runtime based on the <code>ttl</code>. <code>expires</code> and <code>ttl</code> are mutually exclusive.</p>
</td>
</tr>
<tr><td><code>usages</code><br/>
<code>[]string</code>
</td>
<td>
<p><code>usages</code> describes the ways in which this token can be used. Can by default be used
for establishing bidirectional trust, but that can be changed here.</p>
</td>
</tr>
<tr><td><code>groups</code><br/>
<code>[]string</code>
</td>
<td>
<p><code>groups</code> specifies the extra groups that this token will authenticate as when/if
used for authentication</p>
</td>
</tr>
</tbody>
</table>
## `BootstrapTokenString` {#BootstrapTokenString}
**Appears in:**
- [BootstrapToken](#BootstrapToken)
<p>BootstrapTokenString is a token of the format <code>abcdef.abcdef0123456789</code> that is used
for both validation of the practically of the API server from a joining node's point
of view and as an authentication method for the node in the bootstrap phase of
&quot;kubeadm join&quot;. This token is and should be short-lived.</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>-</code> <B>[Required]</B><br/>
<code>string</code>
</td>
<td>
<span class="text-muted">No description provided.</span></td>
</tr>
<tr><td><code>-</code> <B>[Required]</B><br/>
<code>string</code>
</td>
<td>
<span class="text-muted">No description provided.</span></td>
</tr>
</tbody>
</table>

212
content/en/docs/reference/config-api/kubeadm-config.v1beta4.md
View File

@ -291,6 +291,111 @@ node only (e.g. the node ip).</p>
- [ResetConfiguration](#kubeadm-k8s-io-v1beta4-ResetConfiguration)
## `BootstrapToken` {#BootstrapToken}
**Appears in:**
- [InitConfiguration](#kubeadm-k8s-io-v1beta3-InitConfiguration)
- [InitConfiguration](#kubeadm-k8s-io-v1beta4-InitConfiguration)
<p>BootstrapToken describes one bootstrap token, stored as a Secret in the cluster</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>token</code> <B>[Required]</B><br/>
<a href="#BootstrapTokenString"><code>BootstrapTokenString</code></a>
</td>
<td>
<p><code>token</code> is used for establishing bidirectional trust between nodes and control-planes.
Used for joining nodes in the cluster.</p>
</td>
</tr>
<tr><td><code>description</code><br/>
<code>string</code>
</td>
<td>
<p><code>description</code> sets a human-friendly message why this token exists and what it's used
for, so other administrators can know its purpose.</p>
</td>
</tr>
<tr><td><code>ttl</code><br/>
<a href="https://pkg.go.dev/k8s.io/apimachinery/pkg/apis/meta/v1#Duration"><code>meta/v1.Duration</code></a>
</td>
<td>
<p><code>ttl</code> defines the time to live for this token. Defaults to <code>24h</code>.
<code>expires</code> and <code>ttl</code> are mutually exclusive.</p>
</td>
</tr>
<tr><td><code>expires</code><br/>
<a href="https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.28/#time-v1-meta"><code>meta/v1.Time</code></a>
</td>
<td>
<p><code>expires</code> specifies the timestamp when this token expires. Defaults to being set
dynamically at runtime based on the <code>ttl</code>. <code>expires</code> and <code>ttl</code> are mutually exclusive.</p>
</td>
</tr>
<tr><td><code>usages</code><br/>
<code>[]string</code>
</td>
<td>
<p><code>usages</code> describes the ways in which this token can be used. Can by default be used
for establishing bidirectional trust, but that can be changed here.</p>
</td>
</tr>
<tr><td><code>groups</code><br/>
<code>[]string</code>
</td>
<td>
<p><code>groups</code> specifies the extra groups that this token will authenticate as when/if
used for authentication</p>
</td>
</tr>
</tbody>
</table>
## `BootstrapTokenString` {#BootstrapTokenString}
**Appears in:**
- [BootstrapToken](#BootstrapToken)
<p>BootstrapTokenString is a token of the format <code>abcdef.abcdef0123456789</code> that is used
for both validation of the practically of the API server from a joining node's point
of view and as an authentication method for the node in the bootstrap phase of
&quot;kubeadm join&quot;. This token is and should be short-lived.</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>-</code> <B>[Required]</B><br/>
<code>string</code>
</td>
<td>
<span class="text-muted">No description provided.</span></td>
</tr>
<tr><td><code>-</code> <B>[Required]</B><br/>
<code>string</code>
</td>
<td>
<span class="text-muted">No description provided.</span></td>
</tr>
</tbody>
</table>
## `ClusterConfiguration` {#kubeadm-k8s-io-v1beta4-ClusterConfiguration}
@ -424,7 +529,7 @@ information.</p>
<tr><td><code>bootstrapTokens</code><br/>
<code>[]invalid type</code>
<a href="#BootstrapToken"><code>[]BootstrapToken</code></a>
</td>
<td>
<p>BootstrapTokens is respected at <code>kubeadm init</code> time and describes a set of Bootstrap Tokens to create.
@ -1322,107 +1427,4 @@ first alpha-numerically.</p>
</tr>
</tbody>
</table>
## `BootstrapToken` {#BootstrapToken}
**Appears in:**
- [InitConfiguration](#kubeadm-k8s-io-v1beta3-InitConfiguration)
<p>BootstrapToken describes one bootstrap token, stored as a Secret in the cluster</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>token</code> <B>[Required]</B><br/>
<a href="#BootstrapTokenString"><code>BootstrapTokenString</code></a>
</td>
<td>
<p><code>token</code> is used for establishing bidirectional trust between nodes and control-planes.
Used for joining nodes in the cluster.</p>
</td>
</tr>
<tr><td><code>description</code><br/>
<code>string</code>
</td>
<td>
<p><code>description</code> sets a human-friendly message why this token exists and what it's used
for, so other administrators can know its purpose.</p>
</td>
</tr>
<tr><td><code>ttl</code><br/>
<a href="https://pkg.go.dev/k8s.io/apimachinery/pkg/apis/meta/v1#Duration"><code>meta/v1.Duration</code></a>
</td>
<td>
<p><code>ttl</code> defines the time to live for this token. Defaults to <code>24h</code>.
<code>expires</code> and <code>ttl</code> are mutually exclusive.</p>
</td>
</tr>
<tr><td><code>expires</code><br/>
<a href="https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.28/#time-v1-meta"><code>meta/v1.Time</code></a>
</td>
<td>
<p><code>expires</code> specifies the timestamp when this token expires. Defaults to being set
dynamically at runtime based on the <code>ttl</code>. <code>expires</code> and <code>ttl</code> are mutually exclusive.</p>
</td>
</tr>
<tr><td><code>usages</code><br/>
<code>[]string</code>
</td>
<td>
<p><code>usages</code> describes the ways in which this token can be used. Can by default be used
for establishing bidirectional trust, but that can be changed here.</p>
</td>
</tr>
<tr><td><code>groups</code><br/>
<code>[]string</code>
</td>
<td>
<p><code>groups</code> specifies the extra groups that this token will authenticate as when/if
used for authentication</p>
</td>
</tr>
</tbody>
</table>
## `BootstrapTokenString` {#BootstrapTokenString}
**Appears in:**
- [BootstrapToken](#BootstrapToken)
<p>BootstrapTokenString is a token of the format <code>abcdef.abcdef0123456789</code> that is used
for both validation of the practically of the API server from a joining node's point
of view and as an authentication method for the node in the bootstrap phase of
&quot;kubeadm join&quot;. This token is and should be short-lived.</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>-</code> <B>[Required]</B><br/>
<code>string</code>
</td>
<td>
<span class="text-muted">No description provided.</span></td>
</tr>
<tr><td><code>-</code> <B>[Required]</B><br/>
<code>string</code>
</td>
<td>
<span class="text-muted">No description provided.</span></td>
</tr>
</tbody>
</table>

77
content/en/docs/reference/config-api/kubeconfig.v1.md
View File

@ -11,6 +11,83 @@ auto_generated: true
- [Config](#Config)
## `Config` {#Config}
<p>Config holds the information needed to build connect to remote kubernetes clusters as a given user</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>apiVersion</code><br/>string</td><td><code>/v1</code></td></tr>
<tr><td><code>kind</code><br/>string</td><td><code>Config</code></td></tr>
<tr><td><code>kind</code><br/>
<code>string</code>
</td>
<td>
<p>Legacy field from pkg/api/types.go TypeMeta.
TODO(jlowdermilk): remove this after eliminating downstream dependencies.</p>
</td>
</tr>
<tr><td><code>apiVersion</code><br/>
<code>string</code>
</td>
<td>
<p>Legacy field from pkg/api/types.go TypeMeta.
TODO(jlowdermilk): remove this after eliminating downstream dependencies.</p>
</td>
</tr>
<tr><td><code>preferences</code> <B>[Required]</B><br/>
<a href="#Preferences"><code>Preferences</code></a>
</td>
<td>
<p>Preferences holds general information to be use for cli interactions</p>
</td>
</tr>
<tr><td><code>clusters</code> <B>[Required]</B><br/>
<a href="#NamedCluster"><code>[]NamedCluster</code></a>
</td>
<td>
<p>Clusters is a map of referencable names to cluster configs</p>
</td>
</tr>
<tr><td><code>users</code> <B>[Required]</B><br/>
<a href="#NamedAuthInfo"><code>[]NamedAuthInfo</code></a>
</td>
<td>
<p>AuthInfos is a map of referencable names to user configs</p>
</td>
</tr>
<tr><td><code>contexts</code> <B>[Required]</B><br/>
<a href="#NamedContext"><code>[]NamedContext</code></a>
</td>
<td>
<p>Contexts is a map of referencable names to context configs</p>
</td>
</tr>
<tr><td><code>current-context</code> <B>[Required]</B><br/>
<code>string</code>
</td>
<td>
<p>CurrentContext is the name of the context that you would like to use by default</p>
</td>
</tr>
<tr><td><code>extensions</code><br/>
<a href="#NamedExtension"><code>[]NamedExtension</code></a>
</td>
<td>
<p>Extensions holds additional information. This is useful for extenders so that reads and writes don't clobber unknown fields</p>
</td>
</tr>
</tbody>
</table>
## `AuthInfo` {#AuthInfo}

3
content/en/docs/reference/config-api/kubelet-config.v1.md
View File

@ -11,7 +11,6 @@ auto_generated: true
- [CredentialProviderConfig](#kubelet-config-k8s-io-v1-CredentialProviderConfig)
## `CredentialProviderConfig` {#kubelet-config-k8s-io-v1-CredentialProviderConfig}
@ -82,7 +81,7 @@ and URL path.</p>
<p>Each entry in matchImages is a pattern which can optionally contain a port and a path.
Globs can be used in the domain, but not in the port or the path. Globs are supported
as subdomains like '&ast;.k8s.io' or 'k8s.&ast;.io', and top-level-domains such as 'k8s.&ast;'.
Matching partial subdomains like 'app&ast;.k8s.io' is also supported. Each glob can only match
Matching partial subdomains like 'app</em>.k8s.io' is also supported. Each glob can only match
a single subdomain segment, so &ast;.io does not match &ast;.k8s.io.</p>
<p>A match exists between an image and a matchImage when all of the below are true:</p>
<ul>

1
content/en/docs/reference/config-api/kubelet-config.v1alpha1.md
View File

@ -11,7 +11,6 @@ auto_generated: true
- [CredentialProviderConfig](#kubelet-config-k8s-io-v1alpha1-CredentialProviderConfig)
## `CredentialProviderConfig` {#kubelet-config-k8s-io-v1alpha1-CredentialProviderConfig}

545
content/en/docs/reference/config-api/kubelet-config.v1beta1.md
View File

@ -14,6 +14,279 @@ auto_generated: true
- [SerializedNodeConfigSource](#kubelet-config-k8s-io-v1beta1-SerializedNodeConfigSource)
## `FormatOptions` {#FormatOptions}
**Appears in:**
- [LoggingConfiguration](#LoggingConfiguration)
<p>FormatOptions contains options for the different logging formats.</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>json</code> <B>[Required]</B><br/>
<a href="#JSONOptions"><code>JSONOptions</code></a>
</td>
<td>
<p>[Alpha] JSON contains options for logging format &quot;json&quot;.
Only available when the LoggingAlphaOptions feature gate is enabled.</p>
</td>
</tr>
</tbody>
</table>
## `JSONOptions` {#JSONOptions}
**Appears in:**
- [FormatOptions](#FormatOptions)
<p>JSONOptions contains options for logging format &quot;json&quot;.</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>splitStream</code> <B>[Required]</B><br/>
<code>bool</code>
</td>
<td>
<p>[Alpha] SplitStream redirects error messages to stderr while
info messages go to stdout, with buffering. The default is to write
both to stdout, without buffering. Only available when
the LoggingAlphaOptions feature gate is enabled.</p>
</td>
</tr>
<tr><td><code>infoBufferSize</code> <B>[Required]</B><br/>
<a href="https://pkg.go.dev/k8s.io/apimachinery/pkg/api/resource#QuantityValue"><code>k8s.io/apimachinery/pkg/api/resource.QuantityValue</code></a>
</td>
<td>
<p>[Alpha] InfoBufferSize sets the size of the info stream when
using split streams. The default is zero, which disables buffering.
Only available when the LoggingAlphaOptions feature gate is enabled.</p>
</td>
</tr>
</tbody>
</table>
## `LogFormatFactory` {#LogFormatFactory}
<p>LogFormatFactory provides support for a certain additional,
non-default log format.</p>
## `LoggingConfiguration` {#LoggingConfiguration}
**Appears in:**
- [KubeletConfiguration](#kubelet-config-k8s-io-v1beta1-KubeletConfiguration)
<p>LoggingConfiguration contains logging options.</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>format</code> <B>[Required]</B><br/>
<code>string</code>
</td>
<td>
<p>Format Flag specifies the structure of log messages.
default value of format is <code>text</code></p>
</td>
</tr>
<tr><td><code>flushFrequency</code> <B>[Required]</B><br/>
<a href="#TimeOrMetaDuration"><code>TimeOrMetaDuration</code></a>
</td>
<td>
<p>Maximum time between log flushes.
If a string, parsed as a duration (i.e. &quot;1s&quot;)
If an int, the maximum number of nanoseconds (i.e. 1s = 1000000000).
Ignored if the selected logging backend writes log messages without buffering.</p>
</td>
</tr>
<tr><td><code>verbosity</code> <B>[Required]</B><br/>
<a href="#VerbosityLevel"><code>VerbosityLevel</code></a>
</td>
<td>
<p>Verbosity is the threshold that determines which log messages are
logged. Default is zero which logs only the most important
messages. Higher values enable additional messages. Error messages
are always logged.</p>
</td>
</tr>
<tr><td><code>vmodule</code> <B>[Required]</B><br/>
<a href="#VModuleConfiguration"><code>VModuleConfiguration</code></a>
</td>
<td>
<p>VModule overrides the verbosity threshold for individual files.
Only supported for &quot;text&quot; log format.</p>
</td>
</tr>
<tr><td><code>options</code> <B>[Required]</B><br/>
<a href="#FormatOptions"><code>FormatOptions</code></a>
</td>
<td>
<p>[Alpha] Options holds additional parameters that are specific
to the different logging formats. Only the options for the selected
format get used, but all of them get validated.
Only available when the LoggingAlphaOptions feature gate is enabled.</p>
</td>
</tr>
</tbody>
</table>
## `LoggingOptions` {#LoggingOptions}
<p>LoggingOptions can be used with ValidateAndApplyWithOptions to override
certain global defaults.</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>ErrorStream</code> <B>[Required]</B><br/>
<a href="https://pkg.go.dev/io#Writer"><code>io.Writer</code></a>
</td>
<td>
<p>ErrorStream can be used to override the os.Stderr default.</p>
</td>
</tr>
<tr><td><code>InfoStream</code> <B>[Required]</B><br/>
<a href="https://pkg.go.dev/io#Writer"><code>io.Writer</code></a>
</td>
<td>
<p>InfoStream can be used to override the os.Stdout default.</p>
</td>
</tr>
</tbody>
</table>
## `TimeOrMetaDuration` {#TimeOrMetaDuration}
**Appears in:**
- [LoggingConfiguration](#LoggingConfiguration)
<p>TimeOrMetaDuration is present only for backwards compatibility for the
flushFrequency field, and new fields should use metav1.Duration.</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>Duration</code> <B>[Required]</B><br/>
<a href="https://pkg.go.dev/k8s.io/apimachinery/pkg/apis/meta/v1#Duration"><code>meta/v1.Duration</code></a>
</td>
<td>
<p>Duration holds the duration</p>
</td>
</tr>
<tr><td><code>-</code> <B>[Required]</B><br/>
<code>bool</code>
</td>
<td>
<p>SerializeAsString controls whether the value is serialized as a string or an integer</p>
</td>
</tr>
</tbody>
</table>
## `TracingConfiguration` {#TracingConfiguration}
**Appears in:**
- [KubeletConfiguration](#kubelet-config-k8s-io-v1beta1-KubeletConfiguration)
<p>TracingConfiguration provides versioned configuration for OpenTelemetry tracing clients.</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>endpoint</code><br/>
<code>string</code>
</td>
<td>
<p>Endpoint of the collector this component will report traces to.
The connection is insecure, and does not currently support TLS.
Recommended is unset, and endpoint is the otlp grpc default, localhost:4317.</p>
</td>
</tr>
<tr><td><code>samplingRatePerMillion</code><br/>
<code>int32</code>
</td>
<td>
<p>SamplingRatePerMillion is the number of samples to collect per million spans.
Recommended is unset. If unset, sampler respects its parent span's sampling
rate, but otherwise never samples.</p>
</td>
</tr>
</tbody>
</table>
## `VModuleConfiguration` {#VModuleConfiguration}
(Alias of `[]k8s.io/component-base/logs/api/v1.VModuleItem`)
**Appears in:**
- [LoggingConfiguration](#LoggingConfiguration)
<p>VModuleConfiguration is a collection of individual file names or patterns
and the corresponding verbosity threshold.</p>
## `VerbosityLevel` {#VerbosityLevel}
(Alias of `uint32`)
**Appears in:**
- [LoggingConfiguration](#LoggingConfiguration)
<p>VerbosityLevel represents a klog or logr verbosity threshold.</p>
## `CredentialProviderConfig` {#kubelet-config-k8s-io-v1beta1-CredentialProviderConfig}
@ -1698,275 +1971,3 @@ managers (secret, configmap) are discovering object changes.</p>
</tbody>
</table>
## `FormatOptions` {#FormatOptions}
**Appears in:**
- [LoggingConfiguration](#LoggingConfiguration)
<p>FormatOptions contains options for the different logging formats.</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>json</code> <B>[Required]</B><br/>
<a href="#JSONOptions"><code>JSONOptions</code></a>
</td>
<td>
<p>[Alpha] JSON contains options for logging format &quot;json&quot;.
Only available when the LoggingAlphaOptions feature gate is enabled.</p>
</td>
</tr>
</tbody>
</table>
## `JSONOptions` {#JSONOptions}
**Appears in:**
- [FormatOptions](#FormatOptions)
<p>JSONOptions contains options for logging format &quot;json&quot;.</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>splitStream</code> <B>[Required]</B><br/>
<code>bool</code>
</td>
<td>
<p>[Alpha] SplitStream redirects error messages to stderr while
info messages go to stdout, with buffering. The default is to write
both to stdout, without buffering. Only available when
the LoggingAlphaOptions feature gate is enabled.</p>
</td>
</tr>
<tr><td><code>infoBufferSize</code> <B>[Required]</B><br/>
<a href="https://pkg.go.dev/k8s.io/apimachinery/pkg/api/resource#QuantityValue"><code>k8s.io/apimachinery/pkg/api/resource.QuantityValue</code></a>
</td>
<td>
<p>[Alpha] InfoBufferSize sets the size of the info stream when
using split streams. The default is zero, which disables buffering.
Only available when the LoggingAlphaOptions feature gate is enabled.</p>
</td>
</tr>
</tbody>
</table>
## `LogFormatFactory` {#LogFormatFactory}
<p>LogFormatFactory provides support for a certain additional,
non-default log format.</p>
## `LoggingConfiguration` {#LoggingConfiguration}
**Appears in:**
- [KubeletConfiguration](#kubelet-config-k8s-io-v1beta1-KubeletConfiguration)
<p>LoggingConfiguration contains logging options.</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>format</code> <B>[Required]</B><br/>
<code>string</code>
</td>
<td>
<p>Format Flag specifies the structure of log messages.
default value of format is <code>text</code></p>
</td>
</tr>
<tr><td><code>flushFrequency</code> <B>[Required]</B><br/>
<a href="#TimeOrMetaDuration"><code>TimeOrMetaDuration</code></a>
</td>
<td>
<p>Maximum time between log flushes.
If a string, parsed as a duration (i.e. &quot;1s&quot;)
If an int, the maximum number of nanoseconds (i.e. 1s = 1000000000).
Ignored if the selected logging backend writes log messages without buffering.</p>
</td>
</tr>
<tr><td><code>verbosity</code> <B>[Required]</B><br/>
<a href="#VerbosityLevel"><code>VerbosityLevel</code></a>
</td>
<td>
<p>Verbosity is the threshold that determines which log messages are
logged. Default is zero which logs only the most important
messages. Higher values enable additional messages. Error messages
are always logged.</p>
</td>
</tr>
<tr><td><code>vmodule</code> <B>[Required]</B><br/>
<a href="#VModuleConfiguration"><code>VModuleConfiguration</code></a>
</td>
<td>
<p>VModule overrides the verbosity threshold for individual files.
Only supported for &quot;text&quot; log format.</p>
</td>
</tr>
<tr><td><code>options</code> <B>[Required]</B><br/>
<a href="#FormatOptions"><code>FormatOptions</code></a>
</td>
<td>
<p>[Alpha] Options holds additional parameters that are specific
to the different logging formats. Only the options for the selected
format get used, but all of them get validated.
Only available when the LoggingAlphaOptions feature gate is enabled.</p>
</td>
</tr>
</tbody>
</table>
## `LoggingOptions` {#LoggingOptions}
<p>LoggingOptions can be used with ValidateAndApplyWithOptions to override
certain global defaults.</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>ErrorStream</code> <B>[Required]</B><br/>
<a href="https://pkg.go.dev/io#Writer"><code>io.Writer</code></a>
</td>
<td>
<p>ErrorStream can be used to override the os.Stderr default.</p>
</td>
</tr>
<tr><td><code>InfoStream</code> <B>[Required]</B><br/>
<a href="https://pkg.go.dev/io#Writer"><code>io.Writer</code></a>
</td>
<td>
<p>InfoStream can be used to override the os.Stdout default.</p>
</td>
</tr>
</tbody>
</table>
## `TimeOrMetaDuration` {#TimeOrMetaDuration}
**Appears in:**
- [LoggingConfiguration](#LoggingConfiguration)
<p>TimeOrMetaDuration is present only for backwards compatibility for the
flushFrequency field, and new fields should use metav1.Duration.</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>Duration</code> <B>[Required]</B><br/>
<a href="https://pkg.go.dev/k8s.io/apimachinery/pkg/apis/meta/v1#Duration"><code>meta/v1.Duration</code></a>
</td>
<td>
<p>Duration holds the duration</p>
</td>
</tr>
<tr><td><code>-</code> <B>[Required]</B><br/>
<code>bool</code>
</td>
<td>
<p>SerializeAsString controls whether the value is serialized as a string or an integer</p>
</td>
</tr>
</tbody>
</table>
## `TracingConfiguration` {#TracingConfiguration}
**Appears in:**
- [KubeletConfiguration](#kubelet-config-k8s-io-v1beta1-KubeletConfiguration)
<p>TracingConfiguration provides versioned configuration for OpenTelemetry tracing clients.</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>endpoint</code><br/>
<code>string</code>
</td>
<td>
<p>Endpoint of the collector this component will report traces to.
The connection is insecure, and does not currently support TLS.
Recommended is unset, and endpoint is the otlp grpc default, localhost:4317.</p>
</td>
</tr>
<tr><td><code>samplingRatePerMillion</code><br/>
<code>int32</code>
</td>
<td>
<p>SamplingRatePerMillion is the number of samples to collect per million spans.
Recommended is unset. If unset, sampler respects its parent span's sampling
rate, but otherwise never samples.</p>
</td>
</tr>
</tbody>
</table>
## `VModuleConfiguration` {#VModuleConfiguration}
(Alias of `[]k8s.io/component-base/logs/api/v1.VModuleItem`)
**Appears in:**
- [LoggingConfiguration](#LoggingConfiguration)
<p>VModuleConfiguration is a collection of individual file names or patterns
and the corresponding verbosity threshold.</p>
## `VerbosityLevel` {#VerbosityLevel}
(Alias of `uint32`)
**Appears in:**
- [LoggingConfiguration](#LoggingConfiguration)
<p>VerbosityLevel represents a klog or logr verbosity threshold.</p>

1
content/en/docs/reference/config-api/kubelet-credentialprovider.v1.md
View File

@ -12,7 +12,6 @@ auto_generated: true
- [CredentialProviderRequest](#credentialprovider-kubelet-k8s-io-v1-CredentialProviderRequest)
- [CredentialProviderResponse](#credentialprovider-kubelet-k8s-io-v1-CredentialProviderResponse)
## `CredentialProviderRequest` {#credentialprovider-kubelet-k8s-io-v1-CredentialProviderRequest}

1
content/en/docs/reference/config-api/kubelet-credentialprovider.v1alpha1.md
View File

@ -12,7 +12,6 @@ auto_generated: true
- [CredentialProviderRequest](#credentialprovider-kubelet-k8s-io-v1alpha1-CredentialProviderRequest)
- [CredentialProviderResponse](#credentialprovider-kubelet-k8s-io-v1alpha1-CredentialProviderResponse)
## `CredentialProviderRequest` {#credentialprovider-kubelet-k8s-io-v1alpha1-CredentialProviderRequest}

3
content/en/docs/reference/config-api/kubelet-credentialprovider.v1beta1.md
View File

@ -12,7 +12,6 @@ auto_generated: true
- [CredentialProviderRequest](#credentialprovider-kubelet-k8s-io-v1beta1-CredentialProviderRequest)
- [CredentialProviderResponse](#credentialprovider-kubelet-k8s-io-v1beta1-CredentialProviderResponse)
## `CredentialProviderRequest` {#credentialprovider-kubelet-k8s-io-v1beta1-CredentialProviderRequest}
@ -110,7 +109,7 @@ stopping after the first successfully authenticated pull.</p>
<li>123456789.dkr.ecr.us-east-1.amazonaws.com</li>
<li>&ast;.azurecr.io</li>
<li>gcr.io</li>
<li>&ast;.&ast;registry.io</li>
<li>&ast;.&ast;.registry.io</li>
<li>registry.io:8080/path</li>
</ul>
</td>

2
content/en/docs/reference/glossary/container-runtime-interface.md
View File

@ -17,6 +17,6 @@ The main protocol for the communication between the {{< glossary_tooltip text="k
The Kubernetes Container Runtime Interface (CRI) defines the main
[gRPC](https://grpc.io) protocol for the communication between the
[cluster components](/docs/concepts/overview/components/#node-components)
[node components](/docs/concepts/overview/components/#node-components)
{{< glossary_tooltip text="kubelet" term_id="kubelet" >}} and
{{< glossary_tooltip text="container runtime" term_id="container-runtime" >}}.

18
content/en/docs/reference/glossary/group-version-resource.md Normal file
View File

@ -0,0 +1,18 @@
---
title: Group Version Resource
id: gvr
date: 2023-07-24
short_description: >
The API group, API version and name of a Kubernetes API.
aka: ["GVR"]
tags:
- architecture
---
Means of representing unique Kubernetes API resource.
<!--more-->
Group Version Resources (GVRs) specify the API group, API version, and resource (name for the object kind as it appears in the URI) associated with accessing a particular id of object in Kubernetes.
GVRs let you define and distinguish different Kubernetes objects, and to specify a way of accessing
objects that is stable even as APIs change.

33
content/en/docs/reference/issues-security/security.md
View File

@ -13,21 +13,27 @@ weight: 20
<!-- overview -->
This page describes Kubernetes security and disclosure information.
<!-- body -->
## Security Announcements
Join the [kubernetes-security-announce](https://groups.google.com/forum/#!forum/kubernetes-security-announce) group for emails about security and major API announcements.
Join the [kubernetes-security-announce](https://groups.google.com/forum/#!forum/kubernetes-security-announce)
group for emails about security and major API announcements.
## Report a Vulnerability
We're extremely grateful for security researchers and users that report vulnerabilities to the Kubernetes Open Source Community. All reports are thoroughly investigated by a set of community volunteers.
We're extremely grateful for security researchers and users that report vulnerabilities to
the Kubernetes Open Source Community. All reports are thoroughly investigated by a set of community volunteers.
To make a report, submit your vulnerability to the [Kubernetes bug bounty program](https://hackerone.com/kubernetes). This allows triage and handling of the vulnerability with standardized response times.
To make a report, submit your vulnerability to the [Kubernetes bug bounty program](https://hackerone.com/kubernetes).
This allows triage and handling of the vulnerability with standardized response times.
You can also email the private [security@kubernetes.io](mailto:security@kubernetes.io) list with the security details and the details expected for [all Kubernetes bug reports](https://github.com/kubernetes/kubernetes/blob/master/.github/ISSUE_TEMPLATE/bug-report.yaml).
You can also email the private [security@kubernetes.io](mailto:security@kubernetes.io)
list with the security details and the details expected for
[all Kubernetes bug reports](https://github.com/kubernetes/kubernetes/blob/master/.github/ISSUE_TEMPLATE/bug-report.yaml).
You may encrypt your email to this list using the GPG keys of the [Security Response Committee members](https://git.k8s.io/security/README.md#product-security-committee-psc). Encryption using GPG is NOT required to make a disclosure.
You may encrypt your email to this list using the GPG keys of the
[Security Response Committee members](https://git.k8s.io/security/README.md#product-security-committee-psc).
Encryption using GPG is NOT required to make a disclosure.
### When Should I Report a Vulnerability?
@ -36,7 +42,6 @@ You may encrypt your email to this list using the GPG keys of the [Security Resp
- You think you discovered a vulnerability in another project that Kubernetes depends on
- For projects with their own vulnerability reporting and disclosure process, please report it directly there
### When Should I NOT Report a Vulnerability?
- You need help tuning Kubernetes components for security
@ -45,13 +50,19 @@ You may encrypt your email to this list using the GPG keys of the [Security Resp
## Security Vulnerability Response
Each report is acknowledged and analyzed by Security Response Committee members within 3 working days. This will set off the [Security Release Process](https://git.k8s.io/security/security-release-process.md#disclosures).
Each report is acknowledged and analyzed by Security Response Committee members within 3 working days.
This will set off the [Security Release Process](https://git.k8s.io/security/security-release-process.md#disclosures).
Any vulnerability information shared with Security Response Committee stays within Kubernetes project and will not be disseminated to other projects unless it is necessary to get the issue fixed.
Any vulnerability information shared with Security Response Committee stays within Kubernetes project
and will not be disseminated to other projects unless it is necessary to get the issue fixed.
As the security issue moves from triage, to identified fix, to release planning we will keep the reporter updated.
## Public Disclosure Timing
A public disclosure date is negotiated by the Kubernetes Security Response Committee and the bug submitter. We prefer to fully disclose the bug as soon as possible once a user mitigation is available. It is reasonable to delay disclosure when the bug or the fix is not yet fully understood, the solution is not well-tested, or for vendor coordination. The timeframe for disclosure is from immediate (especially if it's already publicly known) to a few weeks. For a vulnerability with a straightforward mitigation, we expect report date to disclosure date to be on the order of 7 days. The Kubernetes Security Response Committee holds the final say when setting a disclosure date.
A public disclosure date is negotiated by the Kubernetes Security Response Committee and the bug submitter.
We prefer to fully disclose the bug as soon as possible once a user mitigation is available. It is reasonable
to delay disclosure when the bug or the fix is not yet fully understood, the solution is not well-tested,
or for vendor coordination. The timeframe for disclosure is from immediate (especially if it's already publicly known)
to a few weeks. For a vulnerability with a straightforward mitigation, we expect report date to disclosure date
to be on the order of 7 days. The Kubernetes Security Response Committee holds the final say when setting a disclosure date.

73
content/en/docs/reference/kubectl/_index.md
View File

@ -25,7 +25,8 @@ For details about each command, including all the supported flags and subcommand
For installation instructions, see [Installing kubectl](/docs/tasks/tools/#kubectl);
for a quick guide, see the [cheat sheet](/docs/reference/kubectl/cheatsheet/).
If you're used to using the `docker` command-line tool, [`kubectl` for Docker Users](/docs/reference/kubectl/docker-cli-to-kubectl/) explains some equivalent commands for Kubernetes.
If you're used to using the `docker` command-line tool,
[`kubectl` for Docker Users](/docs/reference/kubectl/docker-cli-to-kubectl/) explains some equivalent commands for Kubernetes.
<!-- body -->
@ -39,37 +40,41 @@ kubectl [command] [TYPE] [NAME] [flags]
where `command`, `TYPE`, `NAME`, and `flags` are:
* `command`: Specifies the operation that you want to perform on one or more resources,
for example `create`, `get`, `describe`, `delete`.
* `command`: Specifies the operation that you want to perform on one or more resources,
for example `create`, `get`, `describe`, `delete`.
* `TYPE`: Specifies the [resource type](#resource-types). Resource types are case-insensitive and
you can specify the singular, plural, or abbreviated forms.
For example, the following commands produce the same output:
```shell
kubectl get pod pod1
kubectl get pods pod1
kubectl get po pod1
```
```shell
kubectl get pod pod1
kubectl get pods pod1
kubectl get po pod1
```
* `NAME`: Specifies the name of the resource. Names are case-sensitive. If the name is omitted, details for all resources are displayed, for example `kubectl get pods`.
* `NAME`: Specifies the name of the resource. Names are case-sensitive. If the name is omitted,
details for all resources are displayed, for example `kubectl get pods`.
When performing an operation on multiple resources, you can specify each resource by type and name or specify one or more files:
When performing an operation on multiple resources, you can specify each resource by
type and name or specify one or more files:
* To specify resources by type and name:
* To specify resources by type and name:
* To group resources if they are all the same type: `TYPE1 name1 name2 name<#>`.<br/>
* To group resources if they are all the same type: `TYPE1 name1 name2 name<#>`.<br/>
Example: `kubectl get pod example-pod1 example-pod2`
* To specify multiple resource types individually: `TYPE1/name1 TYPE1/name2 TYPE2/name3 TYPE<#>/name<#>`.<br/>
* To specify multiple resource types individually: `TYPE1/name1 TYPE1/name2 TYPE2/name3 TYPE<#>/name<#>`.<br/>
Example: `kubectl get pod/example-pod1 replicationcontroller/example-rc1`
* To specify resources with one or more files: `-f file1 -f file2 -f file<#>`
* To specify resources with one or more files: `-f file1 -f file2 -f file<#>`
* [Use YAML rather than JSON](/docs/concepts/configuration/overview/#general-configuration-tips) since YAML tends to be more user-friendly, especially for configuration files.<br/>
Example: `kubectl get -f ./pod.yaml`
* [Use YAML rather than JSON](/docs/concepts/configuration/overview/#general-configuration-tips)
since YAML tends to be more user-friendly, especially for configuration files.<br/>
Example: `kubectl get -f ./pod.yaml`
* `flags`: Specifies optional flags. For example, you can use the `-s` or `--server` flags to specify the address and port of the Kubernetes API server.<br/>
* `flags`: Specifies optional flags. For example, you can use the `-s` or `--server` flags
to specify the address and port of the Kubernetes API server.<br/>
{{< caution >}}
Flags that you specify from the command line override default values and any corresponding environment variables.
@ -79,19 +84,29 @@ If you need help, run `kubectl help` from the terminal window.
## In-cluster authentication and namespace overrides
By default `kubectl` will first determine if it is running within a pod, and thus in a cluster. It starts by checking for the `KUBERNETES_SERVICE_HOST` and `KUBERNETES_SERVICE_PORT` environment variables and the existence of a service account token file at `/var/run/secrets/kubernetes.io/serviceaccount/token`. If all three are found in-cluster authentication is assumed.
By default `kubectl` will first determine if it is running within a pod, and thus in a cluster.
It starts by checking for the `KUBERNETES_SERVICE_HOST` and `KUBERNETES_SERVICE_PORT` environment
variables and the existence of a service account token file at `/var/run/secrets/kubernetes.io/serviceaccount/token`.
If all three are found in-cluster authentication is assumed.
To maintain backwards compatibility, if the `POD_NAMESPACE` environment variable is set during in-cluster authentication it will override the default namespace from the service account token. Any manifests or tools relying on namespace defaulting will be affected by this.
To maintain backwards compatibility, if the `POD_NAMESPACE` environment variable is set
during in-cluster authentication it will override the default namespace from the
service account token. Any manifests or tools relying on namespace defaulting will be affected by this.
**`POD_NAMESPACE` environment variable**
If the `POD_NAMESPACE` environment variable is set, cli operations on namespaced resources will default to the variable value. For example, if the variable is set to `seattle`, `kubectl get pods` would return pods in the `seattle` namespace. This is because pods are a namespaced resource, and no namespace was provided in the command. Review the output of `kubectl api-resources` to determine if a resource is namespaced.
If the `POD_NAMESPACE` environment variable is set, cli operations on namespaced resources
will default to the variable value. For example, if the variable is set to `seattle`,
`kubectl get pods` would return pods in the `seattle` namespace. This is because pods are
a namespaced resource, and no namespace was provided in the command. Review the output
of `kubectl api-resources` to determine if a resource is namespaced.
Explicit use of `--namespace <value>` overrides this behavior.
Explicit use of `--namespace <value>` overrides this behavior.
**How kubectl handles ServiceAccount tokens**
If:
* there is Kubernetes service account token file mounted at
`/var/run/secrets/kubernetes.io/serviceaccount/token`, and
* the `KUBERNETES_SERVICE_HOST` environment variable is set, and
@ -230,11 +245,15 @@ The following table includes a list of all the supported resource types and thei
## Output options
Use the following sections for information about how you can format or sort the output of certain commands. For details about which commands support the various output options, see the [kubectl](/docs/reference/kubectl/kubectl/) reference documentation.
Use the following sections for information about how you can format or sort the output
of certain commands. For details about which commands support the various output options,
see the [kubectl](/docs/reference/kubectl/kubectl/) reference documentation.
### Formatting output
The default output format for all `kubectl` commands is the human readable plain-text format. To output details to your terminal window in a specific format, you can add either the `-o` or `--output` flags to a supported `kubectl` command.
The default output format for all `kubectl` commands is the human readable plain-text format.
To output details to your terminal window in a specific format, you can add either the `-o`
or `--output` flags to a supported `kubectl` command.
#### Syntax
@ -324,7 +343,9 @@ pod-name 1m
### Sorting list objects
To output objects to a sorted list in your terminal window, you can add the `--sort-by` flag to a supported `kubectl` command. Sort your objects by specifying any numeric or string field with the `--sort-by` flag. To specify a field, use a [jsonpath](/docs/reference/kubectl/jsonpath/) expression.
To output objects to a sorted list in your terminal window, you can add the `--sort-by` flag
to a supported `kubectl` command. Sort your objects by specifying any numeric or string field
with the `--sort-by` flag. To specify a field, use a [jsonpath](/docs/reference/kubectl/jsonpath/) expression.
#### Syntax
@ -508,10 +529,12 @@ The following kubectl-compatible plugins are available:
`kubectl plugin list` also warns you about plugins that are not
executable, or that are shadowed by other plugins; for example:
```shell
sudo chmod -x /usr/local/bin/kubectl-foo # remove execute permission
kubectl plugin list
```
```
The following kubectl-compatible plugins are available:
@ -529,8 +552,10 @@ of the existing kubectl commands:
```shell
cat ./kubectl-whoami
```
The next few examples assume that you already made `kubectl-whoami` have
the following contents:
```shell
#!/bin/bash

5
content/en/docs/reference/kubectl/cheatsheet.md
View File

@ -213,7 +213,7 @@ kubectl get pods --field-selector=status.phase=Running
kubectl get nodes -o jsonpath='{.items[*].status.addresses[?(@.type=="ExternalIP")].address}'
# List Names of Pods that belong to Particular RC
# "jq" command useful for transformations that are too complex for jsonpath, it can be found at https://stedolan.github.io/jq/
# "jq" command useful for transformations that are too complex for jsonpath, it can be found at https://jqlang.github.io/jq/
sel=${$(kubectl get rc my-rc --output=json | jq -j '.spec.selector | to_entries | .[] | "\(.key)=\(.value),"')%?}
echo $(kubectl get pods --selector=$sel --output=jsonpath={.items..metadata.name})
@ -224,6 +224,9 @@ kubectl get pods --show-labels
JSONPATH='{range .items[*]}{@.metadata.name}:{range @.status.conditions[*]}{@.type}={@.status};{end}{end}' \
&& kubectl get nodes -o jsonpath="$JSONPATH" | grep "Ready=True"
# Check which nodes are ready with custom-columns
kubectl get node -o custom-columns='NODE_NAME:.metadata.name,STATUS:.status.conditions[?(@.type=="Ready")].status'
# Output decoded secrets without external tools
kubectl get secret my-secret -o go-template='{{range $k,$v := .data}}{{"### "}}{{$k}}{{"\n"}}{{$v|base64decode}}{{"\n\n"}}{{end}}'

33
content/en/docs/reference/kubectl/jsonpath.md
View File

@ -34,7 +34,12 @@ Given the JSON input:
"items":[
{
"kind":"None",
"metadata":{"name":"127.0.0.1"},
"metadata":{
"name":"127.0.0.1",
"labels":{
"kubernetes.io/hostname":"127.0.0.1"
}
},
"status":{
"capacity":{"cpu":"4"},
"addresses":[{"type": "LegacyHostIP", "address":"127.0.0.1"}]
@ -65,18 +70,19 @@ Given the JSON input:
}
```
Function | Description | Example | Result
--------------------|---------------------------|-----------------------------------------------------------------|------------------
`text` | the plain text | `kind is {.kind}` | `kind is List`
`@` | the current object | `{@}` | the same as input
`.` or `[]` | child operator | `{.kind}`, `{['kind']}` or `{['name\.type']}` | `List`
`..` | recursive descent | `{..name}` | `127.0.0.1 127.0.0.2 myself e2e`
`*` | wildcard. Get all objects | `{.items[*].metadata.name}` | `[127.0.0.1 127.0.0.2]`
`[start:end:step]` | subscript operator | `{.users[0].name}` | `myself`
`[,]` | union operator | `{.items[*]['metadata.name', 'status.capacity']}` | `127.0.0.1 127.0.0.2 map[cpu:4] map[cpu:8]`
`?()` | filter | `{.users[?(@.name=="e2e")].user.password}` | `secret`
`range`, `end` | iterate list | `{range .items[*]}[{.metadata.name}, {.status.capacity}] {end}` | `[127.0.0.1, map[cpu:4]] [127.0.0.2, map[cpu:8]]`
`''` | quote interpreted string | `{range .items[*]}{.metadata.name}{'\t'}{end}` | `127.0.0.1 127.0.0.2`
Function | Description | Example | Result
--------------------|------------------------------|-----------------------------------------------------------------|------------------
`text` | the plain text | `kind is {.kind}` | `kind is List`
`@` | the current object | `{@}` | the same as input
`.` or `[]` | child operator | `{.kind}`, `{['kind']}` or `{['name\.type']}` | `List`
`..` | recursive descent | `{..name}` | `127.0.0.1 127.0.0.2 myself e2e`
`*` | wildcard. Get all objects | `{.items[*].metadata.name}` | `[127.0.0.1 127.0.0.2]`
`[start:end:step]` | subscript operator | `{.users[0].name}` | `myself`
`[,]` | union operator | `{.items[*]['metadata.name', 'status.capacity']}` | `127.0.0.1 127.0.0.2 map[cpu:4] map[cpu:8]`
`?()` | filter | `{.users[?(@.name=="e2e")].user.password}` | `secret`
`range`, `end` | iterate list | `{range .items[*]}[{.metadata.name}, {.status.capacity}] {end}` | `[127.0.0.1, map[cpu:4]] [127.0.0.2, map[cpu:8]]`
`''` | quote interpreted string | `{range .items[*]}{.metadata.name}{'\t'}{end}` | `127.0.0.1 127.0.0.2`
`\` | escape termination character | `{.items[0].metadata.labels.kubernetes\.io/hostname}` | `127.0.0.1`
Examples using `kubectl` and JSONPath expressions:
@ -87,6 +93,7 @@ kubectl get pods -o=jsonpath='{.items[0]}'
kubectl get pods -o=jsonpath='{.items[0].metadata.name}'
kubectl get pods -o=jsonpath="{.items[*]['metadata.name', 'status.capacity']}"
kubectl get pods -o=jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.status.startTime}{"\n"}{end}'
kubectl get pods -o=jsonpath='{.items[0].metadata.labels.kubernetes\.io/hostname}'
```
{{< note >}}

93
content/en/docs/reference/labels-annotations-taints/_index.md
View File

@ -299,6 +299,23 @@ This annotation is part of the Kubernetes Resource Model (KRM) Functions Specifi
which is used by Kustomize and similar third-party tools.
For example, Kustomize removes objects with this annotation from its final build output.
### container.apparmor.security.beta.kubernetes.io/* (beta) {#container-apparmor-security-beta-kubernetes-io}
Type: Annotation
Example: `container.apparmor.security.beta.kubernetes.io/my-container: my-custom-profile`
Used on: Pods
This annotation allows you to specify the AppArmor security profile for a container within a
Kubernetes pod.
To learn more, see the [AppArmor](/docs/tutorials/security/apparmor/) tutorial.
The tutorial illustrates using AppArmor to restrict a container's abilities and access.
The profile specified dictates the set of rules and restrictions that the containerized process must
adhere to. This helps enforce security policies and isolation for your containers.
### internal.config.kubernetes.io/* (reserved prefix) {#internal.config.kubernetes.io-reserved-wildcard}
Type: Annotation
@ -940,6 +957,22 @@ works in that release.
There are no other valid values for this annotation. If you don't want topology aware hints
for a Service, don't add this annotation.
### service.kubernetes.io/topology-mode
Type: Annotation
Example: `service.kubernetes.io/topology-mode: Auto`
Used on: Service
This annotation provides a way to define how Services handle network topology;
for example, you can configure a Service so that Kubernetes prefers keeping traffic between
a client and server within a single topology zone.
In some cases this can help reduce costs or improve network performance.
See [Topology Aware Routing](/docs/concepts/services-networking/topology-aware-routing/)
for more details.
### kubernetes.io/service-name {#kubernetesioservice-name}
Type: Label
@ -1176,6 +1209,27 @@ has been truncated to 1000.
If the number of backend endpoints falls below 1000, the control plane removes this annotation.
### control-plane.alpha.kubernetes.io/leader (deprecated) {#control-plane-alpha-kubernetes-io-leader}
Type: Annotation
Example: `control-plane.alpha.kubernetes.io/leader={"holderIdentity":"controller-0","leaseDurationSeconds":15,"acquireTime":"2023-01-19T13:12:57Z","renewTime":"2023-01-19T13:13:54Z","leaderTransitions":1}`
Used on: Endpoints
The {{< glossary_tooltip text="control plane" term_id="control-plane" >}} previously set annotation on
an [Endpoints](/docs/concepts/services-networking/service/#endpoints) object. This annotation provided
the following detail:
- Who is the current leader.
- The time when the current leadership was acquired.
- The duration of the lease (of the leadership) in seconds.
- The time the current lease (the current leadership) should be renewed.
- The number of leadership transitions that happened in the past.
Kubernetes now uses [Leases](/docs/concepts/architecture/leases/) to
manage leader assignment for the Kubernetes control plane.
### batch.kubernetes.io/job-tracking (deprecated) {#batch-kubernetes-io-job-tracking}
Type: Annotation
@ -1466,10 +1520,23 @@ This annotation records a comma-separated list of
managed by [Node Feature Discovery](https://kubernetes-sigs.github.io/node-feature-discovery/) (NFD).
NFD uses this for an internal mechanism. You should not edit this annotation yourself.
### nfd.node.kubernetes.io/node-name
Type: Label
Example: `nfd.node.kubernetes.io/node-name: node-1`
Used on: Nodes
It specifies which node the NodeFeature object is targeting.
Creators of NodeFeature objects must set this label and
consumers of the objects are supposed to use the label for
filtering features designated for a certain node.
{{< note >}}
These annotations only applies to nodes where NFD is running.
To learn more about NFD and its components go to its official
[documentation](https://kubernetes-sigs.github.io/node-feature-discovery/stable/get-started/).
These Node Feature Discovery (NFD) labels or annotations only apply to
the nodes where NFD is running. To learn more about NFD and
its components go to its official [documentation](https://kubernetes-sigs.github.io/node-feature-discovery/stable/get-started/).
{{< /note >}}
### service.beta.kubernetes.io/aws-load-balancer-access-log-emit-interval (beta) {#service-beta-kubernetes-io-aws-load-balancer-access-log-emit-interval}
@ -1790,6 +1857,26 @@ uses this annotation.
See [annotations](https://kubernetes-sigs.github.io/aws-load-balancer-controller/latest/guide/service/annotations/)
in the AWS load balancer controller documentation.
### service.beta.kubernetes.io/aws-load-balancer-security-groups (deprecated) {#service-beta-kubernetes-io-aws-load-balancer-security-groups}
Example: `service.beta.kubernetes.io/aws-load-balancer-security-groups: "sg-53fae93f,sg-8725gr62r"`
Used on: Service
The AWS load balancer controller uses this annotation to specify a comma seperated list
of security groups you want to attach to an AWS load balancer. Both name and ID of security
are supported where name matches a `Name` tag, not the `groupName` attribute.
When this annotation is added to a Service, the load-balancer controller attaches the security groups
referenced by the annotation to the load balancer. If you omit this annotation, the AWS load balancer
controller automatically creates a new security group and attaches it to the load balancer.
{{< note >}}
Kubernetes v1.27 and later do not directly set or read this annotation. However, the AWS
load balancer controller (part of the Kubernetes project) does still use the
`service.beta.kubernetes.io/aws-load-balancer-security-groups` annotation.
{{< /note >}}
### service.beta.kubernetes.io/load-balancer-source-ranges (deprecated) {#service-beta-kubernetes-io-load-balancer-source-ranges}
Example: `service.beta.kubernetes.io/load-balancer-source-ranges: "192.0.2.0/25"`

6
content/en/docs/reference/setup-tools/kubeadm/kubeadm-init.md
View File

@ -135,7 +135,7 @@ If your configuration is not using the latest version it is **recommended** that
the [kubeadm config migrate](/docs/reference/setup-tools/kubeadm/kubeadm-config/) command.
For more information on the fields and usage of the configuration you can navigate to our
[API reference page](/docs/reference/config-api/kubeadm-config.v1beta4/).
[API reference page](/docs/reference/config-api/kubeadm-config.v1beta3/).
### Using kubeadm init with feature gates {#feature-gates}
@ -145,7 +145,7 @@ of the cluster. Feature gates are removed after a feature graduates to GA.
To pass a feature gate you can either use the `--feature-gates` flag for
`kubeadm init`, or you can add items into the `featureGates` field when you pass
a [configuration file](/docs/reference/config-api/kubeadm-config.v1beta4/#kubeadm-k8s-io-v1beta4-ClusterConfiguration)
a [configuration file](/docs/reference/config-api/kubeadm-config.v1beta3/#kubeadm-k8s-io-v1beta3-ClusterConfiguration)
using `--config`.
Passing [feature gates for core Kubernetes components](/docs/reference/command-line-tools-reference/feature-gates)
@ -314,7 +314,7 @@ kubeadm init phase upload-certs --upload-certs --config=SOME_YAML_FILE
```
{{< note >}}
A predefined `certificateKey` can be provided in `InitConfiguration` when passing the
[configuration file](/docs/reference/config-api/kubeadm-config.v1beta4/) with `--config`.
[configuration file](/docs/reference/config-api/kubeadm-config.v1beta3/) with `--config`.
{{< /note >}}
If a predefined certificate key is not passed to `kubeadm init` and

32
content/en/docs/reference/using-api/api-concepts.md
View File

@ -34,7 +34,7 @@ API concepts:
* A *resource type* is the name used in the URL (`pods`, `namespaces`, `services`)
* All resource types have a concrete representation (their object schema) which is called a *kind*
* A list of instances of a resource is known as a *collection*
* A list of instances of a resource type is known as a *collection*
* A single instance of a resource type is called a *resource*, and also usually represents an *object*
* For some resource types, the API includes one or more *sub-resources*, which are represented as URI paths below the resource
@ -148,7 +148,7 @@ For example:
1. List all of the pods in a given namespace.
```console
```
GET /api/v1/namespaces/test/pods
---
200 OK
@ -204,7 +204,7 @@ to a given `resourceVersion` the client is requesting have already been sent. Th
document representing the `BOOKMARK` event is of the type requested by the request,
but only includes a `.metadata.resourceVersion` field. For example:
```console
```
GET /api/v1/namespaces/test/pods?watch=1&resourceVersion=10245&allowWatchBookmarks=true
---
200 OK
@ -262,7 +262,7 @@ is 10245 and there are two pods: `foo` and `bar`. Then sending the following req
_consistent read_ by setting empty resource version using `resourceVersion=`) could result
in the following sequence of events:
```console
```
GET /api/v1/namespaces/test/pods?watch=1&sendInitialEvents=true&allowWatchBookmarks=true&resourceVersion=&resourceVersionMatch=NotOlderThan
---
200 OK
@ -303,7 +303,7 @@ can be saved and the latency can be reduced.
To verify if `APIResponseCompression` is working, you can send a **get** or **list** request to the
API server with an `Accept-Encoding` header, and check the response size and headers. For example:
```console
```
GET /api/v1/pods
Accept-Encoding: gzip
---
@ -354,7 +354,7 @@ of 500 pods at a time, request those chunks as follows:
1. List all of the pods on a cluster, retrieving up to 500 pods each time.
```console
```
GET /api/v1/pods?limit=500
---
200 OK
@ -375,7 +375,7 @@ of 500 pods at a time, request those chunks as follows:
2. Continue the previous call, retrieving the next set of 500 pods.
```console
```
GET /api/v1/pods?limit=500&continue=ENCODED_CONTINUE_TOKEN
---
200 OK
@ -396,7 +396,7 @@ of 500 pods at a time, request those chunks as follows:
3. Continue the previous call, retrieving the last 253 pods.
```console
```
GET /api/v1/pods?limit=500&continue=ENCODED_CONTINUE_TOKEN_2
---
200 OK
@ -540,7 +540,7 @@ type.
For example, list all of the pods on a cluster in the Table format.
```console
```
GET /api/v1/pods
Accept: application/json;as=Table;g=meta.k8s.io;v=v1
---
@ -561,7 +561,7 @@ For API resource types that do not have a custom Table definition known to the c
plane, the API server returns a default Table response that consists of the resource's
`name` and `creationTimestamp` fields.
```console
```
GET /apis/crd.example.com/v1alpha1/namespaces/default/resources
---
200 OK
@ -596,7 +596,7 @@ uses the Table information and must work against all resource types, including
extensions, you should make requests that specify multiple content types in the
`Accept` header. For example:
```console
```
Accept: application/json;as=Table;g=meta.k8s.io;v=v1, application/json
```
@ -624,7 +624,7 @@ For example:
1. List all of the pods on a cluster in Protobuf format.
```console
```
GET /api/v1/pods
Accept: application/vnd.kubernetes.protobuf
---
@ -637,7 +637,7 @@ For example:
1. Create a pod by sending Protobuf encoded data to the server, but request a response
in JSON.
```console
```
POST /api/v1/namespaces/test/pods
Content-Type: application/vnd.kubernetes.protobuf
Accept: application/json
@ -662,7 +662,7 @@ As a client, if you might need to work with extension types you should specify m
content types in the request `Accept` header to support fallback to JSON.
For example:
```console
```
Accept: application/vnd.kubernetes.protobuf, application/json
```
@ -675,7 +675,7 @@ describes the encoding and type of the underlying object and then contains the o
The wrapper format is:
```console
```
A four byte magic number prefix:
Bytes 0-3: "k8s\x00" [0x6b, 0x38, 0x73, 0x00]
@ -893,7 +893,7 @@ effects on any request marked as dry runs.
Here is an example dry-run request that uses `?dryRun=All`:
```console
```
POST /api/v1/namespaces/test/pods?dryRun=All
Content-Type: application/json
Accept: application/json

4
content/en/docs/setup/production-environment/tools/kubeadm/high-availability.md
View File

@ -218,8 +218,10 @@ option. Your cluster requirements may need a different configuration.
kubeadm certs certificate-key
```
The certificate key is a hex encoded string that is an AES key of size 32 bytes.
{{< note >}}
The `kubeadm-certs` Secret and decryption key expire after two hours.
The `kubeadm-certs` Secret and the decryption key expire after two hours.
{{< /note >}}
{{< caution >}}

205
content/en/docs/setup/production-environment/tools/kubeadm/install-kubeadm.md
View File

@ -15,10 +15,10 @@ This page shows how to install the `kubeadm` toolbox.
For information on how to create a cluster with kubeadm once you have performed this installation process,
see the [Creating a cluster with kubeadm](/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/) page.
{{< doc-versions-list "installation guide" >}}
## {{% heading "prerequisites" %}}
* A compatible Linux host. The Kubernetes project provides generic instructions for Linux distributions
based on Debian and Red Hat, and those distributions without a package manager.
* 2 GB or more of RAM per machine (any less will leave little room for your apps).
@ -33,6 +33,14 @@ see the [Creating a cluster with kubeadm](/docs/setup/production-environment/too
will disable swapping temporarily. To make this change persistent across reboots, make sure swap is disabled in
config files like `/etc/fstab`, `systemd.swap`, depending how it was configured on your system.
{{< note >}}
The `kubeadm` installation is done via binaries that use dynamic linking and assumes that your target system provides `glibc`.
This is a reasonable assumption on many Linux distributions (including Debian, Ubuntu, Fedora, CentOS, etc.)
but it is not always the case with custom and lightweight distributions which don't include `glibc` by default, such as Alpine Linux.
The expectation is that the distribution either includes `glibc` or a [compatibility layer](https://wiki.alpinelinux.org/wiki/Running_glibc_programs)
that provides the expected symbols.
{{< /note >}}
<!-- steps -->
## Verify the MAC address and product_uuid are unique for every node {#verify-mac-address}
@ -51,6 +59,7 @@ If you have more than one network adapter, and your Kubernetes components are no
route, we recommend you add IP route(s) so Kubernetes cluster addresses go via the appropriate adapter.
## Check required ports
These [required ports](/docs/reference/networking/ports-and-protocols/)
need to be open in order for Kubernetes components to communicate with each other.
You can use tools like netcat to check if a port is open. For example:
@ -123,7 +132,7 @@ You will install these packages on all of your machines:
* `kubeadm`: the command to bootstrap the cluster.
* `kubelet`: the component that runs on all of the machines in your cluster
and does things like starting pods and containers.
and does things like starting pods and containers.
* `kubectl`: the command line util to talk to your cluster.
@ -148,30 +157,17 @@ For more information on version skews, see:
* Kubernetes [version and version-skew policy](/docs/setup/release/version-skew-policy/)
* Kubeadm-specific [version skew policy](/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/#version-skew-policy)
{{% legacy-repos-deprecation %}}
{{< note >}}
Kubernetes has two different package repositories starting from August 2023.
The Google-hosted repository is deprecated and it's being replaced with the
Kubernetes (community-owned) package repositories. The Kubernetes project strongly
recommends using the Kubernetes community-owned package repositories, because the
project plans to stop publishing packages to the Google-hosted repository in the future.
There are some important considerations for the Kubernetes package repositories:
- The Kubernetes package repositories contain packages beginning with those
Kubernetes versions that were still under support when the community took
over the package builds. This means that anything before v1.24.0 will only be
available in the Google-hosted repository.
- There's a dedicated package repository for each Kubernetes minor version.
When upgrading to a different minor release, you must bear in mind that
the package repository details also change.
There's a dedicated package repository for each Kubernetes minor version. If you want to install
a minor version other than {{< skew currentVersion >}}, please see the installation guide for
your desired minor version.
{{< /note >}}
{{< tabs name="k8s_install" >}}
{{% tab name="Debian-based distributions" %}}
### Kubernetes package repositories {#dpkg-k8s-package-repo}
These instructions are for Kubernetes {{< skew currentVersion >}}.
1. Update the `apt` package index and install packages needed to use the Kubernetes `apt` repository:
@ -179,16 +175,21 @@ These instructions are for Kubernetes {{< skew currentVersion >}}.
```shell
sudo apt-get update
# apt-transport-https may be a dummy package; if so, you can skip that package
sudo apt-get install -y apt-transport-https ca-certificates curl
sudo apt-get install -y apt-transport-https ca-certificates curl gpg
```
2. Download the public signing key for the Kubernetes package repositories. The same signing key is used for all repositories so you can disregard the version in the URL:
2. Download the public signing key for the Kubernetes package repositories.
The same signing key is used for all repositories so you can disregard the version in the URL:
```shell
curl -fsSL https://pkgs.k8s.io/core:/stable:/{{< param "version" >}}/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
```
3. Add the appropriate Kubernetes `apt` repository:
3. Add the appropriate Kubernetes `apt` repository. Please note that this repository have packages
only for Kubernetes {{< skew currentVersion >}}; for other Kubernetes minor versions, you need to
change the Kubernetes minor version in the URL to match your desired minor version
(you should also check that you are reading the documentation for the version of Kubernetes
that you plan to install).
```shell
# This overwrites any existing configuration in /etc/apt/sources.list.d/kubernetes.list
@ -208,127 +209,57 @@ In releases older than Debian 12 and Ubuntu 22.04, `/etc/apt/keyrings` does not
you can create it by running `sudo mkdir -m 755 /etc/apt/keyrings`
{{< /note >}}
### Google-hosted package repository (deprecated) {#dpkg-google-package-repo}
These instructions are for Kubernetes {{< skew currentVersion >}}.
1. Update the `apt` package index and install packages needed to use the Kubernetes `apt` repository:
```shell
sudo apt-get update
# apt-transport-https may be a dummy package; if so, you can skip that package
sudo apt-get install -y apt-transport-https ca-certificates curl
```
2. Download the Google Cloud public signing key:
```shell
curl -fsSL https://dl.k8s.io/apt/doc/apt-key.gpg | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-archive-keyring.gpg
```
3. Add the Google-hosted `apt` repository:
```shell
# This overwrites any existing configuration in /etc/apt/sources.list.d/kubernetes.list
echo "deb [signed-by=/etc/apt/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee /etc/apt/sources.list.d/kubernetes.list
```
4. Update the `apt` package index, install kubelet, kubeadm and kubectl, and pin their version:
```shell
sudo apt-get update
sudo apt-get install -y kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl
```
{{< note >}}
In releases older than Debian 12 and Ubuntu 22.04, `/etc/apt/keyrings` does not exist by default;
you can create it by running `sudo mkdir -m 755 /etc/apt/keyrings`
{{< /note >}}
{{% /tab %}}
{{% tab name="Red Hat-based distributions" %}}
1. Set SELinux to `permissive` mode:
```shell
# Set SELinux in permissive mode (effectively disabling it)
sudo setenforce 0
sudo sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config
```
These instructions are for Kubernetes {{< skew currentVersion >}}.
```shell
# Set SELinux in permissive mode (effectively disabling it)
sudo setenforce 0
sudo sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config
```
{{< caution >}}
- Setting SELinux in permissive mode by running `setenforce 0` and `sed ...`
effectively disables it. This is required to allow containers to access the host
filesystem; for example, some cluster network plugins require that. You have to
do this until SELinux support is improved in the kubelet.
effectively disables it. This is required to allow containers to access the host
filesystem; for example, some cluster network plugins require that. You have to
do this until SELinux support is improved in the kubelet.
- You can leave SELinux enabled if you know how to configure it but it may require
settings that are not supported by kubeadm.
settings that are not supported by kubeadm.
{{< /caution >}}
### Kubernetes package repositories {#rpm-k8s-package-repo}
These instructions are for Kubernetes {{< skew currentVersion >}}.
2. Add the Kubernetes `yum` repository. The `exclude` parameter in the
repository definition ensures that the packages related to Kubernetes are
not upgraded upon running `yum update` as there's a special procedure that
must be followed for upgrading Kubernetes.
must be followed for upgrading Kubernetes. Please note that this repository
have packages only for Kubernetes {{< skew currentVersion >}}; for other
Kubernetes minor versions, you need to change the Kubernetes minor version
in the URL to match your desired minor version (you should also check that
you are reading the documentation for the version of Kubernetes that you
plan to install).
```shell
# This overwrites any existing configuration in /etc/yum.repos.d/kubernetes.repo
cat <<EOF | sudo tee /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://pkgs.k8s.io/core:/stable:/{{< param "version" >}}/rpm/
enabled=1
gpgcheck=1
gpgkey=https://pkgs.k8s.io/core:/stable:/{{< param "version" >}}/rpm/repodata/repomd.xml.key
exclude=kubelet kubeadm kubectl cri-tools kubernetes-cni
EOF
```
```shell
# This overwrites any existing configuration in /etc/yum.repos.d/kubernetes.repo
cat <<EOF | sudo tee /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://pkgs.k8s.io/core:/stable:/{{< param "version" >}}/rpm/
enabled=1
gpgcheck=1
gpgkey=https://pkgs.k8s.io/core:/stable:/{{< param "version" >}}/rpm/repodata/repomd.xml.key
exclude=kubelet kubeadm kubectl cri-tools kubernetes-cni
EOF
```
3. Install kubelet, kubeadm and kubectl, and enable kubelet to ensure it's automatically started on startup:
```shell
sudo yum install -y kubelet kubeadm kubectl --disableexcludes=kubernetes
sudo systemctl enable --now kubelet
```
### Google-hosted package repository (deprecated) {#rpm-google-package-repo}
These instructions are for Kubernetes {{< skew currentVersion >}}.
2. Add the Google-hosted `yum` repository. The `exclude` parameter in the
repository definition ensures that the packages related to Kubernetes are
not upgraded upon running `yum update` as there's a special procedure that
must be followed for upgrading Kubernetes.
```shell
# This overwrites any existing configuration in /etc/yum.repos.d/kubernetes.repo
cat <<EOF | sudo tee /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-\$basearch
enabled=1
gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
exclude=kubelet kubeadm kubectl
EOF
```
3. Install kubelet, kubeadm and kubectl, and enable kubelet to ensure it's automatically started on startup:
```shell
sudo yum install -y kubelet kubeadm kubectl --disableexcludes=kubernetes
sudo systemctl enable --now kubelet
```
{{< note >}}
If the `baseurl` fails because your RPM-based distribution cannot interpret `$basearch`, replace `\$basearch` with your computer's architecture.
Type `uname -m` to see that value.
For example, the `baseurl` URL for `x86_64` could be: `https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64`.
{{< /note >}}
```shell
sudo yum install -y kubelet kubeadm kubectl --disableexcludes=kubernetes
sudo systemctl enable --now kubelet
```
{{% /tab %}}
{{% tab name="Without a package manager" %}}
@ -342,7 +273,7 @@ sudo mkdir -p "$DEST"
curl -L "https://github.com/containernetworking/plugins/releases/download/${CNI_PLUGINS_VERSION}/cni-plugins-linux-${ARCH}-${CNI_PLUGINS_VERSION}.tgz" | sudo tar -C "$DEST" -xz
```
Define the directory to download command files
Define the directory to download command files:
{{< note >}}
The `DOWNLOAD_DIR` variable must be set to a writable directory.
@ -354,7 +285,7 @@ DOWNLOAD_DIR="/usr/local/bin"
sudo mkdir -p "$DOWNLOAD_DIR"
```
Install crictl (required for kubeadm / Kubelet Container Runtime Interface (CRI))
Install crictl (required for kubeadm / Kubelet Container Runtime Interface (CRI)):
```bash
CRICTL_VERSION="v1.28.0"
@ -371,12 +302,17 @@ cd $DOWNLOAD_DIR
sudo curl -L --remote-name-all https://dl.k8s.io/release/${RELEASE}/bin/linux/${ARCH}/{kubeadm,kubelet}
sudo chmod +x {kubeadm,kubelet}
RELEASE_VERSION="v0.15.1"
curl -sSL "https://raw.githubusercontent.com/kubernetes/release/${RELEASE_VERSION}/cmd/kubepkg/templates/latest/deb/kubelet/lib/systemd/system/kubelet.service" | sed "s:/usr/bin:${DOWNLOAD_DIR}:g" | sudo tee /etc/systemd/system/kubelet.service
RELEASE_VERSION="v0.16.2"
curl -sSL "https://raw.githubusercontent.com/kubernetes/release/${RELEASE_VERSION}/cmd/krel/templates/latest/kubelet/kubelet.service" | sed "s:/usr/bin:${DOWNLOAD_DIR}:g" | sudo tee /etc/systemd/system/kubelet.service
sudo mkdir -p /etc/systemd/system/kubelet.service.d
curl -sSL "https://raw.githubusercontent.com/kubernetes/release/${RELEASE_VERSION}/cmd/kubepkg/templates/latest/deb/kubeadm/10-kubeadm.conf" | sed "s:/usr/bin:${DOWNLOAD_DIR}:g" | sudo tee /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
curl -sSL "https://raw.githubusercontent.com/kubernetes/release/${RELEASE_VERSION}/cmd/krel/templates/latest/kubeadm/10-kubeadm.conf" | sed "s:/usr/bin:${DOWNLOAD_DIR}:g" | sudo tee /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
```
{{< note >}}
Please refer to the note in the [Before you begin](#before-you-begin) section for Linux distributions
that do not include `glibc` by default.
{{< /note >}}
Install `kubectl` by following the instructions on [Install Tools page](/docs/tasks/tools/#kubectl).
Enable and start `kubelet`:
@ -388,12 +324,12 @@ systemctl enable --now kubelet
{{< note >}}
The Flatcar Container Linux distribution mounts the `/usr` directory as a read-only filesystem.
Before bootstrapping your cluster, you need to take additional steps to configure a writable directory.
See the [Kubeadm Troubleshooting guide](/docs/setup/production-environment/tools/kubeadm/troubleshooting-kubeadm/#usr-mounted-read-only/) to learn how to set up a writable directory.
See the [Kubeadm Troubleshooting guide](/docs/setup/production-environment/tools/kubeadm/troubleshooting-kubeadm/#usr-mounted-read-only/)
to learn how to set up a writable directory.
{{< /note >}}
{{% /tab %}}
{{< /tabs >}}
The kubelet is now restarting every few seconds, as it waits in a crashloop for
kubeadm to tell it what to do.
@ -411,7 +347,8 @@ See [Configuring a cgroup driver](/docs/tasks/administer-cluster/kubeadm/configu
## Troubleshooting
If you are running into difficulties with kubeadm, please consult our [troubleshooting docs](/docs/setup/production-environment/tools/kubeadm/troubleshooting-kubeadm/).
If you are running into difficulties with kubeadm, please consult our
[troubleshooting docs](/docs/setup/production-environment/tools/kubeadm/troubleshooting-kubeadm/).
## {{% heading "whatsnext" %}}

6
content/en/docs/setup/production-environment/tools/kubeadm/kubelet-integration.md
View File

@ -162,12 +162,10 @@ Kubeadm deletes the `/etc/kubernetes/bootstrap-kubelet.conf` file after completi
Note that the kubeadm CLI command never touches this drop-in file.
This configuration file installed by the `kubeadm`
[DEB](https://github.com/kubernetes/release/blob/master/cmd/kubepkg/templates/latest/deb/kubeadm/10-kubeadm.conf) or
[RPM package](https://github.com/kubernetes/release/blob/master/cmd/kubepkg/templates/latest/rpm/kubeadm/10-kubeadm.conf) is written to
[package](https://github.com/kubernetes/release/blob/cd53840/cmd/krel/templates/latest/kubeadm/10-kubeadm.conf) is written to
`/etc/systemd/system/kubelet.service.d/10-kubeadm.conf` and is used by systemd.
It augments the basic
[`kubelet.service` for RPM](https://github.com/kubernetes/release/blob/master/cmd/kubepkg/templates/latest/rpm/kubelet/kubelet.service) or
[`kubelet.service` for DEB](https://github.com/kubernetes/release/blob/master/cmd/kubepkg/templates/latest/deb/kubelet/lib/systemd/system/kubelet.service):
[`kubelet.service`](https://github.com/kubernetes/release/blob/cd53840/cmd/krel/templates/latest/kubelet/kubelet.service):
{{< note >}}
The contents below are just an example. If you don't want to use a package manager

4
content/en/docs/tasks/access-application-cluster/ingress-minikube.md
View File

@ -108,6 +108,10 @@ If you haven't already set up a cluster locally, run `minikube start` to create
http://172.17.0.15:31637
```
```shell
curl http://172.17.0.15:31637
```
The output is similar to:
```none

6
content/en/docs/tasks/access-application-cluster/list-all-running-container-images.md
View File

@ -23,7 +23,7 @@ of Containers for each.
- Fetch all Pods in all namespaces using `kubectl get pods --all-namespaces`
- Format the output to include only the list of Container image names
using `-o jsonpath={.items[*].spec.containers[*].image}`. This will recursively parse out the
using `-o jsonpath={.items[*].spec['initContainers', 'containers'][*].image}`. This will recursively parse out the
`image` field from the returned json.
- See the [jsonpath reference](/docs/reference/kubectl/jsonpath/)
for further information on how to use jsonpath.
@ -33,7 +33,7 @@ of Containers for each.
- Use `uniq` to aggregate image counts
```shell
kubectl get pods --all-namespaces -o jsonpath="{.items[*].spec.containers[*].image}" |\
kubectl get pods --all-namespaces -o jsonpath="{.items[*].spec['initContainers', 'containers'][*].image}" |\
tr -s '[[:space:]]' '\n' |\
sort |\
uniq -c
@ -42,7 +42,7 @@ The jsonpath is interpreted as follows:
- `.items[*]`: for each returned value
- `.spec`: get the spec
- `.containers[*]`: for each container
- `['initContainers', 'containers'][*]`: for each container
- `.image`: get the image
{{< note >}}

26
content/en/docs/tasks/administer-cluster/configure-upgrade-etcd.md
View File

@ -275,16 +275,16 @@ that is not currently used by an etcd process. Taking the snapshot will
not affect the performance of the member.
Below is an example for taking a snapshot of the keyspace served by
`$ENDPOINT` to the file `snapshotdb`:
`$ENDPOINT` to the file `snapshot.db`:
```shell
ETCDCTL_API=3 etcdctl --endpoints $ENDPOINT snapshot save snapshotdb
ETCDCTL_API=3 etcdctl --endpoints $ENDPOINT snapshot save snapshot.db
```
Verify the snapshot:
```shell
ETCDCTL_API=3 etcdctl --write-out=table snapshot status snapshotdb
ETCDCTL_API=3 etcdctl --write-out=table snapshot status snapshot.db
```
```console
@ -343,19 +343,25 @@ employed to recover the data of a failed cluster.
Before starting the restore operation, a snapshot file must be present. It can
either be a snapshot file from a previous backup operation, or from a remaining
[data directory](https://etcd.io/docs/current/op-guide/configuration/#--data-dir).
Here is an example:
```shell
ETCDCTL_API=3 etcdctl --endpoints 10.2.0.9:2379 snapshot restore snapshotdb
ETCDCTL_API=3 etcdctl --endpoints 10.2.0.9:2379 snapshot restore snapshot.db
```
Another example for restoring using etcdctl options:
Another example for restoring using `etcdctl` options:
```shell
ETCDCTL_API=3 etcdctl snapshot restore --data-dir <data-dir-location> snapshotdb
ETCDCTL_API=3 etcdctl --data-dir <data-dir-location> snapshot restore snapshot.db
```
Yet another example would be to first export the environment variable
where `<data-dir-location>` is a directory that will be created during the restore process.
Yet another example would be to first export the `ETCDCTL_API` environment variable:
```shell
export ETCDCTL_API=3
etcdctl snapshot restore --data-dir <data-dir-location> snapshotdb
etcdctl --data-dir <data-dir-location> snapshot restore snapshot.db
```
For more information and examples on restoring a cluster from a snapshot file, see
@ -410,4 +416,8 @@ Defragmentation is an expensive operation, so it should be executed as infrequen
as possible. On the other hand, it's also necessary to make sure any etcd member
will not run out of the storage quota. The Kubernetes project recommends that when
you perform defragmentation, you use a tool such as [etcd-defrag](https://github.com/ahrtr/etcd-defrag).
You can also run the defragmentation tool as a Kubernetes CronJob, to make sure that
defragmentation happens regularly. See [`etcd-defrag-cronjob.yaml`](https://github.com/ahrtr/etcd-defrag/blob/main/doc/etcd-defrag-cronjob.yaml)
for details.
{{< /note >}}

55
content/en/docs/tasks/administer-cluster/kubeadm/change-package-repository.md
View File

@ -6,21 +6,25 @@ weight: 120
<!-- overview -->
This page explains how to switch from one Kubernetes package repository to another
when upgrading Kubernetes minor releases. Unlike deprecated Google-hosted
repositories, the Kubernetes package repositories are structured in a way that
there's a dedicated package repository for each Kubernetes minor version.
This page explains how to enable a package repository for a new Kubernetes minor release
for users of the community-owned package repositories hosted at `pkgs.k8s.io`.
Unlike the legacy package repositories, the community-owned package repositories are
structured in a way that there's a dedicated package repository for each Kubernetes
minor version.
## {{% heading "prerequisites" %}}
This document assumes that you're already using the Kubernetes community-owned
package repositories. If that's not the case, it's strongly recommended to migrate
to the Kubernetes package repositories.
This document assumes that you're already using the community-owned
package repositories (`pkgs.k8s.io`). If that's not the case, it's strongly
recommended to migrate to the community-owned package repositories as described
in the [official announcement](/blog/2023/08/15/pkgs-k8s-io-introduction/).
{{% legacy-repos-deprecation %}}
### Verifying if the Kubernetes package repositories are used
If you're unsure whether you're using the Kubernetes package repositories or the
Google-hosted repository, take the following steps to verify:
If you're unsure whether you're using the community-owned package repositories or the
legacy package repositories, take the following steps to verify:
{{< tabs name="k8s_install_versions" >}}
{{% tab name="Ubuntu, Debian or HypriotOS" %}}
@ -39,7 +43,8 @@ deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io
```
**You're using the Kubernetes package repositories and this guide applies to you.**
Otherwise, it's strongly recommended to migrate to the Kubernetes package repositories.
Otherwise, it's strongly recommended to migrate to the Kubernetes package repositories
as described in the [official announcement](/blog/2023/08/15/pkgs-k8s-io-introduction/).
{{% /tab %}}
{{% tab name="CentOS, RHEL or Fedora" %}}
@ -64,7 +69,35 @@ exclude=kubelet kubeadm kubectl
```
**You're using the Kubernetes package repositories and this guide applies to you.**
Otherwise, it's strongly recommended to migrate to the Kubernetes package repositories.
Otherwise, it's strongly recommended to migrate to the Kubernetes package repositories
as described in the [official announcement](/blog/2023/08/15/pkgs-k8s-io-introduction/).
{{% /tab %}}
{{% tab name="openSUSE or SLES" %}}
Print the contents of the file that defines the Kubernetes `zypper` repository:
```shell
# On your system, this configuration file could have a different name
cat /etc/zypp/repos.d/kubernetes.repo
```
If you see a `baseurl` similar to the `baseurl` in the output below:
```
[kubernetes]
name=Kubernetes
baseurl=https://pkgs.k8s.io/core:/stable:/v{{< skew currentVersionAddMinor -1 "." >}}/rpm/
enabled=1
gpgcheck=1
gpgkey=https://pkgs.k8s.io/core:/stable:/v{{< skew currentVersionAddMinor -1 "." >}}/rpm/repodata/repomd.xml.key
exclude=kubelet kubeadm kubectl
```
**You're using the Kubernetes package repositories and this guide applies to you.**
Otherwise, it's strongly recommended to migrate to the Kubernetes package repositories
as described in the [official announcement](/blog/2023/08/15/pkgs-k8s-io-introduction/).
{{% /tab %}}
{{< /tabs >}}

8
content/en/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade.md
View File

@ -54,11 +54,13 @@ The upgrade workflow at high level is the following:
## Changing the package repository
If you're using the Kubernetes community-owned repositories, you need to change
the package repository to one that contains packages for your desired Kubernetes
minor version. This is explained in [Changing the Kubernetes package repository](/docs/tasks/administer-cluster/kubeadm/change-package-repository/)
If you're using the community-owned package repositories (`pkgs.k8s.io`), you need to
enable the package repository for the desired Kubernetes minor release. This is explained in
[Changing the Kubernetes package repository](/docs/tasks/administer-cluster/kubeadm/change-package-repository/)
document.
{{% legacy-repos-deprecation %}}
## Determine which version to upgrade to
Find the latest patch release for Kubernetes {{< skew currentVersion >}} using the OS package manager:

8
content/en/docs/tasks/administer-cluster/kubeadm/upgrading-linux-nodes.md
View File

@ -19,11 +19,13 @@ upgrade the control plane nodes before upgrading your Linux Worker nodes.
## Changing the package repository
If you're using the Kubernetes community-owned repositories, you need to change
the package repository to one that contains packages for your desired Kubernetes
minor version. This is explained in [Changing the Kubernetes package repository](/docs/tasks/administer-cluster/kubeadm/change-package-repository/)
If you're using the community-owned package repositories (`pkgs.k8s.io`), you need to
enable the package repository for the desired Kubernetes minor release. This is explained in
[Changing the Kubernetes package repository](/docs/tasks/administer-cluster/kubeadm/change-package-repository/)
document.
{{% legacy-repos-deprecation %}}
## Upgrading worker nodes
### Upgrade kubeadm

23
content/en/docs/tasks/administer-cluster/kubelet-config-file.md
View File

@ -35,14 +35,22 @@ address: "192.168.0.8"
port: 20250
serializeImagePulls: false
evictionHard:
memory.available: "200Mi"
memory.available: "100Mi"
nodefs.available: "10%"
nodefs.inodesFree: "5%"
imagefs.available: "15%"
```
In the example, the kubelet is configured to serve on IP address 192.168.0.8 and port 20250, pull images in parallel,
and evict Pods when available memory drops below 200Mi. Since only one of the four evictionHard thresholds is configured,
other evictionHard thresholds are reset to 0 from their built-in defaults.
All other kubelet configuration values are left at their built-in defaults, unless overridden
by flags. Command line flags which target the same value as a config file will override that value.
In this example, the kubelet is configured with the following settings:
1. `address`: The kubelet will serve on IP address `192.168.0.8`.
2. `port`: The kubelet will serve on port `20250`.
3. `serializeImagePulls`: Image pulls will be done in parallel.
4. `evictionHard`: The kubelet will evict Pods under one of the following conditions:
- When the node's available memory drops below 100MiB.
- When the node's main filesystem's available space is less than 10%.
- When the image filesystem's available space is less than 15%.
- When more than 95% of the node's main filesystem's inodes are in use.
{{< note >}}
In the example, by changing the default value of only one parameter for
@ -51,6 +59,9 @@ will be set to zero. In order to provide custom values, you should provide all
the threshold values respectively.
{{< /note >}}
The `imagefs` is an optional filesystem that container runtimes use to store container
images and container writable layers.
## Start a kubelet process configured via the config file
{{< note >}}

4
content/en/docs/tasks/administer-cluster/migrating-from-dockershim/migrate-dockershim-dockerd.md
View File

@ -76,6 +76,8 @@ instructions for that tool.
1. Open `/var/lib/kubelet/kubeadm-flags.env` on each affected node.
1. Modify the `--container-runtime-endpoint` flag to
`unix:///var/run/cri-dockerd.sock`.
1. Modify the `--container-runtime` flag to `remote`
(unavailable in Kubernetes v1.27 and later).
The kubeadm tool stores the node's socket as an annotation on the `Node` object
in the control plane. To modify this socket for each affected node:
@ -118,4 +120,4 @@ kubectl uncordon <NODE_NAME>
## {{% heading "whatsnext" %}}
* Read the [dockershim removal FAQ](/dockershim/).
* [Learn how to migrate from Docker Engine with dockershim to containerd](/docs/tasks/administer-cluster/migrating-from-dockershim/change-runtime-containerd/).
* [Learn how to migrate from Docker Engine with dockershim to containerd](/docs/tasks/administer-cluster/migrating-from-dockershim/change-runtime-containerd/).

2
content/en/docs/tasks/administer-cluster/reserve-compute-resources.md
View File

@ -96,7 +96,7 @@ system daemon should ideally run within its own child control group. Refer to
for more details on recommended control group hierarchy.
Note that Kubelet **does not** create `--kube-reserved-cgroup` if it doesn't
exist. Kubelet will fail if an invalid cgroup is specified. With `systemd`
exist. The kubelet will fail to start if an invalid cgroup is specified. With `systemd`
cgroup driver, you should follow a specific pattern for the name of the cgroup you
define: the name should be the value you set for `--kube-reserved-cgroup`,
with `.slice` appended.

2
content/en/docs/tasks/administer-cluster/verify-signed-artifacts.md
View File

@ -15,7 +15,7 @@ You will need to have the following tools installed:
- `cosign` ([install guide](https://docs.sigstore.dev/cosign/installation/))
- `curl` (often provided by your operating system)
- `jq` ([download jq](https://stedolan.github.io/jq/download/))
- `jq` ([download jq](https://jqlang.github.io/jq/download/))
## Verifying binary signatures

8
content/en/docs/tasks/configmap-secret/managing-secret-using-config-file.md
View File

@ -109,6 +109,10 @@ stringData:
password: <password>
```
{{< note >}}
The `stringData` field for a Secret does not work well with server-side apply.
{{< /note >}}
When you retrieve the Secret data, the command returns the encoded values,
and not the plaintext values you provided in `stringData`.
@ -152,6 +156,10 @@ stringData:
username: administrator
```
{{< note >}}
The `stringData` field for a Secret does not work well with server-side apply.
{{< /note >}}
The `Secret` object is created as follows:
```yaml

4
content/en/docs/tasks/configure-pod-container/enforce-standards-admission-controller.md
View File

@ -33,12 +33,12 @@ For v1.22, use [v1alpha1](https://v1-22.docs.kubernetes.io/docs/tasks/configure-
{{< /note >}}
```yaml
apiVersion: apiserver.config.k8s.io/v1 # see compatibility note
apiVersion: apiserver.config.k8s.io/v1
kind: AdmissionConfiguration
plugins:
- name: PodSecurity
configuration:
apiVersion: pod-security.admission.config.k8s.io/v1
apiVersion: pod-security.admission.config.k8s.io/v1 # see compatibility note
kind: PodSecurityConfiguration
# Defaults applied when a mode label is not set.
#

21
content/en/docs/tasks/configure-pod-container/pull-image-private-registry.md
View File

@ -38,7 +38,8 @@ docker login
When prompted, enter your Docker ID, and then the credential you want to use (access token,
or the password for your Docker ID).
The login process creates or updates a `config.json` file that holds an authorization token. Review [how Kubernetes interprets this file](/docs/concepts/containers/images#config-json).
The login process creates or updates a `config.json` file that holds an authorization token.
Review [how Kubernetes interprets this file](/docs/concepts/containers/images#config-json).
View the `config.json` file:
@ -60,7 +61,8 @@ The output contains a section similar to this:
{{< note >}}
If you use a Docker credentials store, you won't see that `auth` entry but a `credsStore` entry with the name of the store as value.
In that case, you can create a secret directly. See [Create a Secret by providing credentials on the command line](#create-a-secret-by-providing-credentials-on-the-command-line).
In that case, you can create a secret directly.
See [Create a Secret by providing credentials on the command line](#create-a-secret-by-providing-credentials-on-the-command-line).
{{< /note >}}
## Create a Secret based on existing credentials {#registry-secret-existing-credentials}
@ -211,7 +213,14 @@ kubectl get pod private-reg
```
{{< note >}}
In case the Pod fails to start with the status `ImagePullBackOff`, view the Pod events:
To use image pull secrets for a Pod (or a Deployment, or other object that
has a pod template that you are using), you need to make sure that the appropriate
Secret does exist in the right namespace. The namespace to use is the same
namespace where you defined the Pod.
{{< /note >}}
Also, in case the Pod fails to start with the status `ImagePullBackOff`, view the Pod events:
```shell
kubectl describe pod private-reg
```
@ -229,12 +238,6 @@ Events:
... FailedToRetrieveImagePullSecret ... Unable to retrieve some image pull secrets (<regcred>); attempting to pull the image may not succeed.
```
{{< /note >}}
## {{% heading "whatsnext" %}}
* Learn more about [Secrets](/docs/concepts/configuration/secret/)

3
content/en/docs/tasks/debug/debug-cluster/_index.md
View File

@ -14,6 +14,9 @@ problem you are experiencing. See
the [application troubleshooting guide](/docs/tasks/debug/debug-application/) for tips on application debugging.
You may also visit the [troubleshooting overview document](/docs/tasks/debug/) for more information.
For troubleshooting {{<glossary_tooltip text="kubectl" term_id="kubectl">}}, refer to
[Troubleshooting kubectl](/docs/tasks/debug/debug-cluster/troubleshoot-kubectl/).
<!-- body -->
## Listing your cluster

Some files were not shown because too many files have changed in this diff Show More