kubernetes
/
website
mirror of https://github.com/kubernetes/website.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

Merge remote-tracking branch 'origin/master'

pull/288/head
scotty 2016-03-30 09:03:26 -07:00
parent 0121f8ba03 7b4ca8b1c3
commit 250be20b53
35 changed files with 1876 additions and 1866 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

10
404.md
View File

@ -15,6 +15,16 @@ $( document ).ready(function() {
notHere = true;
window.location.replace("http://kubernetes.io/kubernetes/third_party/swagger-ui/");
}
if (forwardingURL.indexOf("resource-quota") > -1)
{
notHere = true;
window.location.replace("http://kubernetes.io/docs/admin/resourcequota/");
}
if (forwardingURL.indexOf("horizontal-pod-autoscaler") > -1)
{
notHere = true;
window.location.replace("http://kubernetes.io/docs/user-guide/horizontal-pod-autoscaling/");
}
if (forwardingURL.indexOf("docs/roadmap") > -1)
{
notHere = true;

8
_data/guides.yml
View File

@ -126,8 +126,10 @@ toc:
path: /docs/admin/high-availability/
- title: Accessing Clusters
path: /docs/user-guide/accessing-the-cluster/
- title: Sharing a Cluster
- title: Sharing a Cluster with Namespaces
path: /docs/admin/namespaces/
- title: Namespaces Walkthrough
path: /docs/admin/namespaces/walkthrough/
- title: Changing Cluster Size
path: https://github.com/kubernetes/kubernetes/wiki/User-FAQ#how-do-i-change-the-size-of-my-cluster/
- title: Creating a Custom Cluster from Scratch
@ -219,8 +221,10 @@ toc:
path: /docs/user-guide/compute-resources/
- title: Using kubectl to Manage Resources
path: /docs/user-guide/working-with-resources/
- title: Applying Resource Quotas and Limits
- title: Understanding Resource Quotas
path: /docs/admin/resourcequota/
- title: Applying Resource Quotas and Limits
path: /docs/admin/resourcequota/walkthrough/
- title: Setting Pod CPU and Memory Limits
path: /docs/admin/limitrange/
- title: Configuring Garbage Collection

2
_data/reference.yml
View File

@ -215,7 +215,7 @@ toc:
- title: Ingress Resources
path: /docs/user-guide/ingress/
- title: Horizontal Pod Autoscaling
path: /docs/user-guide/horizontal-pod-autoscaler/
path: /docs/user-guide/horizontal-pod-autoscaling/
- title: Jobs
path: /docs/user-guide/jobs/
- title: Resource Quotas

8
_data/samples.yml
View File

@ -35,6 +35,8 @@ toc:
- title: Persistent Volume Samples
section:
- title: Persistent Volumes Walkthrough
path: /docs/user-guide/persistent-volumes/walkthrough/
- title: WordPress on a Kubernetes Persistent Volume
path: https://github.com/kubernetes/kubernetes/tree/release-1.2/examples/mysql-wordpress-pd/
- title: GlusterFS
@ -60,3 +62,9 @@ toc:
- title: ConfigMap Example
path: https://github.com/kubernetes/kubernetes.github.io/tree/master/docs/user-guide/configmap
- title: Horizontal Pod Autoscaling Example
path: /docs/user-guide/horizontal-pod-autoscaling/walkthrough/
- title: Secrets Walkthrough
path: /docs/user-guide/secrets/walkthrough/

11
docs/admin/high-availability.md → docs/admin/high-availability/index.md
View File

@ -27,10 +27,9 @@ The steps involved are as follows:
* [Starting replicated, load balanced Kubernetes API servers](#replicated-api-servers)
* [Setting up master-elected Kubernetes scheduler and controller-manager daemons](#master-elected-components)
Here's what the system should look like when it's finished:
![High availability Kubernetes diagram](/images/docs/ha.svg)
Here's what the system should look like when it's finished:
Ready? Let's get started.
![High availability Kubernetes diagram](/images/docs/ha.svg)
## Initial set-up
@ -55,11 +54,11 @@ choices. For example, on systemd-based systems (e.g. RHEL, CentOS), you can run
If you are extending from a standard Kubernetes installation, the `kubelet` binary should already be present on your system. You can run
`which kubelet` to determine if the binary is in fact installed. If it is not installed,
you should install the [kubelet binary](https://storage.googleapis.com/kubernetes-release/release/v0.19.3/bin/linux/amd64/kubelet), the
[kubelet init file](http://releases.k8s.io/{{page.githubbranch}}/cluster/saltbase/salt/kubelet/initd) and [high-availability/default-kubelet](/docs/admin/high-availability/default-kubelet)
[kubelet init file](http://releases.k8s.io/{{page.githubbranch}}/cluster/saltbase/salt/kubelet/initd) and [default-kubelet](/docs/admin/high-availability/default-kubelet)
scripts.
If you are using monit, you should also install the monit daemon (`apt-get install monit`) and the [high-availability/monit-kubelet](/docs/admin/high-availability/monit-kubelet) and
[high-availability/monit-docker](/docs/admin/high-availability/monit-docker) configs.
If you are using monit, you should also install the monit daemon (`apt-get install monit`) and the [monit-kubelet](/docs/admin/high-availability/monit-kubelet) and
[monit-docker](/docs/admin/high-availability/monit-docker) configs.
On systemd systems you `systemctl enable kubelet` and `systemctl enable docker`.

2
docs/admin/index.md
View File

@ -61,7 +61,7 @@ project](/docs/admin/salt).
## Multi-tenant support
* **Resource Quota** ([resource-quota.md](/docs/admin/resource-quota))
* **Resource Quota** ([resourcequota/](/docs/admin/resourcequota/))
## Security

108
docs/admin/limitrange/index.md
View File

@ -22,7 +22,7 @@ may be too small to be useful, but big enough for the waste to be costly over th
the cluster operator may want to set limits that a pod must consume at least 20% of the memory and cpu of their
average node size in order to provide for more uniform scheduling and to limit waste.
This example demonstrates how limits can be applied to a Kubernetes namespace to control
This example demonstrates how limits can be applied to a Kubernetes [namespace](/docs/admin/namespaces/walkthrough/) to control
min/max resource limits per pod. In addition, this example demonstrates how you can
apply default resource limits to pods in the absence of an end-user specified value.
@ -41,12 +41,17 @@ This example will work in a custom namespace to demonstrate the concepts involve
Let's create a new namespace called limit-example:
```shell
$ kubectl create -f docs/admin/limitrange/namespace.yaml
namespace "limit-example" created
$ kubectl create namespace limit-example
namespace "limit-example" created
```
Note that `kubectl` commands will print the type and name of the resource created or mutated, which can then be used in subsequent commands:
```shell
$ kubectl get namespaces
NAME LABELS STATUS AGE
default <none> Active 5m
limit-example <none> Active 53s
NAME STATUS AGE
default Active 51s
limit-example Active 45s
```
## Step 2: Apply a limit to the namespace
@ -95,36 +100,45 @@ were previously created in a namespace.
If a resource (cpu or memory) is being restricted by a limit, the user will get an error at time
of creation explaining why.
Let's first spin up a deployment that creates a single container pod to demonstrate
Let's first spin up a [Deployment](/docs/user-guide/deployments) that creates a single container Pod to demonstrate
how default values are applied to each pod.
```shell
$ kubectl run nginx --image=nginx --replicas=1 --namespace=limit-example
deployment "nginx" created
```
Note that `kubectl run` creates a Deployment named "nginx" on Kubernetes cluster >= v1.2. If you are running older versions, it creates replication controllers instead.
If you want to obtain the old behavior, use `--generator=run/v1` to create replication controllers. See [`kubectl run`](/docs/user-guide/kubectl/kubectl_run/) for more details.
The Deployment manages 1 replica of single container Pod. Let's take a look at the Pod it manages. First, find the name of the Pod:
```shell
$ kubectl get pods --namespace=limit-example
NAME READY STATUS RESTARTS AGE
nginx-2040093540-s8vzu 1/1 Running 0 11s
$ kubectl get pods nginx-2040093540-s8vzu --namespace=limit-example -o yaml | grep resources -C 8
```
```yaml
resourceVersion: "127"
selfLink: /api/v1/namespaces/limit-example/pods/nginx-aq0mf
uid: 51be42a7-7156-11e5-9921-286ed488f785
spec:
containers:
- image: nginx
imagePullPolicy: IfNotPresent
name: nginx
resources:
limits:
cpu: 300m
memory: 200Mi
requests:
cpu: 200m
memory: 100Mi
terminationMessagePath: /dev/termination-log
volumeMounts:
Let's print this Pod with yaml output format (using `-o yaml` flag), and then `grep` the `resources` field. Note that your pod name will be different.
``` shell
$ kubectl get pods nginx-2040093540-s8vzu --namespace=limit-example -o yaml | grep resources -C 8
resourceVersion: "57"
selfLink: /api/v1/namespaces/limit-example/pods/nginx-2040093540-ivimu
uid: 67b20741-f53b-11e5-b066-64510658e388
spec:
containers:
- image: nginx
imagePullPolicy: Always
name: nginx
resources:
limits:
cpu: 300m
memory: 200Mi
requests:
cpu: 200m
memory: 100Mi
terminationMessagePath: /dev/termination-log
volumeMounts:
```
Note that our nginx container has picked up the namespace default cpu and memory resource *limits* and *requests*.
@ -141,37 +155,39 @@ Let's create a pod that falls within the allowed limit boundaries.
```shell
$ kubectl create -f docs/admin/limitrange/valid-pod.yaml --namespace=limit-example
pod "valid-pod" created
$ kubectl get pods valid-pod --namespace=limit-example -o yaml | grep -C 6 resources
```
```yaml
uid: 162a12aa-7157-11e5-9921-286ed488f785
spec:
containers:
- image: gcr.io/google_containers/serve_hostname
imagePullPolicy: IfNotPresent
name: kubernetes-serve-hostname
resources:
limits:
cpu: "1"
memory: 512Mi
requests:
cpu: "1"
memory: 512Mi
Now look at the Pod's resources field:
```shell
$ kubectl get pods valid-pod --namespace=limit-example -o yaml | grep -C 6 resources
uid: 3b1bfd7a-f53c-11e5-b066-64510658e388
spec:
containers:
- image: gcr.io/google_containers/serve_hostname
imagePullPolicy: Always
name: kubernetes-serve-hostname
resources:
limits:
cpu: "1"
memory: 512Mi
requests:
cpu: "1"
memory: 512Mi
```
Note that this pod specifies explicit resource *limits* and *requests* so it did not pick up the namespace
default values.
Note: The *limits* for CPU resource are not enforced in the default Kubernetes setup on the physical node
Note: The *limits* for CPU resource are enforced in the default Kubernetes setup on the physical node
that runs the container unless the administrator deploys the kubelet with the folllowing flag:
```shell
$ kubelet --help
Usage of kubelet
....
--cpu-cfs-quota[=false]: Enable CPU CFS quota enforcement for containers that specify CPU limits
$ kubelet --cpu-cfs-quota=true ...
--cpu-cfs-quota[=true]: Enable CPU CFS quota enforcement for containers that specify CPU limits
$ kubelet --cpu-cfs-quota=false ...
```
## Step 4: Cleanup
@ -182,8 +198,8 @@ To remove the resources used by this example, you can just delete the limit-exam
$ kubectl delete namespace limit-example
namespace "limit-example" deleted
$ kubectl get namespaces
NAME LABELS STATUS AGE
default <none> Active 20m
NAME STATUS AGE
default Active 12m
```
## Summary

145
docs/admin/namespaces.md
View File

@ -1,145 +0,0 @@
---
---
A Namespace is a mechanism to partition resources created by users into
a logically named group.
## Motivation
A single cluster should be able to satisfy the needs of multiple users or groups of users (henceforth a 'user community').
Each user community wants to be able to work in isolation from other communities.
Each user community has its own:
1. resources (pods, services, replication controllers, etc.)
2. policies (who can or cannot perform actions in their community)
3. constraints (this community is allowed this much quota, etc.)
A cluster operator may create a Namespace for each unique user community.
The Namespace provides a unique scope for:
1. named resources (to avoid basic naming collisions)
2. delegated management authority to trusted users
3. ability to limit community resource consumption
## Use cases
1. As a cluster operator, I want to support multiple user communities on a single cluster.
2. As a cluster operator, I want to delegate authority to partitions of the cluster to trusted users
in those communities.
3. As a cluster operator, I want to limit the amount of resources each community can consume in order
to limit the impact to other communities using the cluster.
4. As a cluster user, I want to interact with resources that are pertinent to my user community in
isolation of what other user communities are doing on the cluster.
## Usage
Look [here](/docs/admin/namespaces/) for an in depth example of namespaces.
### Viewing namespaces
You can list the current namespaces in a cluster using:
```shell
$ kubectl get namespaces
NAME LABELS STATUS
default <none> Active
kube-system <none> Active
```
Kubernetes starts with two initial namespaces:
* `default` The default namespace for objects with no other namespace
* `kube-system` The namespace for objects created by the Kubernetes system
You can also get the summary of a specific namespace using:
```shell
$ kubectl get namespaces <name>
```
Or you can get detailed information with:
```shell
$ kubectl describe namespaces <name>
Name: default
Labels: <none>
Status: Active
No resource quota.
Resource Limits
Type Resource Min Max Default
---- -------- --- --- ---
Container cpu - - 100m
```
Note that these details show both resource quota (if present) as well as resource limit ranges.
Resource quota tracks aggregate usage of resources in the *Namespace* and allows cluster operators
to define *Hard* resource usage limits that a *Namespace* may consume.
A limit range defines min/max constraints on the amount of resources a single entity can consume in
a *Namespace*.
See [Admission control: Limit Range](https://github.com/kubernetes/kubernetes/blob/{{page.githubbranch}}/docs/design/admission_control_limit_range.md)
A namespace can be in one of two phases:
* `Active` the namespace is in use
* `Terminating` the namespace is being deleted, and can not be used for new objects
See the [design doc](https://github.com/kubernetes/kubernetes/blob/{{page.githubbranch}}/docs/design/namespaces.md#phases) for more details.
### Creating a new namespace
To create a new namespace, first create a new YAML file called `my-namespace.yaml` with the contents:
```yaml
apiVersion: v1
kind: Namespace
metadata:
name: <insert-namespace-name-here>
```
Note that the name of your namespace must be a DNS compatible label.
More information on the `finalizers` field can be found in the namespace [design doc](https://github.com/kubernetes/kubernetes/blob/{{page.githubbranch}}/docs/design/namespaces.md#finalizers).
Then run:
```shell
$ kubectl create -f ./my-namespace.yaml
```
### Working in namespaces
See [Setting the namespace for a request](/docs/user-guide/namespaces/#setting-the-namespace-for-a-request)
and [Setting the namespace preference](/docs/user-guide/namespaces/#setting-the-namespace-preference).
### Deleting a namespace
You can delete a namespace with
```shell
$ kubectl delete namespaces <insert-some-namespace-name>
```
**WARNING, this deletes _everything_ under the namespace!**
This delete is asynchronous, so for a time you will see the namespace in the `Terminating` state.
## Namespaces and DNS
When you create a [Service](/docs/user-guide/services), it creates a corresponding [DNS entry](/docs/admin/dns).
This entry is of the form `<service-name>.<namespace-name>.svc.cluster.local`, which means
that if a container just uses `<service-name>` it will resolve to the service which
is local to a namespace. This is useful for using the same configuration across
multiple namespaces such as Development, Staging and Production. If you want to reach
across namespaces, you need to use the fully qualified domain name (FQDN).
## Design
Details of the design of namespaces in Kubernetes, including a [detailed example](https://github.com/kubernetes/kubernetes/blob/{{page.githubbranch}}/docs/design/namespaces.md#example-openshift-origin-managing-a-kubernetes-namespace)
can be found in the [namespaces design doc](https://github.com/kubernetes/kubernetes/blob/{{page.githubbranch}}/docs/design/namespaces.md)

272
docs/admin/namespaces/index.md
View File

@ -1,231 +1,145 @@
---
---
Kubernetes _namespaces_ help different projects, teams, or customers to share a Kubernetes cluster.
A Namespace is a mechanism to partition resources created by users into
a logically named group.
It does this by providing the following:
## Motivation
1. A scope for [Names](/docs/user-guide/identifiers).
2. A mechanism to attach authorization and policy to a subsection of the cluster.
A single cluster should be able to satisfy the needs of multiple users or groups of users (henceforth a 'user community').
Use of multiple namespaces is optional.
Each user community wants to be able to work in isolation from other communities.
This example demonstrates how to use Kubernetes namespaces to subdivide your cluster.
Each user community has its own:
### Step Zero: Prerequisites
1. resources (pods, services, replication controllers, etc.)
2. policies (who can or cannot perform actions in their community)
3. constraints (this community is allowed this much quota, etc.)
This example assumes the following:
A cluster operator may create a Namespace for each unique user community.
1. You have an [existing Kubernetes cluster](/docs/getting-started-guides/).
2. You have a basic understanding of Kubernetes _[Pods](/docs/user-guide/pods)_, _[Services](/docs/user-guide/services)_, and _[Deployments](/docs/user-guide/deployments)_.
The Namespace provides a unique scope for:
### Step One: Understand the default namespace
1. named resources (to avoid basic naming collisions)
2. delegated management authority to trusted users
3. ability to limit community resource consumption
By default, a Kubernetes cluster will instantiate a default namespace when provisioning the cluster to hold the default set of Pods,
Services, and Deployments used by the cluster.
## Use cases
Assuming you have a fresh cluster, you can introspect the available namespace's by doing the following:
1. As a cluster operator, I want to support multiple user communities on a single cluster.
2. As a cluster operator, I want to delegate authority to partitions of the cluster to trusted users
in those communities.
3. As a cluster operator, I want to limit the amount of resources each community can consume in order
to limit the impact to other communities using the cluster.
4. As a cluster user, I want to interact with resources that are pertinent to my user community in
isolation of what other user communities are doing on the cluster.
## Usage
Look [here](/docs/admin/namespaces/) for an in depth example of namespaces.
### Viewing namespaces
You can list the current namespaces in a cluster using:
```shell
$ kubectl get namespaces
NAME LABELS
default <none>
NAME LABELS STATUS
default <none> Active
kube-system <none> Active
```
### Step Two: Create new namespaces
Kubernetes starts with two initial namespaces:
* `default` The default namespace for objects with no other namespace
* `kube-system` The namespace for objects created by the Kubernetes system
For this exercise, we will create two additional Kubernetes namespaces to hold our content.
Let's imagine a scenario where an organization is using a shared Kubernetes cluster for development and production use cases.
The development team would like to maintain a space in the cluster where they can get a view on the list of Pods, Services, and Deployments
they use to build and run their application. In this space, Kubernetes resources come and go, and the restrictions on who can or cannot modify resources
are relaxed to enable agile development.
The operations team would like to maintain a space in the cluster where they can enforce strict procedures on who can or cannot manipulate the set of
pods, services, and Deployments that run the production site.
One pattern this organization could follow is to partition the Kubernetes cluster into two namespaces: development and production.
Let's create two new namespaces to hold our work.
Use the file [`namespace-dev.json`](/docs/admin/namespaces/namespace-dev.json) which describes a development namespace:
{% include code.html language="json" file="namespace-dev.json" ghlink="/docs/admin/namespaces/namespace-dev.json" %}
Create the development namespace using kubectl.
You can also get the summary of a specific namespace using:
```shell
$ kubectl create -f docs/admin/namespaces/namespace-dev.json
$ kubectl get namespaces <name>
```
And then lets create the production namespace using kubectl.
Or you can get detailed information with:
```shell
$ kubectl create -f docs/admin/namespaces/namespace-prod.json
$ kubectl describe namespaces <name>
Name: default
Labels: <none>
Status: Active
No resource quota.
Resource Limits
Type Resource Min Max Default
---- -------- --- --- ---
Container cpu - - 100m
```
To be sure things are right, let's list all of the namespaces in our cluster.
Note that these details show both resource quota (if present) as well as resource limit ranges.
```shell
$ kubectl get namespaces
NAME LABELS STATUS
default <none> Active
development name=development Active
production name=production Active
```
Resource quota tracks aggregate usage of resources in the *Namespace* and allows cluster operators
to define *Hard* resource usage limits that a *Namespace* may consume.
### Step Three: Create pods in each namespace
A limit range defines min/max constraints on the amount of resources a single entity can consume in
a *Namespace*.
A Kubernetes namespace provides the scope for Pods, Services, and Deployments in the cluster.
See [Admission control: Limit Range](https://github.com/kubernetes/kubernetes/blob/{{page.githubbranch}}/docs/design/admission_control_limit_range.md)
Users interacting with one namespace do not see the content in another namespace.
A namespace can be in one of two phases:
* `Active` the namespace is in use
* `Terminating` the namespace is being deleted, and can not be used for new objects
To demonstrate this, let's spin up a simple Deployment and Pods in the development namespace.
See the [design doc](https://github.com/kubernetes/kubernetes/blob/{{page.githubbranch}}/docs/design/namespaces.md#phases) for more details.
We first check what is the current context:
### Creating a new namespace
To create a new namespace, first create a new YAML file called `my-namespace.yaml` with the contents:
```yaml
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: REDACTED
server: https://130.211.122.180
name: lithe-cocoa-92103_kubernetes
contexts:
- context:
cluster: lithe-cocoa-92103_kubernetes
user: lithe-cocoa-92103_kubernetes
name: lithe-cocoa-92103_kubernetes
current-context: lithe-cocoa-92103_kubernetes
kind: Config
preferences: {}
users:
- name: lithe-cocoa-92103_kubernetes
user:
client-certificate-data: REDACTED
client-key-data: REDACTED
token: 65rZW78y8HbwXXtSXuUw9DbP4FLjHi4b
- name: lithe-cocoa-92103_kubernetes-basic-auth
user:
password: h5M0FtUUIflBSdI7
username: admin
kind: Namespace
metadata:
name: <insert-namespace-name-here>
```
The next step is to define a context for the kubectl client to work in each namespace. The value of "cluster" and "user" fields are copied from the current context.
Note that the name of your namespace must be a DNS compatible label.
More information on the `finalizers` field can be found in the namespace [design doc](https://github.com/kubernetes/kubernetes/blob/{{page.githubbranch}}/docs/design/namespaces.md#finalizers).
Then run:
```shell
$ kubectl config set-context dev --namespace=development --cluster=lithe-cocoa-92103_kubernetes --user=lithe-cocoa-92103_kubernetes
$ kubectl config set-context prod --namespace=production --cluster=lithe-cocoa-92103_kubernetes --user=lithe-cocoa-92103_kubernetes
$ kubectl create -f ./my-namespace.yaml
```
The above commands provided two request contexts you can alternate against depending on what namespace you
wish to work against.
### Working in namespaces
Let's switch to operate in the development namespace.
See [Setting the namespace for a request](/docs/user-guide/namespaces/#setting-the-namespace-for-a-request)
and [Setting the namespace preference](/docs/user-guide/namespaces/#setting-the-namespace-preference).
### Deleting a namespace
You can delete a namespace with
```shell
$ kubectl config use-context dev
$ kubectl delete namespaces <insert-some-namespace-name>
```
You can verify your current context by doing the following:
**WARNING, this deletes _everything_ under the namespace!**
```shell
$ kubectl config view
```
This delete is asynchronous, so for a time you will see the namespace in the `Terminating` state.
```yaml
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: REDACTED
server: https://130.211.122.180
name: lithe-cocoa-92103_kubernetes
contexts:
- context:
cluster: lithe-cocoa-92103_kubernetes
namespace: development
user: lithe-cocoa-92103_kubernetes
name: dev
- context:
cluster: lithe-cocoa-92103_kubernetes
user: lithe-cocoa-92103_kubernetes
name: lithe-cocoa-92103_kubernetes
- context:
cluster: lithe-cocoa-92103_kubernetes
namespace: production
user: lithe-cocoa-92103_kubernetes
name: prod
current-context: dev
kind: Config
preferences: {}
users:
- name: lithe-cocoa-92103_kubernetes
user:
client-certificate-data: REDACTED
client-key-data: REDACTED
token: 65rZW78y8HbwXXtSXuUw9DbP4FLjHi4b
- name: lithe-cocoa-92103_kubernetes-basic-auth
user:
password: h5M0FtUUIflBSdI7
username: admin
```
## Namespaces and DNS
At this point, all requests we make to the Kubernetes cluster from the command line are scoped to the development namespace.
When you create a [Service](/docs/user-guide/services), it creates a corresponding [DNS entry](/docs/admin/dns).
This entry is of the form `<service-name>.<namespace-name>.svc.cluster.local`, which means
that if a container just uses `<service-name>` it will resolve to the service which
is local to a namespace. This is useful for using the same configuration across
multiple namespaces such as Development, Staging and Production. If you want to reach
across namespaces, you need to use the fully qualified domain name (FQDN).
Let's create some content.
## Design
```shell
$ kubectl run snowflake --image=kubernetes/serve_hostname --replicas=2
```
We have just created a deployment whose replica size is 2 that is running the pod called snowflake with a basic container that just serves the hostname.
Note that `kubectl run` creates deployments only on kubernetes cluster >= v1.2. If you are running older versions, it creates replication controllers instead.
```shell
$ kubectl get deployment
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
snowflake 2 2 2 2 2m
$ kubectl get pods -l run=snowflake
NAME READY STATUS RESTARTS AGE
snowflake-3968820950-9dgr8 1/1 Running 0 2m
snowflake-3968820950-vgc4n 1/1 Running 0 2m
```
And this is great, developers are able to do what they want, and they do not have to worry about affecting content in the production namespace.
Let's switch to the production namespace and show how resources in one namespace are hidden from the other.
```shell
$ kubectl config use-context prod
```
The production namespace should be empty, and the following commands should return nothing.
```shell
$ kubectl get deployment
$ kubectl get pods
```
Production likes to run cattle, so let's create some cattle pods.
```shell
$ kubectl run cattle --image=kubernetes/serve_hostname --replicas=5
$ kubectl get deployment
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
cattle 5 5 5 5 10s
kubectl get pods -l run=cattle
NAME READY STATUS RESTARTS AGE
cattle-2263376956-41xy6 1/1 Running 0 34s
cattle-2263376956-kw466 1/1 Running 0 34s
cattle-2263376956-n4v97 1/1 Running 0 34s
cattle-2263376956-p5p3i 1/1 Running 0 34s
cattle-2263376956-sxpth 1/1 Running 0 34s
```
At this point, it should be clear that the resources users create in one namespace are hidden from the other namespace.
As the policy support in Kubernetes evolves, we will extend this scenario to show how you can provide different
authorization rules for each namespace.
Details of the design of namespaces in Kubernetes, including a [detailed example](https://github.com/kubernetes/kubernetes/blob/{{page.githubbranch}}/docs/design/namespaces.md#example-openshift-origin-managing-a-kubernetes-namespace)
can be found in the [namespaces design doc](https://github.com/kubernetes/kubernetes/blob/{{page.githubbranch}}/docs/design/namespaces.md)

200
docs/admin/namespaces/walkthrough.md Normal file
View File

@ -0,0 +1,200 @@
---
---
Kubernetes _namespaces_ help different projects, teams, or customers to share a Kubernetes cluster.
It does this by providing the following:
1. A scope for [Names](/docs/user-guide/identifiers/).
2. A mechanism to attach authorization and policy to a subsection of the cluster.
Use of multiple namespaces is optional.
This example demonstrates how to use Kubernetes namespaces to subdivide your cluster.
### Step Zero: Prerequisites
This example assumes the following:
1. You have an [existing Kubernetes cluster](/docs/getting-started-guides/).
2. You have a basic understanding of Kubernetes _[Pods](/docs/user-guide/pods/)_, _[Services](/docs/user-guide/services/)_, and _[Deployments](/docs/user-guide/deployments/)_.
### Step One: Understand the default namespace
By default, a Kubernetes cluster will instantiate a default namespace when provisioning the cluster to hold the default set of Pods,
Services, and Deployments used by the cluster.
Assuming you have a fresh cluster, you can introspect the available namespace's by doing the following:
```shell
$ kubectl get namespaces
NAME STATUS AGE
default Active 13m
```
### Step Two: Create new namespaces
For this exercise, we will create two additional Kubernetes namespaces to hold our content.
Let's imagine a scenario where an organization is using a shared Kubernetes cluster for development and production use cases.
The development team would like to maintain a space in the cluster where they can get a view on the list of Pods, Services, and Deployments
they use to build and run their application. In this space, Kubernetes resources come and go, and the restrictions on who can or cannot modify resources
are relaxed to enable agile development.
The operations team would like to maintain a space in the cluster where they can enforce strict procedures on who can or cannot manipulate the set of
Pods, Services, and Deployments that run the production site.
One pattern this organization could follow is to partition the Kubernetes cluster into two namespaces: development and production.
Let's create two new namespaces to hold our work.
Use the file [`namespace-dev.json`](/docs/admin/namespaces/namespace-dev.json) which describes a development namespace:
{% include code.html language="json" file="namespace-dev.json" ghlink="/docs/admin/namespaces/namespace-dev.json" %}
Create the development namespace using kubectl.
```shell
$ kubectl create -f docs/admin/namespaces/namespace-dev.json
```
And then lets create the production namespace using kubectl.
```shell
$ kubectl create -f docs/admin/namespaces/namespace-prod.json
```
To be sure things are right, let's list all of the namespaces in our cluster.
```shell
$ kubectl get namespaces --show-labels
NAME STATUS AGE LABELS
default Active 32m <none>
development Active 29s name=development
production Active 23s name=production
```
### Step Three: Create pods in each namespace
A Kubernetes namespace provides the scope for Pods, Services, and Deployments in the cluster.
Users interacting with one namespace do not see the content in another namespace.
To demonstrate this, let's spin up a simple Deployment and Pods in the development namespace.
We first check what is the current context:
```shell
$ kubectl config view
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: REDACTED
server: https://130.211.122.180
name: lithe-cocoa-92103_kubernetes
contexts:
- context:
cluster: lithe-cocoa-92103_kubernetes
user: lithe-cocoa-92103_kubernetes
name: lithe-cocoa-92103_kubernetes
current-context: lithe-cocoa-92103_kubernetes
kind: Config
preferences: {}
users:
- name: lithe-cocoa-92103_kubernetes
user:
client-certificate-data: REDACTED
client-key-data: REDACTED
token: 65rZW78y8HbwXXtSXuUw9DbP4FLjHi4b
- name: lithe-cocoa-92103_kubernetes-basic-auth
user:
password: h5M0FtUUIflBSdI7
username: admin
$ kubectl config current-context
lithe-cocoa-92103_kubernetes
```
The next step is to define a context for the kubectl client to work in each namespace. The value of "cluster" and "user" fields are copied from the current context.
```shell
$ kubectl config set-context dev --namespace=development --cluster=lithe-cocoa-92103_kubernetes --user=lithe-cocoa-92103_kubernetes
$ kubectl config set-context prod --namespace=production --cluster=lithe-cocoa-92103_kubernetes --user=lithe-cocoa-92103_kubernetes
```
The above commands provided two request contexts you can alternate against depending on what namespace you
wish to work against.
Let's switch to operate in the development namespace.
```shell
$ kubectl config use-context dev
```
You can verify your current context by doing the following:
```shell
$ kubectl config current-context
dev
```
At this point, all requests we make to the Kubernetes cluster from the command line are scoped to the development namespace.
Let's create some content.
```shell
$ kubectl run snowflake --image=kubernetes/serve_hostname --replicas=2
```
We have just created a deployment whose replica size is 2 that is running the pod called snowflake with a basic container that just serves the hostname.
Note that `kubectl run` creates deployments only on kubernetes cluster >= v1.2. If you are running older versions, it creates replication controllers instead.
If you want to obtain the old behavior, use `--generator=run/v1` to create replication controllers. See [`kubectl run`](/docs/user-guide/kubectl/kubectl_run/) for more details.
```shell
$ kubectl get deployment
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
snowflake 2 2 2 2 2m
$ kubectl get pods -l run=snowflake
NAME READY STATUS RESTARTS AGE
snowflake-3968820950-9dgr8 1/1 Running 0 2m
snowflake-3968820950-vgc4n 1/1 Running 0 2m
```
And this is great, developers are able to do what they want, and they do not have to worry about affecting content in the production namespace.
Let's switch to the production namespace and show how resources in one namespace are hidden from the other.
```shell
$ kubectl config use-context prod
```
The production namespace should be empty, and the following commands should return nothing.
```shell
$ kubectl get deployment
$ kubectl get pods
```
Production likes to run cattle, so let's create some cattle pods.
```shell
$ kubectl run cattle --image=kubernetes/serve_hostname --replicas=5
$ kubectl get deployment
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
cattle 5 5 5 5 10s
kubectl get pods -l run=cattle
NAME READY STATUS RESTARTS AGE
cattle-2263376956-41xy6 1/1 Running 0 34s
cattle-2263376956-kw466 1/1 Running 0 34s
cattle-2263376956-n4v97 1/1 Running 0 34s
cattle-2263376956-p5p3i 1/1 Running 0 34s
cattle-2263376956-sxpth 1/1 Running 0 34s
```
At this point, it should be clear that the resources users create in one namespace are hidden from the other namespace.
As the policy support in Kubernetes evolves, we will extend this scenario to show how you can provide different
authorization rules for each namespace.

154
docs/admin/resource-quota.md
View File

@ -1,154 +0,0 @@
---
---
When several users or teams share a cluster with a fixed number of nodes,
there is a concern that one team could use more than its fair share of resources.
Resource quotas are a tool for administrators to address this concern. Resource quotas
work like this:
- Different teams work in different namespaces. Currently this is voluntary, but
support for making this mandatory via ACLs is planned.
- The administrator creates a Resource Quota for each namespace.
- Users put compute resource requests on their pods. The sum of all resource requests across
all pods in the same namespace must not exceed any hard resource limit in any Resource Quota
document for the namespace. Note that we used to verify Resource Quota by taking the sum of
resource limits of the pods, but this was altered to use resource requests. Backwards compatibility
for those pods previously created is preserved because pods that only specify a resource limit have
their resource requests defaulted to match their defined limits. The user is only charged for the
resources they request in the Resource Quota versus their limits because the request is the minimum
amount of resource guaranteed by the cluster during scheduling. For more information on over commit,
see [compute-resources](/docs/user-guide/compute-resources).
- If creating a pod would cause the namespace to exceed any of the limits specified in the
the Resource Quota for that namespace, then the request will fail with HTTP status
code `403 FORBIDDEN`.
- If quota is enabled in a namespace and the user does not specify *requests* on the pod for each
of the resources for which quota is enabled, then the POST of the pod will fail with HTTP
status code `403 FORBIDDEN`. Hint: Use the LimitRange admission controller to force default
values of *limits* (then resource *requests* would be equal to *limits* by default, see
[admission controller](/docs/admin/admission-controllers)) before the quota is checked to avoid this problem.
Examples of policies that could be created using namespaces and quotas are:
- In a cluster with a capacity of 32 GiB RAM, and 16 cores, let team A use 20 Gib and 10 cores,
let B use 10GiB and 4 cores, and hold 2GiB and 2 cores in reserve for future allocation.
- Limit the "testing" namespace to using 1 core and 1GiB RAM. Let the "production" namespace
use any amount.
In the case where the total capacity of the cluster is less than the sum of the quotas of the namespaces,
there may be contention for resources. This is handled on a first-come-first-served basis.
Neither contention nor changes to quota will affect already-running pods.
## Enabling Resource Quota
Resource Quota support is enabled by default for many Kubernetes distributions. It is
enabled when the apiserver `--admission-control=` flag has `ResourceQuota` as
one of its arguments.
Resource Quota is enforced in a particular namespace when there is a
`ResourceQuota` object in that namespace. There should be at most one
`ResourceQuota` object in a namespace.
## Compute Resource Quota
The total sum of [compute resources](/docs/user-guide/compute-resources) requested by pods
in a namespace can be limited. The following compute resource types are supported:
| ResourceName | Description |
| ------------ | ----------- |
| cpu | Total cpu requests of containers |
| memory | Total memory requests of containers
For example, `cpu` quota sums up the `resources.requests.cpu` fields of every
container of every pod in the namespace, and enforces a maximum on that sum.
## Object Count Quota
The number of objects of a given type can be restricted. The following types
are supported:
| ResourceName | Description |
| ------------ | ----------- |
| pods | Total number of pods |
| services | Total number of services |
| replicationcontrollers | Total number of replication controllers |
| resourcequotas | Total number of [resource quotas](/docs/admin/admission-controllers/#resourcequota) |
| secrets | Total number of secrets |
| persistentvolumeclaims | Total number of [persistent volume claims](/docs/user-guide/persistent-volumes/#persistentvolumeclaims) |
For example, `pods` quota counts and enforces a maximum on the number of `pods`
created in a single namespace.
You might want to set a pods quota on a namespace
to avoid the case where a user creates many small pods and exhausts the cluster's
supply of Pod IPs.
## Viewing and Setting Quotas
Kubectl supports creating, updating, and viewing quotas:
```shell
$ kubectl namespace myspace
$ cat <<EOF > quota.json
{
"apiVersion": "v1",
"kind": "ResourceQuota",
"metadata": {
"name": "quota"
},
"spec": {
"hard": {
"memory": "1Gi",
"cpu": "20",
"pods": "10",
"services": "5",
"replicationcontrollers":"20",
"resourcequotas":"1"
}
}
}
EOF
$ kubectl create -f ./quota.json
$ kubectl get quota
NAME
quota
$ kubectl describe quota quota
Name: quota
Resource Used Hard
-------- ---- ----
cpu 0m 20
memory 0 1Gi
pods 5 10
replicationcontrollers 5 20
resourcequotas 1 1
services 3 5
```
## Quota and Cluster Capacity
Resource Quota objects are independent of the Cluster Capacity. They are
expressed in absolute units. So, if you add nodes to your cluster, this does *not*
automatically give each namespace the ability to consume more resources.
Sometimes more complex policies may be desired, such as:
- proportionally divide total cluster resources among several teams.
- allow each tenant to grow resource usage as needed, but have a generous
limit to prevent accidental resource exhaustion.
- detect demand from one namespace, add nodes, and increase quota.
Such policies could be implemented using ResourceQuota as a building-block, by
writing a 'controller' which watches the quota usage and adjusts the quota
hard limits of each namespace according to other signals.
Note that resource quota divides up aggregate cluster resources, but it creates no
restrictions around nodes: pods from several namespaces may run on the same node.
## Example
See a [detailed example for how to use resource quota](/docs/admin/resourcequota/).
## Read More
See [ResourceQuota design doc](https://github.com/kubernetes/kubernetes/blob/{{page.githubbranch}}/docs/design/admission_control_resource_quota.md) for more information.

304
docs/admin/resourcequota/index.md
View File

@ -1,158 +1,154 @@
---
---
This example demonstrates how [resource quota](/docs/admin/admission-controllers/#resourcequota) and
[limitsranger](/docs/admin/admission-controllers/#limitranger) can be applied to a Kubernetes namespace.
When several users or teams share a cluster with a fixed number of nodes,
there is a concern that one team could use more than its fair share of resources.
Resource quotas are a tool for administrators to address this concern. Resource quotas
work like this:
- Different teams work in different namespaces. Currently this is voluntary, but
support for making this mandatory via ACLs is planned.
- The administrator creates a Resource Quota for each namespace.
- Users put compute resource requests on their pods. The sum of all resource requests across
all pods in the same namespace must not exceed any hard resource limit in any Resource Quota
document for the namespace. Note that we used to verify Resource Quota by taking the sum of
resource limits of the pods, but this was altered to use resource requests. Backwards compatibility
for those pods previously created is preserved because pods that only specify a resource limit have
their resource requests defaulted to match their defined limits. The user is only charged for the
resources they request in the Resource Quota versus their limits because the request is the minimum
amount of resource guaranteed by the cluster during scheduling. For more information on over commit,
see [compute-resources](/docs/user-guide/compute-resources).
- If creating a pod would cause the namespace to exceed any of the limits specified in the
the Resource Quota for that namespace, then the request will fail with HTTP status
code `403 FORBIDDEN`.
- If quota is enabled in a namespace and the user does not specify *requests* on the pod for each
of the resources for which quota is enabled, then the POST of the pod will fail with HTTP
status code `403 FORBIDDEN`. Hint: Use the LimitRange admission controller to force default
values of *limits* (then resource *requests* would be equal to *limits* by default, see
[admission controller](/docs/admin/admission-controllers)) before the quota is checked to avoid this problem.
Examples of policies that could be created using namespaces and quotas are:
- In a cluster with a capacity of 32 GiB RAM, and 16 cores, let team A use 20 Gib and 10 cores,
let B use 10GiB and 4 cores, and hold 2GiB and 2 cores in reserve for future allocation.
- Limit the "testing" namespace to using 1 core and 1GiB RAM. Let the "production" namespace
use any amount.
In the case where the total capacity of the cluster is less than the sum of the quotas of the namespaces,
there may be contention for resources. This is handled on a first-come-first-served basis.
Neither contention nor changes to quota will affect already-running pods.
## Enabling Resource Quota
Resource Quota support is enabled by default for many Kubernetes distributions. It is
enabled when the apiserver `--admission-control=` flag has `ResourceQuota` as
one of its arguments.
Resource Quota is enforced in a particular namespace when there is a
`ResourceQuota` object in that namespace. There should be at most one
`ResourceQuota` object in a namespace.
## Compute Resource Quota
The total sum of [compute resources](/docs/user-guide/compute-resources) requested by pods
in a namespace can be limited. The following compute resource types are supported:
| ResourceName | Description |
| ------------ | ----------- |
| cpu | Total cpu requests of containers |
| memory | Total memory requests of containers
For example, `cpu` quota sums up the `resources.requests.cpu` fields of every
container of every pod in the namespace, and enforces a maximum on that sum.
## Object Count Quota
The number of objects of a given type can be restricted. The following types
are supported:
| ResourceName | Description |
| ------------ | ----------- |
| pods | Total number of pods |
| services | Total number of services |
| replicationcontrollers | Total number of replication controllers |
| resourcequotas | Total number of [resource quotas](/docs/admin/admission-controllers/#resourcequota) |
| secrets | Total number of secrets |
| persistentvolumeclaims | Total number of [persistent volume claims](/docs/user-guide/persistent-volumes/#persistentvolumeclaims) |
For example, `pods` quota counts and enforces a maximum on the number of `pods`
created in a single namespace.
You might want to set a pods quota on a namespace
to avoid the case where a user creates many small pods and exhausts the cluster's
supply of Pod IPs.
## Viewing and Setting Quotas
Kubectl supports creating, updating, and viewing quotas:
```shell
$ kubectl namespace myspace
$ cat <<EOF > quota.json
{
"apiVersion": "v1",
"kind": "ResourceQuota",
"metadata": {
"name": "quota"
},
"spec": {
"hard": {
"memory": "1Gi",
"cpu": "20",
"pods": "10",
"services": "5",
"replicationcontrollers":"20",
"resourcequotas":"1"
}
}
}
EOF
$ kubectl create -f ./quota.json
$ kubectl get quota
NAME
quota
$ kubectl describe quota quota
Name: quota
Resource Used Hard
-------- ---- ----
cpu 0m 20
memory 0 1Gi
pods 5 10
replicationcontrollers 5 20
resourcequotas 1 1
services 3 5
```
## Quota and Cluster Capacity
Resource Quota objects are independent of the Cluster Capacity. They are
expressed in absolute units. So, if you add nodes to your cluster, this does *not*
automatically give each namespace the ability to consume more resources.
Sometimes more complex policies may be desired, such as:
- proportionally divide total cluster resources among several teams.
- allow each tenant to grow resource usage as needed, but have a generous
limit to prevent accidental resource exhaustion.
- detect demand from one namespace, add nodes, and increase quota.
Such policies could be implemented using ResourceQuota as a building-block, by
writing a 'controller' which watches the quota usage and adjusts the quota
hard limits of each namespace according to other signals.
Note that resource quota divides up aggregate cluster resources, but it creates no
restrictions around nodes: pods from several namespaces may run on the same node.
## Example
See a [detailed example for how to use resource quota](/docs/admin/resourcequota/).
## Read More
See [ResourceQuota design doc](https://github.com/kubernetes/kubernetes/blob/{{page.githubbranch}}/docs/design/admission_control_resource_quota.md) for more information.
This example assumes you have a functional Kubernetes setup.
## Step 1: Create a namespace
This example will work in a custom namespace to demonstrate the concepts involved.
Let's create a new namespace called quota-example:
```shell
$ kubectl create -f docs/admin/resourcequota/namespace.yaml
namespace "quota-example" created
$ kubectl get namespaces
NAME LABELS STATUS AGE
default <none> Active 2m
quota-example <none> Active 39s
```
## Step 2: Apply a quota to the namespace
By default, a pod will run with unbounded CPU and memory requests/limits. This means that any pod in the
system will be able to consume as much CPU and memory on the node that executes the pod.
Users may want to restrict how much of the cluster resources a given namespace may consume
across all of its pods in order to manage cluster usage. To do this, a user applies a quota to
a namespace. A quota lets the user set hard limits on the total amount of node resources (cpu, memory)
and API resources (pods, services, etc.) that a namespace may consume. In term of resources, Kubernetes
checks the total resource *requests*, not resource *limits* of all containers/pods in the namespace.
Let's create a simple quota in our namespace:
```shell
$ kubectl create -f docs/admin/resourcequota/quota.yaml --namespace=quota-example
resourcequota "quota" created
```
Once your quota is applied to a namespace, the system will restrict any creation of content
in the namespace until the quota usage has been calculated. This should happen quickly.
You can describe your current quota usage to see what resources are being consumed in your
namespace.
```shell
$ kubectl describe quota quota --namespace=quota-example
Name: quota
Namespace: quota-example
Resource Used Hard
-------- ---- ----
cpu 0 20
memory 0 1Gi
persistentvolumeclaims 0 10
pods 0 10
replicationcontrollers 0 20
resourcequotas 1 1
secrets 1 10
services 0 5
```
## Step 3: Applying default resource requests and limits
Pod authors rarely specify resource requests and limits for their pods.
Since we applied a quota to our project, let's see what happens when an end-user creates a pod that has unbounded
cpu and memory by creating an nginx container.
To demonstrate, lets create a Deployment that runs nginx:
```shell
$ kubectl run nginx --image=nginx --replicas=1 --namespace=quota-example
deployment "nginx" created
```
This creates a Deployment "nginx" with its underlying resource, a ReplicaSet, which handles the creation and deletion of Pod replicas. Now let's look at the pods that were created.
```shell
$ kubectl get pods --namespace=quota-example
NAME READY STATUS RESTARTS AGE
```
What happened? I have no pods! Let's describe the ReplicaSet managed by the nginx Deployment to get a view of what is happening.
Note that `kubectl describe rs` works only on kubernetes cluster >= v1.2. If you are running older versions, use `kubectl describe rc` instead.
```shell
$ kubectl describe rs -l run=nginx --namespace=quota-example
Name: nginx-2040093540
Namespace: quota-example
Image(s): nginx
Selector: pod-template-hash=2040093540,run=nginx
Labels: pod-template-hash=2040093540,run=nginx
Replicas: 0 current / 1 desired
Pods Status: 0 Running / 0 Waiting / 0 Succeeded / 0 Failed
No volumes.
Events:
FirstSeen LastSeen Count From SubobjectPath Type Reason Message
--------- -------- ----- ---- ------------- -------- ------ -------
48s 26s 4 {replicaset-controller } Warning FailedCreate Error creating: pods "nginx-2040093540-" is forbidden: Failed quota: quota: must specify cpu,memory
```
The Kubernetes API server is rejecting the ReplicaSet requests to create a pod because our pods
do not specify any memory usage *request*.
So let's set some default values for the amount of cpu and memory a pod can consume:
```shell
$ kubectl create -f docs/admin/resourcequota/limits.yaml --namespace=quota-example
limitrange "limits" created
$ kubectl describe limits limits --namespace=quota-example
Name: limits
Namespace: quota-example
Type Resource Min Max Default Request Default Limit Max Limit/Request Ratio
---- -------- --- --- --------------- ------------- -----------------------
Container cpu - - 100m 200m -
Container memory - - 256Mi 512Mi -
```
Now any time a pod is created in this namespace, if it has not specified any resource request/limit, the default
amount of cpu and memory per container will be applied, and the request will be used as part of admission control.
Now that we have applied default resource *request* for our namespace, our Deployment should be able to
create its pods.
```shell
$ kubectl get pods --namespace=quota-example
NAME READY STATUS RESTARTS AGE
nginx-2040093540-miohp 1/1 Running 0 5s
```
And if we print out our quota usage in the namespace:
```shell
$ kubectl describe quota quota --namespace=quota-example
Name: quota
Namespace: quota-example
Resource Used Hard
-------- ---- ----
cpu 100m 20
memory 256Mi 1Gi
persistentvolumeclaims 0 10
pods 1 10
replicationcontrollers 1 20
resourcequotas 1 1
secrets 1 10
services 0 5
```
You can now see the pod that was created is consuming explicit amounts of resources (specified by resource *request*), and the usage is being tracked by the Kubernetes system properly.
## Summary
Actions that consume node resources for cpu and memory can be subject to hard quota limits defined by the namespace quota. The resource consumption is measured by resource *request* in pod specification.
Any action that consumes those resources can be tweaked, or can pick up namespace level defaults to meet your end goal.

164
docs/admin/resourcequota/walkthrough.md Normal file
View File

@ -0,0 +1,164 @@
---
---
This example demonstrates how [resource quota](/docs/admin/admission-controllers/#resourcequota) and
[limitsranger](/docs/admin/admission-controllers/#limitranger) can be applied to a Kubernetes namespace.
See [ResourceQuota design doc](https://github.com/kubernetes/kubernetes/blob/{{page.githubbranch}}/docs/design/admission_control_resource_quota.md) for more information.
This example assumes you have a functional Kubernetes setup.
## Step 1: Create a namespace
This example will work in a custom namespace to demonstrate the concepts involved.
Let's create a new namespace called quota-example:
```shell
$ kubectl create namespace quota-example
namespace "quota-example" created
```
Note that `kubectl` commands will print the type and name of the resource created or mutated, which can then be used in subsequent commands:
```shell
$ kubectl get namespaces
NAME STATUS AGE
default Active 50m
quota-example Active 2s
```
## Step 2: Apply a quota to the namespace
By default, a pod will run with unbounded CPU and memory requests/limits. This means that any pod in the
system will be able to consume as much CPU and memory on the node that executes the pod.
Users may want to restrict how much of the cluster resources a given namespace may consume
across all of its pods in order to manage cluster usage. To do this, a user applies a quota to
a namespace. A quota lets the user set hard limits on the total amount of node resources (cpu, memory)
and API resources (pods, services, etc.) that a namespace may consume. In term of resources, Kubernetes
checks the total resource *requests*, not resource *limits* of all containers/pods in the namespace.
Let's create a simple quota in our namespace:
```shell
$ kubectl create -f docs/admin/resourcequota/quota.yaml --namespace=quota-example
resourcequota "quota" created
```
Once your quota is applied to a namespace, the system will restrict any creation of content
in the namespace until the quota usage has been calculated. This should happen quickly.
You can describe your current quota usage to see what resources are being consumed in your
namespace.
```shell
$ kubectl describe quota quota --namespace=quota-example
Name: quota
Namespace: quota-example
Resource Used Hard
-------- ---- ----
cpu 0 20
memory 0 1Gi
persistentvolumeclaims 0 10
pods 0 10
replicationcontrollers 0 20
resourcequotas 1 1
secrets 1 10
services 0 5
```
## Step 3: Applying default resource requests and limits
Pod authors rarely specify resource requests and limits for their pods.
Since we applied a quota to our project, let's see what happens when an end-user creates a pod that has unbounded
cpu and memory by creating an nginx container.
To demonstrate, lets create a Deployment that runs nginx:
```shell
$ kubectl run nginx --image=nginx --replicas=1 --namespace=quota-example
deployment "nginx" created
```
This creates a Deployment "nginx" with its underlying resource, a ReplicaSet, which handles the creation and deletion of Pod replicas. Now let's look at the pods that were created.
```shell
$ kubectl get pods --namespace=quota-example
NAME READY STATUS RESTARTS AGE
```
What happened? I have no pods! Let's describe the ReplicaSet managed by the nginx Deployment to get a view of what is happening.
Note that `kubectl describe rs` works only on kubernetes cluster >= v1.2. If you are running older versions, use `kubectl describe rc` instead.
If you want to obtain the old behavior, use `--generator=run/v1` to create replication controllers. See [`kubectl run`](/docs/user-guide/kubectl/kubectl_run/) for more details.
```shell
$ kubectl describe rs -l run=nginx --namespace=quota-example
Name: nginx-2040093540
Namespace: quota-example
Image(s): nginx
Selector: pod-template-hash=2040093540,run=nginx
Labels: pod-template-hash=2040093540,run=nginx
Replicas: 0 current / 1 desired
Pods Status: 0 Running / 0 Waiting / 0 Succeeded / 0 Failed
No volumes.
Events:
FirstSeen LastSeen Count From SubobjectPath Type Reason Message
--------- -------- ----- ---- ------------- -------- ------ -------
48s 26s 4 {replicaset-controller } Warning FailedCreate Error creating: pods "nginx-2040093540-" is forbidden: Failed quota: quota: must specify cpu,memory
```
The Kubernetes API server is rejecting the ReplicaSet requests to create a pod because our pods
do not specify any memory usage *request*.
So let's set some default values for the amount of cpu and memory a pod can consume:
```shell
$ kubectl create -f docs/admin/resourcequota/limits.yaml --namespace=quota-example
limitrange "limits" created
$ kubectl describe limits limits --namespace=quota-example
Name: limits
Namespace: quota-example
Type Resource Min Max Default Request Default Limit Max Limit/Request Ratio
---- -------- --- --- --------------- ------------- -----------------------
Container cpu - - 100m 200m -
Container memory - - 256Mi 512Mi -
```
Now any time a pod is created in this namespace, if it has not specified any resource request/limit, the default
amount of cpu and memory per container will be applied, and the request will be used as part of admission control.
Now that we have applied default resource *request* for our namespace, our Deployment should be able to
create its pods.
```shell
$ kubectl get pods --namespace=quota-example
NAME READY STATUS RESTARTS AGE
nginx-2040093540-miohp 1/1 Running 0 5s
```
And if we print out our quota usage in the namespace:
```shell
$ kubectl describe quota quota --namespace=quota-example
Name: quota
Namespace: quota-example
Resource Used Hard
-------- ---- ----
cpu 100m 20
memory 256Mi 1Gi
persistentvolumeclaims 0 10
pods 1 10
replicationcontrollers 1 20
resourcequotas 1 1
secrets 1 10
services 0 5
```
You can now see the pod that was created is consuming explicit amounts of resources (specified by resource *request*), and the usage is being tracked by the Kubernetes system properly.
## Summary
Actions that consume node resources for cpu and memory can be subject to hard quota limits defined by the namespace quota. The resource consumption is measured by resource *request* in pod specification.
Any action that consumes those resources can be tweaked, or can pick up namespace level defaults to meet your end goal.

0
docs/getting-started-guides/coreos.md → docs/getting-started-guides/coreos/index.md
View File

0
docs/getting-started-guides/docker-multinode.md → docs/getting-started-guides/docker-multinode/index.md
View File

0
docs/getting-started-guides/mesos.md → docs/getting-started-guides/mesos/index.md
View File

2
docs/user-guide/compute-resources.md
View File

@ -171,7 +171,7 @@ Here you can see from the `Allocated resources` section that that a pod which as
Looking at the `Pods` section, you can see which pods are taking up space on the node.
The [resource quota](/docs/admin/resource-quota) feature can be configured
The [resource quota](/docs/admin/resourcequota/) feature can be configured
to limit the total amount of resources that can be consumed. If used in conjunction
with namespaces, it can prevent one team from hogging all the resources.

0
docs/user-guide/configmap.md → docs/user-guide/configmap/index.md
View File

2
docs/user-guide/debugging-pods-and-replication-controllers.md
View File

@ -45,7 +45,7 @@ case you can try several things:
kubectl get nodes -o yaml | grep '\sname\|cpu\|memory'
kubectl get nodes -o json | jq '.items[] | {name: .metadata.name, cap: .status.capacity}'
The [resource quota](/docs/admin/resource-quota/)
The [resource quota](/docs/admin/resourcequota/)
feature can be configured to limit the total amount of
resources that can be consumed. If used in conjunction with namespaces, it can
prevent one team from hogging all the resources.

33
docs/user-guide/debugging-services.md
View File

@ -40,37 +40,27 @@ OUTPUT
## Running commands in a Pod
For many steps here you will want to see what a `Pod` running in the cluster
sees. Kubernetes does not directly support interactive `Pod`s (yet), but you can
approximate it:
sees. You can start a busybox `Pod` and run commands in it:
```shell
$ cat <<EOF | kubectl create -f -
apiVersion: v1
kind: Pod
metadata:
name: busybox-sleep
spec:
containers:
- name: busybox
image: busybox
args:
- sleep
- "1000000"
EOF
pods/busybox-sleep
$ kubectl run -i --tty busybox --image=busybox --generator="run-pod/v1"
Waiting for pod default/busybox to be running, status is Pending, pod ready: false
Hit enter for command prompt
/ #
```
Now, when you need to run a command (even an interactive shell) in a `Pod`-like
context, use:
If you already have a running `Pod`, run a command in it using:
```shell
$ kubectl exec busybox-sleep -- <COMMAND>
$ kubectl exec <POD-NAME> -c <CONTAINER-NAME> -- <COMMAND>
```
or
or run an interactive shell with:
```shell
$ kubectl exec -ti busybox-sleep sh
$ kubectl exec -ti <POD-NAME> -c <CONTAINER-NAME> sh
/ #
```
@ -88,6 +78,7 @@ $ kubectl run hostnames --image=gcr.io/google_containers/serve_hostname \
deployment "hostnames" created
```
`kubectl` commands will print the type and name of the resource created or mutated, which can then be used in subsequent commands.
Note that this is the same as if you had started the `Deployment` with
the following YAML:

7
docs/user-guide/docker-cli-to-kubectl.md
View File

@ -26,6 +26,13 @@ With kubectl:
# start the pod running nginx
$ kubectl run --image=nginx nginx-app --port=80 --env="DOMAIN=cluster"
deployment "nginx-app" created
```
`kubectl run` creates a Deployment named "nginx" on Kubernetes cluster >= v1.2. If you are running older versions, it creates replication controllers instead.
If you want to obtain the old behavior, use `--generator=run/v1` to create replication controllers. See [`kubectl run`](/docs/user-guide/kubectl/kubectl_run/) for more details.
Note that `kubectl` commands will print the type and name of the resource created or mutated, which can then be used in subsequent commands. Now, we can expose a new Service with the deployment created above:
```shell
# expose a port through with a service
$ kubectl expose deployment nginx-app --port=80 --name=nginx-http
service "nginx-http" exposed

4
docs/user-guide/downward-api.md → docs/user-guide/downward-api/index.md
View File

@ -52,7 +52,7 @@ environment variable they want.
This is an example of a pod that consumes its name and namespace via the
downward API:
{% include code.html language="yaml" file="downward-api/dapi-pod.yaml" ghlink="/docs/user-guide/downward-api/dapi-pod.yaml" %}
{% include code.html language="yaml" file="dapi-pod.yaml" ghlink="/docs/user-guide/downward-api/dapi-pod.yaml" %}
### Downward API volume
@ -86,7 +86,7 @@ In future, it will be possible to specify a specific annotation or label.
This is an example of a pod that consumes its labels and annotations via the downward API volume, labels and annotations are dumped in `/etc/labels` and in `/etc/annotations`, respectively:
{% include code.html language="yaml" file="downward-api/volume/dapi-volume.yaml" ghlink="/docs/user-guide/downward-api/volume/dapi-volume.yaml" %}
{% include code.html language="yaml" file="volume/dapi-volume.yaml" ghlink="/docs/user-guide/downward-api/volume/dapi-volume.yaml" %}
Some more thorough examples:

83
docs/user-guide/horizontal-pod-autoscaler.md
View File

@ -1,83 +0,0 @@
---
---
This document describes the current state of Horizontal Pod Autoscaler in Kubernetes.
## What is Horizontal Pod Autoscaler?
Horizontal pod autoscaling allows to automatically scale the number of pods
in a replication controller, deployment or replica set based on observed CPU utilization.
The autoscaler is implemented as a Kubernetes API resource and a controller.
The resource describes behavior of the controller.
The controller periodically adjusts the number of replicas in a replication controller or deployment
to match the observed average CPU utilization to the target specified by user.
## How does Horizontal Pod Autoscaler work?
![Horizontal Pod Autoscaler diagram](/images/docs/horizontal-pod-autoscaler.svg)
The autoscaler is implemented as a control loop.
It periodically queries CPU utilization for the pods it targets.
(The period of the autoscaler is controlled by `--horizontal-pod-autoscaler-sync-period` flag of controller manager.
The default value is 30 seconds).
Then, it compares the arithmetic mean of the pods' CPU utilization with the target and adjust the number of replicas if needed.
CPU utilization is the recent CPU usage of a pod divided by the sum of CPU requested by the pod's containers.
Please note that if some of the pod's containers do not have CPU request set,
CPU utilization for the pod will not be defined and the autoscaler will not take any action.
Further details of the autoscaling algorithm are given [here](https://github.com/kubernetes/kubernetes/blob/{{page.githubbranch}}/docs/design/horizontal-pod-autoscaler.md#autoscaling-algorithm).
Autoscaler uses heapster to collect CPU utilization.
Therefore, it is required to deploy heapster monitoring in your cluster for autoscaling to work.
Autoscaler accesses corresponding replication controller, deployment or replica set by scale sub-resource.
Scale is an interface which allows to dynamically set the number of replicas and to learn the current state of them.
More details on scale sub-resource can be found [here](https://github.com/kubernetes/kubernetes/blob/{{page.githubbranch}}/docs/design/horizontal-pod-autoscaler.md#scale-subresource).
## API Object
Horizontal pod autoscaler is a top-level resource in the Kubernetes REST API.
In Kubernetes 1.2 HPA was graduated from beta to stable (more details about [api versioning](/docs/api/#api-versioning)) with compatibility between versions.
The stable version is available in `autoscaling/v1` api group whereas the beta vesion is available in `extensions/v1beta1` api group as before.
The transition plan is to depracate beta version of HPA in Kubernetes 1.3 and get it rid off completely in Kubernetes 1.4.
**Warning!** Please have in mind that all Kubernetes components still use HPA in version `extensions/v1beta1` in Kubernetes 1.2.
More details about the API object can be found at
[HorizontalPodAutoscaler Object](https://github.com/kubernetes/kubernetes/blob/{{page.githubbranch}}/docs/design/horizontal-pod-autoscaler.md#horizontalpodautoscaler-object).
## Support for horizontal pod autoscaler in kubectl
Horizontal pod autoscaler, like every API resource, is supported in a standard way by `kubectl`.
We can create a new autoscaler using `kubectl create` command.
We can list autoscalers by `kubectl get hpa` and get detailed description by `kubectl describe hpa`.
Finally, we can delete an autoscaler using `kubectl delete hpa`.
In addition, there is a special `kubectl autoscale` command that allows for easy creation of horizontal pod autoscaler.
For instance, executing `kubectl autoscale rc foo --min=2 --max=5 --cpu-percent=80`
will create an autoscaler for replication controller *foo*, with target CPU utilization set to `80%`
and the number of replicas between 2 and 5.
The detailed documentation of `kubectl autoscale` can be found [here](/docs/user-guide/kubectl/kubectl_autoscale).
## Autoscaling during rolling update
Currently in Kubernetes, it is possible to perform a rolling update by managing replication controllers directly,
or by using the deployment object, which manages the underlying replication controllers for you.
Horizontal pod autoscaler only supports the latter approach: the horizontal pod autoscaler is bound to the deployment object,
it sets the size for the deployment object, and the deployment is responsible for setting sizes of underlying replication controllers.
Horizontal pod autoscaler does not work with rolling update using direct manipulation of replication controllers,
i.e. you cannot bind a horizontal pod autoscaler to a replication controller and do rolling update (e.g. using `kubectl rolling-update`).
The reason this doesn't work is that when rolling update creates a new replication controller,
the horizontal pod autoscaler will not be bound to the new replication controller.
## Further reading
* Design documentation: [Horizontal Pod Autoscaling](https://github.com/kubernetes/kubernetes/blob/{{page.githubbranch}}/docs/design/horizontal-pod-autoscaler.md).
* Manual of autoscale command in kubectl: [kubectl autoscale](/docs/user-guide/kubectl/kubectl_autoscale).
* Usage example of [Horizontal Pod Autoscaler](/docs/user-guide/horizontal-pod-autoscaling/).

174
docs/user-guide/horizontal-pod-autoscaling/index.md
View File

@ -1,147 +1,83 @@
---
---
This document describes the current state of Horizontal Pod Autoscaler in Kubernetes.
## What is Horizontal Pod Autoscaler?
Horizontal pod autoscaling allows to automatically scale the number of pods
in a replication controller, deployment or replica set based on observed CPU utilization.
In the future also other metrics will be supported.
In this document we explain how this feature works by walking you through an example of enabling horizontal pod autoscaling for the php-apache server.
The autoscaler is implemented as a Kubernetes API resource and a controller.
The resource describes behavior of the controller.
The controller periodically adjusts the number of replicas in a replication controller or deployment
to match the observed average CPU utilization to the target specified by user.
## Prerequisites
This example requires a running Kubernetes cluster and kubectl in the version at least 1.2.
[Heapster](https://github.com/kubernetes/heapster) monitoring needs to be deployed in the cluster
as horizontal pod autoscaler uses it to collect metrics
(if you followed [getting started on GCE guide](/docs/getting-started-guides/gce),
heapster monitoring will be turned-on by default).
## How does Horizontal Pod Autoscaler work?
## Step One: Run & expose php-apache server
![Horizontal Pod Autoscaler diagram](/images/docs/horizontal-pod-autoscaler.svg)
To demonstrate horizontal pod autoscaler we will use a custom docker image based on php-apache server.
The image can be found [here](/docs/user-guide/horizontal-pod-autoscaling/image).
It defines [index.php](/docs/user-guide/horizontal-pod-autoscaling/image/index.php) page which performs some CPU intensive computations.
The autoscaler is implemented as a control loop.
It periodically queries CPU utilization for the pods it targets.
(The period of the autoscaler is controlled by `--horizontal-pod-autoscaler-sync-period` flag of controller manager.
The default value is 30 seconds).
Then, it compares the arithmetic mean of the pods' CPU utilization with the target and adjust the number of replicas if needed.
First, we will start a deployment running the image and expose it as a service:
CPU utilization is the recent CPU usage of a pod divided by the sum of CPU requested by the pod's containers.
Please note that if some of the pod's containers do not have CPU request set,
CPU utilization for the pod will not be defined and the autoscaler will not take any action.
Further details of the autoscaling algorithm are given [here](https://github.com/kubernetes/kubernetes/blob/{{page.githubbranch}}/docs/design/horizontal-pod-autoscaler.md#autoscaling-algorithm).
```shell
$ kubectl run php-apache --image=gcr.io/google_containers/hpa-example --requests=cpu=200m --expose --port=80
service "php-apache" created
deployment "php-apache" created
```
Autoscaler uses heapster to collect CPU utilization.
Therefore, it is required to deploy heapster monitoring in your cluster for autoscaling to work.
## Step Two: Create horizontal pod autoscaler
Autoscaler accesses corresponding replication controller, deployment or replica set by scale sub-resource.
Scale is an interface which allows to dynamically set the number of replicas and to learn the current state of them.
More details on scale sub-resource can be found [here](https://github.com/kubernetes/kubernetes/blob/{{page.githubbranch}}/docs/design/horizontal-pod-autoscaler.md#scale-subresource).
Now that the server is running, we will create the autoscaler using
[kubectl autoscale](https://github.com/kubernetes/kubernetes/blob/{{page.githubbranch}}/docs/user-guide/kubectl/kubectl_autoscale.md).
The following command will create a horizontal pod autoscaler that maintains between 1 and 10 replicas of the Pods
controlled by the php-apache deployment we created in the first step of these instructions.
Roughly speaking, the horizontal autoscaler will increase and decrease the number of replicas
(via the deployment) to maintain an average CPU utilization across all Pods of 50%
(since each pod requests 200 milli-cores by [kubectl run](#kubectl-run), this means average CPU usage of 100 milli-cores).
See [here](https://github.com/kubernetes/kubernetes/blob/{{page.githubbranch}}/docs/design/horizontal-pod-autoscaler.md#autoscaling-algorithm) for more details on the algorithm.
```shell
$ kubectl autoscale deployment php-apache --cpu-percent=50 --min=1 --max=10
deployment "php-apache" autoscaled
```
## API Object
We may check the current status of autoscaler by running:
Horizontal pod autoscaler is a top-level resource in the Kubernetes REST API.
In Kubernetes 1.2 HPA was graduated from beta to stable (more details about [api versioning](/docs/api/#api-versioning)) with compatibility between versions.
The stable version is available in `autoscaling/v1` api group whereas the beta vesion is available in `extensions/v1beta1` api group as before.
The transition plan is to depracate beta version of HPA in Kubernetes 1.3 and get it rid off completely in Kubernetes 1.4.
```shell
$ kubectl get hpa
NAME REFERENCE TARGET CURRENT MINPODS MAXPODS AGE
php-apache Deployment/php-apache/scale 50% 0% 1 10 18s
**Warning!** Please have in mind that all Kubernetes components still use HPA in version `extensions/v1beta1` in Kubernetes 1.2.
```
More details about the API object can be found at
[HorizontalPodAutoscaler Object](https://github.com/kubernetes/kubernetes/blob/{{page.githubbranch}}/docs/design/horizontal-pod-autoscaler.md#horizontalpodautoscaler-object).
Please note that the current CPU consumption is 0% as we are not sending any requests to the server
(the ``CURRENT`` column shows the average across all the pods controlled by the corresponding deployment).
## Support for horizontal pod autoscaler in kubectl
## Step Three: Increase load
Horizontal pod autoscaler, like every API resource, is supported in a standard way by `kubectl`.
We can create a new autoscaler using `kubectl create` command.
We can list autoscalers by `kubectl get hpa` and get detailed description by `kubectl describe hpa`.
Finally, we can delete an autoscaler using `kubectl delete hpa`.
Now, we will see how the autoscaler reacts on the increased load on the server.
We will start a container with `busybox` image and an infinite loop of queries to our server inside (please run it in a different terminal):
In addition, there is a special `kubectl autoscale` command that allows for easy creation of horizontal pod autoscaler.
For instance, executing `kubectl autoscale rc foo --min=2 --max=5 --cpu-percent=80`
will create an autoscaler for replication controller *foo*, with target CPU utilization set to `80%`
and the number of replicas between 2 and 5.
The detailed documentation of `kubectl autoscale` can be found [here](/docs/user-guide/kubectl/kubectl_autoscale).
```shell
$ kubectl run -i --tty load-generator --image=busybox /bin/sh
Hit enter for command prompt
## Autoscaling during rolling update
$ while true; do wget -q -O- http://php-apache.default.svc.cluster.local; done
```
Currently in Kubernetes, it is possible to perform a rolling update by managing replication controllers directly,
or by using the deployment object, which manages the underlying replication controllers for you.
Horizontal pod autoscaler only supports the latter approach: the horizontal pod autoscaler is bound to the deployment object,
it sets the size for the deployment object, and the deployment is responsible for setting sizes of underlying replication controllers.
We may examine, how CPU load was increased by executing (it usually takes 1 minute):
Horizontal pod autoscaler does not work with rolling update using direct manipulation of replication controllers,
i.e. you cannot bind a horizontal pod autoscaler to a replication controller and do rolling update (e.g. using `kubectl rolling-update`).
The reason this doesn't work is that when rolling update creates a new replication controller,
the horizontal pod autoscaler will not be bound to the new replication controller.
```shell
$ kubectl get hpa
NAME REFERENCE TARGET CURRENT MINPODS MAXPODS AGE
php-apache Deployment/php-apache/scale 50% 305% 1 10 3m
```
## Further reading
In the case presented here, it bumped CPU consumption to 305% of the request.
As a result, the deployment was resized to 7 replicas:
```shell
$ kubectl get deployment php-apache
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
php-apache 7 7 7 7 19m
```
**Warning!** Sometimes it may take few steps to stabilize the number of replicas.
Since the amount of load is not controlled in any way it may happen that the final number of replicas will
differ from this example.
## Step Four: Stop load
We will finish our example by stopping the user load.
In the terminal where we created container with `busybox` image we will terminate
infinite ``while`` loop by sending `SIGINT` signal,
which can be done using `<Ctrl> + C` combination.
Then we will verify the result state:
```shell
$ kubectl get hpa
NAME REFERENCE TARGET CURRENT MINPODS MAXPODS AGE
php-apache Deployment/php-apache/scale 50% 0% 1 10 11m
$ kubectl get deployment php-apache
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
php-apache 1 1 1 1 27m
```
As we see, in the presented case CPU utilization dropped to 0, and the number of replicas dropped to 1.
**Warning!** Sometimes dropping number of replicas may take few steps.
## Appendix: Other possible scenarios
### Creating the autoscaler from a .yaml file
Instead of using `kubectl autoscale` command we can use the [hpa-php-apache.yaml](/docs/user-guide/horizontal-pod-autoscaling/hpa-php-apache.yaml) file, which looks like this:
```yaml
apiVersion: extensions/v1beta1
kind: HorizontalPodAutoscaler
metadata:
name: php-apache
namespace: default
spec:
scaleRef:
kind: Deployment
name: php-apache
subresource: scale
minReplicas: 1
maxReplicas: 10
cpuUtilization:
targetPercentage: 50
```
We will create the autoscaler by executing the following command:
```shell
$ kubectl create -f docs/user-guide/horizontal-pod-autoscaling/hpa-php-apache.yaml
horizontalpodautoscaler "php-apache" created
```
* Design documentation: [Horizontal Pod Autoscaling](https://github.com/kubernetes/kubernetes/blob/{{page.githubbranch}}/docs/design/horizontal-pod-autoscaler.md).
* Manual of autoscale command in kubectl: [kubectl autoscale](/docs/user-guide/kubectl/kubectl_autoscale).
* Usage example of [Horizontal Pod Autoscaler](/docs/user-guide/horizontal-pod-autoscaling/).

147
docs/user-guide/horizontal-pod-autoscaling/walkthrough.md Normal file
View File

@ -0,0 +1,147 @@
---
---
Horizontal pod autoscaling allows to automatically scale the number of pods
in a replication controller, deployment or replica set based on observed CPU utilization.
In the future also other metrics will be supported.
In this document we explain how this feature works by walking you through an example of enabling horizontal pod autoscaling for the php-apache server.
## Prerequisites
This example requires a running Kubernetes cluster and kubectl in the version at least 1.2.
[Heapster](https://github.com/kubernetes/heapster) monitoring needs to be deployed in the cluster
as horizontal pod autoscaler uses it to collect metrics
(if you followed [getting started on GCE guide](/docs/getting-started-guides/gce),
heapster monitoring will be turned-on by default).
## Step One: Run & expose php-apache server
To demonstrate horizontal pod autoscaler we will use a custom docker image based on php-apache server.
The image can be found [here](/docs/user-guide/horizontal-pod-autoscaling/image).
It defines [index.php](/docs/user-guide/horizontal-pod-autoscaling/image/index.php) page which performs some CPU intensive computations.
First, we will start a deployment running the image and expose it as a service:
```shell
$ kubectl run php-apache --image=gcr.io/google_containers/hpa-example --requests=cpu=200m --expose --port=80
service "php-apache" created
deployment "php-apache" created
```
## Step Two: Create horizontal pod autoscaler
Now that the server is running, we will create the autoscaler using
[kubectl autoscale](https://github.com/kubernetes/kubernetes/blob/{{page.githubbranch}}/docs/user-guide/kubectl/kubectl_autoscale.md).
The following command will create a horizontal pod autoscaler that maintains between 1 and 10 replicas of the Pods
controlled by the php-apache deployment we created in the first step of these instructions.
Roughly speaking, the horizontal autoscaler will increase and decrease the number of replicas
(via the deployment) to maintain an average CPU utilization across all Pods of 50%
(since each pod requests 200 milli-cores by [kubectl run](#kubectl-run), this means average CPU usage of 100 milli-cores).
See [here](https://github.com/kubernetes/kubernetes/blob/{{page.githubbranch}}/docs/design/horizontal-pod-autoscaler.md#autoscaling-algorithm) for more details on the algorithm.
```shell
$ kubectl autoscale deployment php-apache --cpu-percent=50 --min=1 --max=10
deployment "php-apache" autoscaled
```
We may check the current status of autoscaler by running:
```shell
$ kubectl get hpa
NAME REFERENCE TARGET CURRENT MINPODS MAXPODS AGE
php-apache Deployment/php-apache/scale 50% 0% 1 10 18s
```
Please note that the current CPU consumption is 0% as we are not sending any requests to the server
(the ``CURRENT`` column shows the average across all the pods controlled by the corresponding deployment).
## Step Three: Increase load
Now, we will see how the autoscaler reacts on the increased load on the server.
We will start a container with `busybox` image and an infinite loop of queries to our server inside (please run it in a different terminal):
```shell
$ kubectl run -i --tty load-generator --image=busybox /bin/sh
Hit enter for command prompt
$ while true; do wget -q -O- http://php-apache.default.svc.cluster.local; done
```
We may examine, how CPU load was increased by executing (it usually takes 1 minute):
```shell
$ kubectl get hpa
NAME REFERENCE TARGET CURRENT MINPODS MAXPODS AGE
php-apache Deployment/php-apache/scale 50% 305% 1 10 3m
```
In the case presented here, it bumped CPU consumption to 305% of the request.
As a result, the deployment was resized to 7 replicas:
```shell
$ kubectl get deployment php-apache
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
php-apache 7 7 7 7 19m
```
**Warning!** Sometimes it may take few steps to stabilize the number of replicas.
Since the amount of load is not controlled in any way it may happen that the final number of replicas will
differ from this example.
## Step Four: Stop load
We will finish our example by stopping the user load.
In the terminal where we created container with `busybox` image we will terminate
infinite ``while`` loop by sending `SIGINT` signal,
which can be done using `<Ctrl> + C` combination.
Then we will verify the result state:
```shell
$ kubectl get hpa
NAME REFERENCE TARGET CURRENT MINPODS MAXPODS AGE
php-apache Deployment/php-apache/scale 50% 0% 1 10 11m
$ kubectl get deployment php-apache
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
php-apache 1 1 1 1 27m
```
As we see, in the presented case CPU utilization dropped to 0, and the number of replicas dropped to 1.
**Warning!** Sometimes dropping number of replicas may take few steps.
## Appendix: Other possible scenarios
### Creating the autoscaler from a .yaml file
Instead of using `kubectl autoscale` command we can use the [hpa-php-apache.yaml](/docs/user-guide/horizontal-pod-autoscaling/hpa-php-apache.yaml) file, which looks like this:
```yaml
apiVersion: extensions/v1beta1
kind: HorizontalPodAutoscaler
metadata:
name: php-apache
namespace: default
spec:
scaleRef:
kind: Deployment
name: php-apache
subresource: scale
minReplicas: 1
maxReplicas: 10
cpuUtilization:
targetPercentage: 50
```
We will create the autoscaler by executing the following command:
```shell
$ kubectl create -f docs/user-guide/horizontal-pod-autoscaling/hpa-php-apache.yaml
horizontalpodautoscaler "php-apache" created
```

3
docs/user-guide/jobs.md
View File

@ -333,5 +333,4 @@ object, but complete control over what pods are created and how work is assigned
## Future work
Support for creating Jobs at specified times/dates (i.e. cron) is expected in the next minor
release.
Support for creating Jobs at specified times/dates (i.e. cron) is expected in [1.3](https://github.com/kubernetes/kubernetes/pull/11980).

2
docs/user-guide/namespaces.md
View File

@ -13,7 +13,7 @@ need the features they provide.
Namespaces provide a scope for names. Names of resources need to be unique within a namespace, but not across namespaces.
Namespaces are a way to divide cluster resources between multiple uses (via [resource quota](/docs/admin/resource-quota)).
Namespaces are a way to divide cluster resources between multiple uses (via [resource quota](/docs/admin/resourcequota/)).
In future versions of Kubernetes, objects in the same namespace will have the same
access control policies by default.

171
docs/user-guide/persistent-volumes.md
View File

@ -1,171 +0,0 @@
---
---
This document describes the current state of `PersistentVolumes` in Kubernetes. Familiarity with [volumes](/docs/user-guide/volumes/) is suggested.
* TOC
{:toc}
## Introduction
Managing storage is a distinct problem from managing compute. The `PersistentVolume` subsystem provides an API for users and administrators that abstracts details of how storage is provided from how it is consumed. To do this we introduce two new API resources: `PersistentVolume` and `PersistentVolumeClaim`.
A `PersistentVolume` (PV) is a piece of networked storage in the cluster that has been provisioned by an administrator. It is a resource in the cluster just like a node is a cluster resource. PVs are volume plugins like Volumes, but have a lifecycle independent of any individual pod that uses the PV. This API object captures the details of the implementation of the storage, be that NFS, iSCSI, or a cloud-provider-specific storage system.
A `PersistentVolumeClaim` (PVC) is a request for storage by a user. It is similar to a pod. Pods consume node resources and PVCs consume PV resources. Pods can request specific levels of resources (CPU and Memory). Claims can request specific size and access modes (e.g, can be mounted once read/write or many times read-only).
Please see the [detailed walkthrough with working examples](/docs/user-guide/persistent-volumes/).
## Lifecycle of a volume and claim
PVs are resources in the cluster. PVCs are requests for those resources and also act as claim checks to the resource. The interaction between PVs and PVCs follows this lifecycle:
### Provisioning
A cluster administrator will create a number of PVs. They carry the details of the real storage which is available for use by cluster users. They exist in the Kubernetes API and are available for consumption.
### Binding
A user creates a `PersistentVolumeClaim` with a specific amount of storage requested and with certain access modes. A control loop in the master watches for new PVCs, finds a matching PV (if possible), and binds them together. The user will always get at least what they asked for, but the volume may be in excess of what was requested.
Claims will remain unbound indefinitely if a matching volume does not exist. Claims will be bound as matching volumes become available. For example, a cluster provisioned with many 50Gi PVs would not match a PVC requesting 100Gi. The PVC can be bound when a 100Gi PV is added to the cluster.
### Using
Pods use claims as volumes. The cluster inspects the claim to find the bound volume and mounts that volume for a pod. For volumes which support multiple access modes, the user specifies which mode desired when using their claim as a volume in a pod.
Once a user has a claim and that claim is bound, the bound PV belongs to the user for as long as they need it. Users schedule Pods and access their claimed PVs by including a persistentVolumeClaim in their Pod's volumes block. [See below for syntax details](#claims-as-volumes).
### Releasing
When a user is done with their volume, they can delete the PVC objects from the API which allows reclamation of the resource. The volume is considered "released" when the claim is deleted, but it is not yet available for another claim. The previous claimant's data remains on the volume which must be handled according to policy.
### Reclaiming
The reclaim policy for a `PersistentVolume` tells the cluster what to do with the volume after it has been released. Currently, volumes can either be Retained or Recycled. Retention allows for manual reclamation of the resource. For those volume plugins that support it, recycling performs a basic scrub (`rm -rf /thevolume/*`) on the volume and makes it available again for a new claim.
## Types of Persistent Volumes
`PersistentVolume` types are implemented as plugins. Kubernetes currently supports the following plugins:
* GCEPersistentDisk
* AWSElasticBlockStore
* NFS
* iSCSI
* RBD (Ceph Block Device)
* Glusterfs
* HostPath (single node testing only -- local storage is not supported in any way and WILL NOT WORK in a multi-node cluster)
## Persistent Volumes
Each PV contains a spec and status, which is the specification and status of the volume.
```yaml
apiVersion: v1
kind: PersistentVolume
metadata:
name: pv0003
spec:
capacity:
storage: 5Gi
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Recycle
nfs:
path: /tmp
server: 172.17.0.2
```
### Capacity
Generally, a PV will have a specific storage capacity. This is set using the PV's `capacity` attribute. See the Kubernetes [Resource Model](https://github.com/kubernetes/kubernetes/blob/{{page.githubbranch}}/docs/design/resources.md) to understand the units expected by `capacity`.
Currently, storage size is the only resource that can be set or requested. Future attributes may include IOPS, throughput, etc.
### Access Modes
A `PersistentVolume` can be mounted on a host in any way supported by the resource provider. Providers will have different capabilities and each PV's access modes are set to the specific modes supported by that particular volume. For example, NFS can support multiple read/write clients, but a specific NFS PV might be exported on the server as read-only. Each PV gets its own set of access modes describing that specific PV's capabilities.
The access modes are:
* ReadWriteOnce -- the volume can be mounted as read-write by a single node
* ReadOnlyMany -- the volume can be mounted read-only by many nodes
* ReadWriteMany -- the volume can be mounted as read-write by many nodes
In the CLI, the access modes are abbreviated to:
* RWO - ReadWriteOnce
* ROX - ReadOnlyMany
* RWX - ReadWriteMany
> __Important!__ A volume can only be mounted using one access mode at a time, even if it supports many. For example, a GCEPersistentDisk can be mounted as ReadWriteOnce by a single node or ReadOnlyMany by many nodes, but not at the same time.
### Recycling Policy
Current recycling policies are:
* Retain -- manual reclamation
* Recycle -- basic scrub ("rm -rf /thevolume/*")
Currently, NFS and HostPath support recycling.
### Phase
A volume will be in one of the following phases:
* Available -- a free resource that is not yet bound to a claim
* Bound -- the volume is bound to a claim
* Released -- the claim has been deleted, but the resource is not yet reclaimed by the cluster
* Failed -- the volume has failed its automatic reclamation
The CLI will show the name of the PVC bound to the PV.
## PersistentVolumeClaims
Each PVC contains a spec and status, which is the specification and status of the claim.
```yaml
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: myclaim
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 8Gi
```
### Access Modes
Claims use the same conventions as volumes when requesting storage with specific access modes.
### Resources
Claims, like pods, can request specific quantities of a resource. In this case, the request is for storage. The same [resource model](https://github.com/kubernetes/kubernetes/blob/{{page.githubbranch}}/docs/design/resources.md) applies to both volumes and claims.
## Claims As Volumes
Pods access storage by using the claim as a volume. Claims must exist in the same namespace as the pod using the claim. The cluster finds the claim in the pod's namespace and uses it to get the `PersistentVolume` backing the claim. The volume is then mounted to the host and into the pod.
```yaml
kind: Pod
apiVersion: v1
metadata:
name: mypod
spec:
containers:
- name: myfrontend
image: dockerfile/nginx
volumeMounts:
- mountPath: "/var/www/html"
name: mypd
volumes:
- name: mypd
persistentVolumeClaim:
claimName: myclaim
```

223
docs/user-guide/persistent-volumes/index.md
View File

@ -1,96 +1,171 @@
---
---
The purpose of this guide is to help you become familiar with [Kubernetes Persistent Volumes](/docs/user-guide/persistent-volumes/). By the end of the guide, we'll have
nginx serving content from your persistent volume.
This document describes the current state of `PersistentVolumes` in Kubernetes. Familiarity with [volumes](/docs/user-guide/volumes/) is suggested.
You can view all the files for this example in [the docs repo
here](https://github.com/kubernetes/kubernetes.github.io/tree/{{page.docsbranch}}/docs/user-guide/persistent-volumes).
* TOC
{:toc}
This guide assumes knowledge of Kubernetes fundamentals and that you have a cluster up and running.
## Introduction
See [Persistent Storage design document](https://github.com/kubernetes/kubernetes/blob/{{page.githubbranch}}/docs/design/persistent-storage.md) for more information.
Managing storage is a distinct problem from managing compute. The `PersistentVolume` subsystem provides an API for users and administrators that abstracts details of how storage is provided from how it is consumed. To do this we introduce two new API resources: `PersistentVolume` and `PersistentVolumeClaim`.
## Provisioning
A `PersistentVolume` (PV) is a piece of networked storage in the cluster that has been provisioned by an administrator. It is a resource in the cluster just like a node is a cluster resource. PVs are volume plugins like Volumes, but have a lifecycle independent of any individual pod that uses the PV. This API object captures the details of the implementation of the storage, be that NFS, iSCSI, or a cloud-provider-specific storage system.
A Persistent Volume (PV) in Kubernetes represents a real piece of underlying storage capacity in the infrastructure. Cluster administrators
must first create storage (create their Google Compute Engine (GCE) disks, export their NFS shares, etc.) in order for Kubernetes to mount it.
A `PersistentVolumeClaim` (PVC) is a request for storage by a user. It is similar to a pod. Pods consume node resources and PVCs consume PV resources. Pods can request specific levels of resources (CPU and Memory). Claims can request specific size and access modes (e.g, can be mounted once read/write or many times read-only).
PVs are intended for "network volumes" like GCE Persistent Disks, NFS shares, and AWS ElasticBlockStore volumes. `HostPath` was included
for ease of development and testing. You'll create a local `HostPath` for this example.
Please see the [detailed walkthrough with working examples](/docs/user-guide/persistent-volumes/).
> IMPORTANT! For `HostPath` to work, you will need to run a single node cluster. Kubernetes does not
support local storage on the host at this time. There is no guarantee your pod ends up on the correct node where the `HostPath` resides.
```shell
# This will be nginx's webroot
$ mkdir /tmp/data01
$ echo 'I love Kubernetes storage!' > /tmp/data01/index.html
## Lifecycle of a volume and claim
PVs are resources in the cluster. PVCs are requests for those resources and also act as claim checks to the resource. The interaction between PVs and PVCs follows this lifecycle:
### Provisioning
A cluster administrator will create a number of PVs. They carry the details of the real storage which is available for use by cluster users. They exist in the Kubernetes API and are available for consumption.
### Binding
A user creates a `PersistentVolumeClaim` with a specific amount of storage requested and with certain access modes. A control loop in the master watches for new PVCs, finds a matching PV (if possible), and binds them together. The user will always get at least what they asked for, but the volume may be in excess of what was requested.
Claims will remain unbound indefinitely if a matching volume does not exist. Claims will be bound as matching volumes become available. For example, a cluster provisioned with many 50Gi PVs would not match a PVC requesting 100Gi. The PVC can be bound when a 100Gi PV is added to the cluster.
### Using
Pods use claims as volumes. The cluster inspects the claim to find the bound volume and mounts that volume for a pod. For volumes which support multiple access modes, the user specifies which mode desired when using their claim as a volume in a pod.
Once a user has a claim and that claim is bound, the bound PV belongs to the user for as long as they need it. Users schedule Pods and access their claimed PVs by including a persistentVolumeClaim in their Pod's volumes block. [See below for syntax details](#claims-as-volumes).
### Releasing
When a user is done with their volume, they can delete the PVC objects from the API which allows reclamation of the resource. The volume is considered "released" when the claim is deleted, but it is not yet available for another claim. The previous claimant's data remains on the volume which must be handled according to policy.
### Reclaiming
The reclaim policy for a `PersistentVolume` tells the cluster what to do with the volume after it has been released. Currently, volumes can either be Retained or Recycled. Retention allows for manual reclamation of the resource. For those volume plugins that support it, recycling performs a basic scrub (`rm -rf /thevolume/*`) on the volume and makes it available again for a new claim.
## Types of Persistent Volumes
`PersistentVolume` types are implemented as plugins. Kubernetes currently supports the following plugins:
* GCEPersistentDisk
* AWSElasticBlockStore
* NFS
* iSCSI
* RBD (Ceph Block Device)
* Glusterfs
* HostPath (single node testing only -- local storage is not supported in any way and WILL NOT WORK in a multi-node cluster)
## Persistent Volumes
Each PV contains a spec and status, which is the specification and status of the volume.
```yaml
apiVersion: v1
kind: PersistentVolume
metadata:
name: pv0003
spec:
capacity:
storage: 5Gi
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Recycle
nfs:
path: /tmp
server: 172.17.0.2
```
PVs are created by posting them to the API server.
### Capacity
```shell
$ kubectl create -f docs/user-guide/persistent-volumes/volumes/local-01.yaml
NAME LABELS CAPACITY ACCESSMODES STATUS CLAIM REASON
pv0001 type=local 10737418240 RWO Available
Generally, a PV will have a specific storage capacity. This is set using the PV's `capacity` attribute. See the Kubernetes [Resource Model](https://github.com/kubernetes/kubernetes/blob/{{page.githubbranch}}/docs/design/resources.md) to understand the units expected by `capacity`.
Currently, storage size is the only resource that can be set or requested. Future attributes may include IOPS, throughput, etc.
### Access Modes
A `PersistentVolume` can be mounted on a host in any way supported by the resource provider. Providers will have different capabilities and each PV's access modes are set to the specific modes supported by that particular volume. For example, NFS can support multiple read/write clients, but a specific NFS PV might be exported on the server as read-only. Each PV gets its own set of access modes describing that specific PV's capabilities.
The access modes are:
* ReadWriteOnce -- the volume can be mounted as read-write by a single node
* ReadOnlyMany -- the volume can be mounted read-only by many nodes
* ReadWriteMany -- the volume can be mounted as read-write by many nodes
In the CLI, the access modes are abbreviated to:
* RWO - ReadWriteOnce
* ROX - ReadOnlyMany
* RWX - ReadWriteMany
> __Important!__ A volume can only be mounted using one access mode at a time, even if it supports many. For example, a GCEPersistentDisk can be mounted as ReadWriteOnce by a single node or ReadOnlyMany by many nodes, but not at the same time.
### Recycling Policy
Current recycling policies are:
* Retain -- manual reclamation
* Recycle -- basic scrub ("rm -rf /thevolume/*")
Currently, NFS and HostPath support recycling.
### Phase
A volume will be in one of the following phases:
* Available -- a free resource that is not yet bound to a claim
* Bound -- the volume is bound to a claim
* Released -- the claim has been deleted, but the resource is not yet reclaimed by the cluster
* Failed -- the volume has failed its automatic reclamation
The CLI will show the name of the PVC bound to the PV.
## PersistentVolumeClaims
Each PVC contains a spec and status, which is the specification and status of the claim.
```yaml
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: myclaim
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 8Gi
```
## Requesting storage
### Access Modes
Users of Kubernetes request persistent storage for their pods. They don't know how the underlying cluster is provisioned.
They just know they can rely on their claim to storage and can manage its lifecycle independently from the many pods that may use it.
Claims use the same conventions as volumes when requesting storage with specific access modes.
Claims must be created in the same namespace as the pods that use them.
### Resources
```shell
$ kubectl create -f docs/user-guide/persistent-volumes/claims/claim-01.yaml
Claims, like pods, can request specific quantities of a resource. In this case, the request is for storage. The same [resource model](https://github.com/kubernetes/kubernetes/blob/{{page.githubbranch}}/docs/design/resources.md) applies to both volumes and claims.
$ kubectl get pvc
NAME LABELS STATUS VOLUME
myclaim-1 map[]
# A background process will attempt to match this claim to a volume.
# The eventual state of your claim will look something like this:
## Claims As Volumes
$ kubectl get pvc
NAME LABELS STATUS VOLUME
myclaim-1 map[] Bound pv0001
Pods access storage by using the claim as a volume. Claims must exist in the same namespace as the pod using the claim. The cluster finds the claim in the pod's namespace and uses it to get the `PersistentVolume` backing the claim. The volume is then mounted to the host and into the pod.
$ kubectl get pv
NAME LABELS CAPACITY ACCESSMODES STATUS CLAIM REASON
pv0001 type=local 10737418240 RWO Bound default/myclaim-1
```
## Using your claim as a volume
Claims are used as volumes in pods. Kubernetes uses the claim to look up its bound PV. The PV is then exposed to the pod.
```shell
$ kubectl create -f docs/user-guide/persistent-volumes/simpletest/pod.yaml
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
mypod 1/1 Running 0 1h
$ kubectl create -f docs/user-guide/persistent-volumes/simpletest/service.json
$ kubectl get services
NAME CLUSTER_IP EXTERNAL_IP PORT(S) SELECTOR AGE
frontendservice 10.0.0.241 <none> 3000/TCP name=frontendhttp 1d
kubernetes 10.0.0.2 <none> 443/TCP <none> 2d
```
## Next steps
You should be able to query your service endpoint and see what content nginx is serving. A "forbidden" error might mean you
need to disable SELinux (setenforce 0).
```shell
$ curl 10.0.0.241:3000
I love Kubernetes storage!
```
Hopefully this simple guide is enough to get you started with PersistentVolumes. If you have any questions, join the team on [Slack](/docs/troubleshooting/#slack) and ask!
Enjoy!
```yaml
kind: Pod
apiVersion: v1
metadata:
name: mypod
spec:
containers:
- name: myfrontend
image: dockerfile/nginx
volumeMounts:
- mountPath: "/var/www/html"
name: mypd
volumes:
- name: mypd
persistentVolumeClaim:
claimName: myclaim
```

96
docs/user-guide/persistent-volumes/walkthrough.md Normal file
View File

@ -0,0 +1,96 @@
---
---
The purpose of this guide is to help you become familiar with [Kubernetes Persistent Volumes](/docs/user-guide/persistent-volumes/). By the end of the guide, we'll have
nginx serving content from your persistent volume.
You can view all the files for this example in [the docs repo
here](https://github.com/kubernetes/kubernetes.github.io/tree/{{page.docsbranch}}/docs/user-guide/persistent-volumes).
This guide assumes knowledge of Kubernetes fundamentals and that you have a cluster up and running.
See [Persistent Storage design document](https://github.com/kubernetes/kubernetes/blob/{{page.githubbranch}}/docs/design/persistent-storage.md) for more information.
## Provisioning
A Persistent Volume (PV) in Kubernetes represents a real piece of underlying storage capacity in the infrastructure. Cluster administrators
must first create storage (create their Google Compute Engine (GCE) disks, export their NFS shares, etc.) in order for Kubernetes to mount it.
PVs are intended for "network volumes" like GCE Persistent Disks, NFS shares, and AWS ElasticBlockStore volumes. `HostPath` was included
for ease of development and testing. You'll create a local `HostPath` for this example.
> IMPORTANT! For `HostPath` to work, you will need to run a single node cluster. Kubernetes does not
support local storage on the host at this time. There is no guarantee your pod ends up on the correct node where the `HostPath` resides.
```shell
# This will be nginx's webroot
$ mkdir /tmp/data01
$ echo 'I love Kubernetes storage!' > /tmp/data01/index.html
```
PVs are created by posting them to the API server.
```shell
$ kubectl create -f docs/user-guide/persistent-volumes/volumes/local-01.yaml
NAME LABELS CAPACITY ACCESSMODES STATUS CLAIM REASON
pv0001 type=local 10737418240 RWO Available
```
## Requesting storage
Users of Kubernetes request persistent storage for their pods. They don't know how the underlying cluster is provisioned.
They just know they can rely on their claim to storage and can manage its lifecycle independently from the many pods that may use it.
Claims must be created in the same namespace as the pods that use them.
```shell
$ kubectl create -f docs/user-guide/persistent-volumes/claims/claim-01.yaml
$ kubectl get pvc
NAME LABELS STATUS VOLUME
myclaim-1 map[]
# A background process will attempt to match this claim to a volume.
# The eventual state of your claim will look something like this:
$ kubectl get pvc
NAME LABELS STATUS VOLUME
myclaim-1 map[] Bound pv0001
$ kubectl get pv
NAME LABELS CAPACITY ACCESSMODES STATUS CLAIM REASON
pv0001 type=local 10737418240 RWO Bound default/myclaim-1
```
## Using your claim as a volume
Claims are used as volumes in pods. Kubernetes uses the claim to look up its bound PV. The PV is then exposed to the pod.
```shell
$ kubectl create -f docs/user-guide/persistent-volumes/simpletest/pod.yaml
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
mypod 1/1 Running 0 1h
$ kubectl create -f docs/user-guide/persistent-volumes/simpletest/service.json
$ kubectl get services
NAME CLUSTER_IP EXTERNAL_IP PORT(S) SELECTOR AGE
frontendservice 10.0.0.241 <none> 3000/TCP name=frontendhttp 1d
kubernetes 10.0.0.2 <none> 443/TCP <none> 2d
```
## Next steps
You should be able to query your service endpoint and see what content nginx is serving. A "forbidden" error might mean you
need to disable SELinux (setenforce 0).
```shell
$ curl 10.0.0.241:3000
I love Kubernetes storage!
```
Hopefully this simple guide is enough to get you started with PersistentVolumes. If you have any questions, join the team on [Slack](/docs/troubleshooting/#slack) and ask!
Enjoy!

1
docs/user-guide/pods/single-container.md
View File

@ -35,6 +35,7 @@ $ kubectl run NAME
Where:
* `kubectl run` creates a Deployment named "nginx" on Kubernetes cluster >= v1.2. If you are running older versions, it creates replication controllers instead. If you want to obtain the old behavior, use `--generator=run/v1` to create replication controllers. See [`kubectl run`](/docs/user-guide/kubectl/kubectl_run/) for more details.
* `NAME` (required) is the name of the container to create. This value is also
applied as the name of the Deployment, and as the prefix of the
pod name. For example:

672
docs/user-guide/secrets.md
View File

@ -1,672 +0,0 @@
---
---
Objects of type `secret` are intended to hold sensitive information, such as
passwords, OAuth tokens, and ssh keys. Putting this information in a `secret`
is safer and more flexible than putting it verbatim in a `pod` definition or in
a docker image. See [Secrets design document](https://github.com/kubernetes/kubernetes/blob/{{page.githubbranch}}/docs/design/secrets.md) for more information.
* TOC
{:toc}
## Overview of Secrets
A Secret is an object that contains a small amount of sensitive data such as
a password, a token, or a key. Such information might otherwise be put in a
Pod specification or in an image; putting it in a Secret object allows for
more control over how it is used, and reduces the risk of accidental exposure.
Users can create secrets, and the system also creates some secrets.
To use a secret, a pod needs to reference the secret.
A secret can be used with a pod in two ways: as files in a [volume](/docs/user-guide/volumes) mounted on one or more of
its containers, or used by kubelet when pulling images for the pod.
### Built-in Secrets
#### Service Accounts Automatically Create and Attach Secrets with API Credentials
Kubernetes automatically creates secrets which contain credentials for
accessing the API and it automatically modifies your pods to use this type of
secret.
The automatic creation and use of API credentials can be disabled or overridden
if desired. However, if all you need to do is securely access the apiserver,
this is the recommended workflow.
See the [Service Account](/docs/user-guide/service-accounts) documentation for more
information on how Service Accounts work.
### Creating your own Secrets
#### Creating a Secret Using kubectl create secret
Say that some pods need to access a database. The
username and password that the pods should use is in the files
`./username.txt` and `./password.txt` on your local machine.
```shell
# Create files needed for rest of example.
$ echo "admin" > ./username.txt
$ echo "1f2d1e2e67df" > ./password.txt
```
The `kubectl create secret` command
packages these files into a Secret and creates
the object on the Apiserver.
```shell
$ kubectl create secret generic db-user-pass --from-file=./username.txt --from-file=./password.txt
secret "db-user-pass" created
```
You can check that the secret was created like this:
```shell
$ kubectl get secrets
NAME TYPE DATA AGE
db-user-pass Opaque 2 51s
$ kubectl describe secrets/db-user-pass
Name: db-user-pass
Namespace: default
Labels: <none>
Annotations: <none>
Type: Opaque
Data
====
password.txt: 13 bytes
username.txt: 6 bytes
```
Note that neither `get` nor `describe` shows the contents of the file by default.
This is to protect the secret from being exposed accidentally to someone looking
or from being stored in a terminal log.
See [decoding a secret](#decoding-a-secret) for how to see the contents.
#### Creating a Secret Manually
You can also create a secret object in a file first,
in json or yaml format, and then create that object.
Each item must be base64 encoded:
```shell
$ echo "admin" | base64
YWRtaW4K
$ echo "1f2d1e2e67df" | base64
MWYyZDFlMmU2N2RmCg==
```
Now write a secret object that looks like this:
```yaml
apiVersion: v1
kind: Secret
metadata:
name: mysecret
type: Opaque
data:
password: MWYyZDFlMmU2N2RmCg==
username: YWRtaW4K
```
The data field is a map. Its keys must match
[`DNS_SUBDOMAIN`](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/docs/design/identifiers.md), except that leading dots are also
allowed. The values are arbitrary data, encoded using base64.
Create the secret using [`kubectl create`](/docs/user-guide/kubectl/kubectl_create/):
```shell
$ kubectl create -f ./secret.yaml
secret "mysecret" created
```
**Encoding Note:** The serialized JSON and YAML values of secret data are encoded as
base64 strings. Newlines are not valid within these strings and must be
omitted (i.e. do not use `-b` option of `base64` which breaks long lines.)
#### Decoding a Secret
Get back the secret created in the previous section:
```shell
$ kubectl get secret mysecret -o yaml
apiVersion: v1
data:
password: MWYyZDFlMmU2N2RmCg==
username: YWRtaW4K
kind: Secret
metadata:
creationTimestamp: 2016-01-22T18:41:56Z
name: mysecret
namespace: default
resourceVersion: "164619"
selfLink: /api/v1/namespaces/default/secrets/mysecret
uid: cfee02d6-c137-11e5-8d73-42010af00002
type: Opaque
```
Decode the password field:
```shell
$ echo "MWYyZDFlMmU2N2RmCg==" | base64 -D
1f2d1e2e67df
```
### Using Secrets
Secrets can be mounted as data volumes or be exposed as environment variables to
be used by a container in a pod. They can also be used by other parts of the
system, without being directly exposed to the pod. For example, they can hold
credentials that other parts of the system should use to interact with external
systems on your behalf.
#### Using Secrets as Files from a Pod
To consume a Secret in a volume in a Pod:
1. Create a secret or use an existing one. Multiple pods can reference the same secret.
1. Modify your Pod definition to add a volume under `spec.volumes[]`. Name the volume anything, and have a `spec.volumes[].secret.secretName` field equal to the name of the secret object.
1. Add a `spec.containers[].volumeMounts[]` to each container that needs the secret. Specify `spec.containers[].volumeMounts[].readOnly = true` and `spec.containers[].volumeMounts[].mountPath` to an unused directory name where you would like the secrets to appear.
1. Modify your image and/or command line so that the the program looks for files in that directory. Each key in the secret `data` map becomes the filename under `mountPath`.
This is an example of a pod that mounts a secret in a volume:
```json
{
"apiVersion": "v1",
"kind": "Pod",
"metadata": {
"name": "mypod",
"namespace": "myns"
},
"spec": {
"containers": [{
"name": "mypod",
"image": "redis",
"volumeMounts": [{
"name": "foo",
"mountPath": "/etc/foo",
"readOnly": true
}]
}],
"volumes": [{
"name": "foo",
"secret": {
"secretName": "mysecret"
}
}]
}
}
```
Each secret you want to use needs to be referred to in `spec.volumes`.
If there are multiple containers in the pod, then each container needs its
own `volumeMounts` block, but only one `spec.volumes` is needed per secret.
You can package many files into one secret, or use many secrets, whichever is convenient.
See another example of creating a secret and a pod that consumes that secret in a volume [here](/docs/user-guide/secrets/).
##### Consuming Secret Values from Volumes
Inside the container that mounts a secret volume, the secret keys appear as
files and the secret values are base-64 decoded and stored inside these files.
This is the result of commands
executed inside the container from the example above:
```shell
$ ls /etc/foo/
username
password
$ cat /etc/foo/username
admin
$ cat /etc/foo/password
1f2d1e2e67df
```
The program in a container is responsible for reading the secret(s) from the
files.
#### Using Secrets as Environment Variables
To use a secret in an environment variable in a pod:
1. Create a secret or use an existing one. Multiple pods can reference the same secret.
1. Modify your Pod definition in each container that you wish to consume the value of a secret key to add an environment variable for each secret key you wish to consume. The environment variable that consumes the secret key should populate the secret's name and key in `env[x].valueFrom.secretKeyRef`.
1. Modify your image and/or command line so that the the program looks for values in the specified environment variabless
This is an example of a pod that mounts a secret in a volume:
```yaml
apiVersion: v1
kind: Pod
metadata:
name: secret-env-pod
spec:
containers:
- name: mycontainer
image: redis
env:
- name: SECRET_USERNAME
valueFrom:
secretKeyRef:
name: mysecret
key: username
- name: SECRET_PASSWORD
valueFrom:
secretKeyRef:
name: mysecret
key: password
restartPolicy: Never
```
##### Consuming Secret Values from Environment Variables
Inside a container that consumes a secret in an environment variables, the secret keys appear as
normal environment variables containing the base-64 decoded values of the secret data.
This is the result of commands executed inside the container from the example above:
```shell
$ echo $SECRET_USERNAME
admin
$ cat /etc/foo/password
1f2d1e2e67df
```
#### Using imagePullSecrets
An imagePullSecret is a way to pass a secret that contains a Docker (or other) image registry
password to the Kubelet so it can pull a private image on behalf of your Pod.
##### Manually specifying an imagePullSecret
Use of imagePullSecrets is described in the [images documentation](/docs/user-guide/images/#specifying-imagepullsecrets-on-a-pod)
### Arranging for imagePullSecrets to be Automatically Attached
You can manually create an imagePullSecret, and reference it from
a serviceAccount. Any pods created with that serviceAccount
or that default to use that serviceAccount, will get their imagePullSecret
field set to that of the service account.
See [here](/docs/user-guide/service-accounts/#adding-imagepullsecrets-to-a-service-account)
for a detailed explanation of that process.
#### Automatic Mounting of Manually Created Secrets
We plan to extend the service account behavior so that manually created
secrets (e.g. one containing a token for accessing a github account)
can be automatically attached to pods based on their service account.
*This is not implemented yet. See [issue 9902](http://issue.k8s.io/9902).*
## Details
### Restrictions
Secret volume sources are validated to ensure that the specified object
reference actually points to an object of type `Secret`. Therefore, a secret
needs to be created before any pods that depend on it.
Secret API objects reside in a namespace. They can only be referenced by pods
in that same namespace.
Individual secrets are limited to 1MB in size. This is to discourage creation
of very large secrets which would exhaust apiserver and kubelet memory.
However, creation of many smaller secrets could also exhaust memory. More
comprehensive limits on memory usage due to secrets is a planned feature.
Kubelet only supports use of secrets for Pods it gets from the API server.
This includes any pods created using kubectl, or indirectly via a replication
controller. It does not include pods created via the kubelets
`--manifest-url` flag, its `--config` flag, or its REST API (these are
not common ways to create pods.)
### Secret and Pod Lifetime interaction
When a pod is created via the API, there is no check whether a referenced
secret exists. Once a pod is scheduled, the kubelet will try to fetch the
secret value. If the secret cannot be fetched because it does not exist or
because of a temporary lack of connection to the API server, kubelet will
periodically retry. It will report an event about the pod explaining the
reason it is not started yet. Once the a secret is fetched, the kubelet will
create and mount a volume containing it. None of the pod's containers will
start until all the pod's volumes are mounted.
Once the kubelet has started a pod's containers, its secret volumes will not
change, even if the secret resource is modified. To change the secret used,
the original pod must be deleted, and a new pod (perhaps with an identical
`PodSpec`) must be created. Therefore, updating a secret follows the same
workflow as deploying a new container image. The `kubectl rolling-update`
command can be used ([man page](/docs/user-guide/kubectl/kubectl_rolling-update)).
The [`resourceVersion`](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/docs/devel/api-conventions.md#concurrency-control-and-consistency)
of the secret is not specified when it is referenced.
Therefore, if a secret is updated at about the same time as pods are starting,
then it is not defined which version of the secret will be used for the pod. It
is not possible currently to check what resource version of a secret object was
used when a pod was created. It is planned that pods will report this
information, so that a replication controller restarts ones using an old
`resourceVersion`. In the interim, if this is a concern, it is recommended to not
update the data of existing secrets, but to create new ones with distinct names.
## Use cases
### Use-Case: Pod with ssh keys
Create a secret containing some ssh keys:
```shell
$ kubectl create secret generic my-secret --from-file=ssh-privatekey=/path/to/.ssh/id_rsa --from-file=ssh-publickey=/path/to/.ssh/id_rsa.pub
```
**Security Note:** think carefully before sending your own ssh keys: other users of the cluster may have access to the secret. Use a service account which you want to have accessible to all the users with whom you share the kubernetes cluster, and can revoke if they are compromised.
Now we can create a pod which references the secret with the ssh key and
consumes it in a volume:
```json
{
"kind": "Pod",
"apiVersion": "v1",
"metadata": {
"name": "secret-test-pod",
"labels": {
"name": "secret-test"
}
},
"spec": {
"volumes": [
{
"name": "secret-volume",
"secret": {
"secretName": "ssh-key-secret"
}
}
],
"containers": [
{
"name": "ssh-test-container",
"image": "mySshImage",
"volumeMounts": [
{
"name": "secret-volume",
"readOnly": true,
"mountPath": "/etc/secret-volume"
}
]
}
]
}
}
```
When the container's command runs, the pieces of the key will be available in:
```shell
/etc/secret-volume/id-rsa.pub
/etc/secret-volume/id-rsa
```
The container is then free to use the secret data to establish an ssh connection.
### Use-Case: Pods with prod / test credentials
This example illustrates a pod which consumes a secret containing prod
credentials and another pod which consumes a secret with test environment
credentials.
Make the secrets:
```shell
$ kubectl create secret generic prod-db-password --from-literal=user=produser --from-literal=password=Y4nys7f11
secret "prod-db-password" created
$ kubectl create secret generic test-db-password --from-literal=user=testuser --from-literal=password=iluvtests
secret "test-db-password" created
```
Now make the pods:
```json
{
"apiVersion": "v1",
"kind": "List",
"items":
[{
"kind": "Pod",
"apiVersion": "v1",
"metadata": {
"name": "prod-db-client-pod",
"labels": {
"name": "prod-db-client"
}
},
"spec": {
"volumes": [
{
"name": "secret-volume",
"secret": {
"secretName": "prod-db-secret"
}
}
],
"containers": [
{
"name": "db-client-container",
"image": "myClientImage",
"volumeMounts": [
{
"name": "secret-volume",
"readOnly": true,
"mountPath": "/etc/secret-volume"
}
]
}
]
}
},
{
"kind": "Pod",
"apiVersion": "v1",
"metadata": {
"name": "test-db-client-pod",
"labels": {
"name": "test-db-client"
}
},
"spec": {
"volumes": [
{
"name": "secret-volume",
"secret": {
"secretName": "test-db-secret"
}
}
],
"containers": [
{
"name": "db-client-container",
"image": "myClientImage",
"volumeMounts": [
{
"name": "secret-volume",
"readOnly": true,
"mountPath": "/etc/secret-volume"
}
]
}
]
}
}]
}
```
Both containers will have the following files present on their filesystems:
```shell
/etc/secret-volume/username
/etc/secret-volume/password
```
Note how the specs for the two pods differ only in one field; this facilitates
creating pods with different capabilities from a common pod config template.
You could further simplify the base pod specification by using two Service Accounts:
one called, say, `prod-user` with the `prod-db-secret`, and one called, say,
`test-user` with the `test-db-secret`. Then, the pod spec can be shortened to, for example:
```json
{
"kind": "Pod",
"apiVersion": "v1",
"metadata": {
"name": "prod-db-client-pod",
"labels": {
"name": "prod-db-client"
}
},
"spec": {
"serviceAccount": "prod-db-client",
"containers": [
{
"name": "db-client-container",
"image": "myClientImage"
}
]
}
```
### Use-case: Dotfiles in secret volume
In order to make piece of data 'hidden' (ie, in a file whose name begins with a dot character), simply
make that key begin with a dot. For example, when the following secret secret is mounted into a volume:
```json
{
"kind": "Secret",
"apiVersion": "v1",
"metadata": {
"name": "dotfile-secret"
},
"data": {
".secret-file": "dmFsdWUtMg0KDQo=",
}
}
{
"kind": "Pod",
"apiVersion": "v1",
"metadata": {
"name": "secret-dotfiles-pod",
},
"spec": {
"volumes": [
{
"name": "secret-volume",
"secret": {
"secretName": "dotfile-secret"
}
}
],
"containers": [
{
"name": "dotfile-test-container",
"image": "gcr.io/google_containers/busybox",
"command": "ls -l /etc/secret-volume"
"volumeMounts": [
{
"name": "secret-volume",
"readOnly": true,
"mountPath": "/etc/secret-volume"
}
]
}
]
}
}
```
The `secret-volume` will contain a single file, called `.secret-file`, and
the `dotfile-test-container` will have this file present at the path
`/etc/secret-volume/.secret-file`.
**NOTE**
Files beginning with dot characters are hidden from the output of `ls -l`;
you must use `ls -la` to see them when listing directory contents.
### Use-case: Secret visible to one container in a pod
<a name="use-case-two-containers"></a>
Consider a program that needs to handle HTTP requests, do some complex business
logic, and then sign some messages with an HMAC. Because it has complex
application logic, there might be an unnoticed remote file reading exploit in
the server, which could expose the private key to an attacker.
This could be divided into two processes in two containers: a frontend container
which handles user interaction and business logic, but which cannot see the
private key; and a signer container that can see the private key, and responds
to simple signing requests from the frontend (e.g. over localhost networking).
With this partitioned approach, an attacker now has to trick the application
server into doing something rather arbitrary, which may be harder than getting
it to read a file.
<!-- TODO: explain how to do this while still using automation. -->
## Security Properties
### Protections
Because `secret` objects can be created independently of the `pods` that use
them, there is less risk of the secret being exposed during the workflow of
creating, viewing, and editing pods. The system can also take additional
precautions with `secret` objects, such as avoiding writing them to disk where
possible.
A secret is only sent to a node if a pod on that node requires it. It is not
written to disk. It is stored in a tmpfs. It is deleted once the pod that
depends on it is deleted.
On most Kubernetes-project-maintained distributions, communication between user
to the apiserver, and from apiserver to the kubelets, is protected by SSL/TLS.
Secrets are protected when transmitted over these channels.
Secret data on nodes is stored in tmpfs volumes and thus does not come to rest
on the node.
There may be secrets for several pods on the same node. However, only the
secrets that a pod requests are potentially visible within its containers.
Therefore, one Pod does not have access to the secrets of another pod.
There may be several containers in a pod. However, each container in a pod has
to request the secret volume in its `volumeMounts` for it to be visible within
the container. This can be used to construct useful [security partitions at the
Pod level](#use-case-two-containers).
### Risks
- In the API server secret data is stored as plaintext in etcd; therefore:
- Administrators should limit access to etcd to admin users
- Secret data in the API server is at rest on the disk that etcd uses; admins may want to wipe/shred disks
used by etcd when no longer in use
- Applications still need to protect the value of secret after reading it from the volume,
such as not accidentally logging it or transmitting it to an untrusted party.
- A user who can create a pod that uses a secret can also see the value of that secret. Even
if apiserver policy does not allow that user to read the secret object, the user could
run a pod which exposes the secret.
- If multiple replicas of etcd are run, then the secrets will be shared between them.
By default, etcd does not secure peer-to-peer communication with SSL/TLS, though this can be configured.
- It is not possible currently to control which users of a Kubernetes cluster can
access a secret. Support for this is planned.
- Currently, anyone with root on any node can read any secret from the apiserver,
by impersonating the kubelet. It is a planned feature to only send secrets to
nodes that actually require them, to restrict the impact of a root exploit on a
single node.

673
docs/user-guide/secrets/index.md
View File

@ -1,59 +1,672 @@
---
---
Following this example, you will create a secret and a [pod](/docs/user-guide/pods/) that consumes that secret in a [volume](/docs/user-guide/volumes/). See [Secrets design document](https://github.com/kubernetes/kubernetes/blob/{{page.githubbranch}}/docs/design/secrets.md) for more information.
Objects of type `secret` are intended to hold sensitive information, such as
passwords, OAuth tokens, and ssh keys. Putting this information in a `secret`
is safer and more flexible than putting it verbatim in a `pod` definition or in
a docker image. See [Secrets design document](https://github.com/kubernetes/kubernetes/blob/{{page.githubbranch}}/docs/design/secrets.md) for more information.
## Step Zero: Prerequisites
* TOC
{:toc}
This example assumes you have a Kubernetes cluster installed and running, and that you have
installed the `kubectl` command line tool somewhere in your path. Please see the [getting
started](/docs/getting-started-guides/) for installation instructions for your platform.
## Overview of Secrets
## Step One: Create the secret
A Secret is an object that contains a small amount of sensitive data such as
a password, a token, or a key. Such information might otherwise be put in a
Pod specification or in an image; putting it in a Secret object allows for
more control over how it is used, and reduces the risk of accidental exposure.
A secret contains a set of named byte arrays.
Users can create secrets, and the system also creates some secrets.
Use the [`secret.yaml`](/docs/user-guide/secrets/secret.yaml) file to create a secret:
To use a secret, a pod needs to reference the secret.
A secret can be used with a pod in two ways: as files in a [volume](/docs/user-guide/volumes) mounted on one or more of
its containers, or used by kubelet when pulling images for the pod.
### Built-in Secrets
#### Service Accounts Automatically Create and Attach Secrets with API Credentials
Kubernetes automatically creates secrets which contain credentials for
accessing the API and it automatically modifies your pods to use this type of
secret.
The automatic creation and use of API credentials can be disabled or overridden
if desired. However, if all you need to do is securely access the apiserver,
this is the recommended workflow.
See the [Service Account](/docs/user-guide/service-accounts) documentation for more
information on how Service Accounts work.
### Creating your own Secrets
#### Creating a Secret Using kubectl create secret
Say that some pods need to access a database. The
username and password that the pods should use is in the files
`./username.txt` and `./password.txt` on your local machine.
```shell
$ kubectl create -f docs/user-guide/secrets/secret.yaml
# Create files needed for rest of example.
$ echo "admin" > ./username.txt
$ echo "1f2d1e2e67df" > ./password.txt
```
You can use `kubectl` to see information about the secret:
The `kubectl create secret` command
packages these files into a Secret and creates
the object on the Apiserver.
```shell
$ kubectl create secret generic db-user-pass --from-file=./username.txt --from-file=./password.txt
secret "db-user-pass" created
```
You can check that the secret was created like this:
```shell
$ kubectl get secrets
NAME TYPE DATA
test-secret Opaque 2
NAME TYPE DATA AGE
db-user-pass Opaque 2 51s
$ kubectl describe secrets/db-user-pass
Name: db-user-pass
Namespace: default
Labels: <none>
Annotations: <none>
$ kubectl describe secret test-secret
Name: test-secret
Labels: <none>
Annotations: <none>
Type: Opaque
Type: Opaque
Data
====
data-1: 9 bytes
data-2: 11 bytes
password.txt: 13 bytes
username.txt: 6 bytes
```
## Step Two: Create a pod that consumes a secret
Note that neither `get` nor `describe` shows the contents of the file by default.
This is to protect the secret from being exposed accidentally to someone looking
or from being stored in a terminal log.
Pods consume secrets in volumes. Now that you have created a secret, you can create a pod that
consumes it.
See [decoding a secret](#decoding-a-secret) for how to see the contents.
Use the [`secret-pod.yaml`](/docs/user-guide/secrets/secret-pod.yaml) file to create a Pod that consumes the secret.
#### Creating a Secret Manually
You can also create a secret object in a file first,
in json or yaml format, and then create that object.
Each item must be base64 encoded:
```shell
$ kubectl create -f docs/user-guide/secrets/secret-pod.yaml
$ echo "admin" | base64
YWRtaW4K
$ echo "1f2d1e2e67df" | base64
MWYyZDFlMmU2N2RmCg==
```
This pod runs a binary that displays the content of one of the pieces of secret data in the secret
volume:
Now write a secret object that looks like this:
```yaml
apiVersion: v1
kind: Secret
metadata:
name: mysecret
type: Opaque
data:
password: MWYyZDFlMmU2N2RmCg==
username: YWRtaW4K
```
The data field is a map. Its keys must match
[`DNS_SUBDOMAIN`](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/docs/design/identifiers.md), except that leading dots are also
allowed. The values are arbitrary data, encoded using base64.
Create the secret using [`kubectl create`](/docs/user-guide/kubectl/kubectl_create/):
```shell
$ kubectl logs secret-test-pod
2015-04-29T21:17:24.712206409Z content of file "/etc/secret-volume/data-1": value-1
```
$ kubectl create -f ./secret.yaml
secret "mysecret" created
```
**Encoding Note:** The serialized JSON and YAML values of secret data are encoded as
base64 strings. Newlines are not valid within these strings and must be
omitted (i.e. do not use `-b` option of `base64` which breaks long lines.)
#### Decoding a Secret
Get back the secret created in the previous section:
```shell
$ kubectl get secret mysecret -o yaml
apiVersion: v1
data:
password: MWYyZDFlMmU2N2RmCg==
username: YWRtaW4K
kind: Secret
metadata:
creationTimestamp: 2016-01-22T18:41:56Z
name: mysecret
namespace: default
resourceVersion: "164619"
selfLink: /api/v1/namespaces/default/secrets/mysecret
uid: cfee02d6-c137-11e5-8d73-42010af00002
type: Opaque
```
Decode the password field:
```shell
$ echo "MWYyZDFlMmU2N2RmCg==" | base64 -D
1f2d1e2e67df
```
### Using Secrets
Secrets can be mounted as data volumes or be exposed as environment variables to
be used by a container in a pod. They can also be used by other parts of the
system, without being directly exposed to the pod. For example, they can hold
credentials that other parts of the system should use to interact with external
systems on your behalf.
#### Using Secrets as Files from a Pod
To consume a Secret in a volume in a Pod:
1. Create a secret or use an existing one. Multiple pods can reference the same secret.
1. Modify your Pod definition to add a volume under `spec.volumes[]`. Name the volume anything, and have a `spec.volumes[].secret.secretName` field equal to the name of the secret object.
1. Add a `spec.containers[].volumeMounts[]` to each container that needs the secret. Specify `spec.containers[].volumeMounts[].readOnly = true` and `spec.containers[].volumeMounts[].mountPath` to an unused directory name where you would like the secrets to appear.
1. Modify your image and/or command line so that the the program looks for files in that directory. Each key in the secret `data` map becomes the filename under `mountPath`.
This is an example of a pod that mounts a secret in a volume:
```json
{
"apiVersion": "v1",
"kind": "Pod",
"metadata": {
"name": "mypod",
"namespace": "myns"
},
"spec": {
"containers": [{
"name": "mypod",
"image": "redis",
"volumeMounts": [{
"name": "foo",
"mountPath": "/etc/foo",
"readOnly": true
}]
}],
"volumes": [{
"name": "foo",
"secret": {
"secretName": "mysecret"
}
}]
}
}
```
Each secret you want to use needs to be referred to in `spec.volumes`.
If there are multiple containers in the pod, then each container needs its
own `volumeMounts` block, but only one `spec.volumes` is needed per secret.
You can package many files into one secret, or use many secrets, whichever is convenient.
See another example of creating a secret and a pod that consumes that secret in a volume [here](/docs/user-guide/secrets/).
##### Consuming Secret Values from Volumes
Inside the container that mounts a secret volume, the secret keys appear as
files and the secret values are base-64 decoded and stored inside these files.
This is the result of commands
executed inside the container from the example above:
```shell
$ ls /etc/foo/
username
password
$ cat /etc/foo/username
admin
$ cat /etc/foo/password
1f2d1e2e67df
```
The program in a container is responsible for reading the secret(s) from the
files.
#### Using Secrets as Environment Variables
To use a secret in an environment variable in a pod:
1. Create a secret or use an existing one. Multiple pods can reference the same secret.
1. Modify your Pod definition in each container that you wish to consume the value of a secret key to add an environment variable for each secret key you wish to consume. The environment variable that consumes the secret key should populate the secret's name and key in `env[x].valueFrom.secretKeyRef`.
1. Modify your image and/or command line so that the the program looks for values in the specified environment variabless
This is an example of a pod that mounts a secret in a volume:
```yaml
apiVersion: v1
kind: Pod
metadata:
name: secret-env-pod
spec:
containers:
- name: mycontainer
image: redis
env:
- name: SECRET_USERNAME
valueFrom:
secretKeyRef:
name: mysecret
key: username
- name: SECRET_PASSWORD
valueFrom:
secretKeyRef:
name: mysecret
key: password
restartPolicy: Never
```
##### Consuming Secret Values from Environment Variables
Inside a container that consumes a secret in an environment variables, the secret keys appear as
normal environment variables containing the base-64 decoded values of the secret data.
This is the result of commands executed inside the container from the example above:
```shell
$ echo $SECRET_USERNAME
admin
$ cat /etc/foo/password
1f2d1e2e67df
```
#### Using imagePullSecrets
An imagePullSecret is a way to pass a secret that contains a Docker (or other) image registry
password to the Kubelet so it can pull a private image on behalf of your Pod.
##### Manually specifying an imagePullSecret
Use of imagePullSecrets is described in the [images documentation](/docs/user-guide/images/#specifying-imagepullsecrets-on-a-pod)
### Arranging for imagePullSecrets to be Automatically Attached
You can manually create an imagePullSecret, and reference it from
a serviceAccount. Any pods created with that serviceAccount
or that default to use that serviceAccount, will get their imagePullSecret
field set to that of the service account.
See [here](/docs/user-guide/service-accounts/#adding-imagepullsecrets-to-a-service-account)
for a detailed explanation of that process.
#### Automatic Mounting of Manually Created Secrets
We plan to extend the service account behavior so that manually created
secrets (e.g. one containing a token for accessing a github account)
can be automatically attached to pods based on their service account.
*This is not implemented yet. See [issue 9902](http://issue.k8s.io/9902).*
## Details
### Restrictions
Secret volume sources are validated to ensure that the specified object
reference actually points to an object of type `Secret`. Therefore, a secret
needs to be created before any pods that depend on it.
Secret API objects reside in a namespace. They can only be referenced by pods
in that same namespace.
Individual secrets are limited to 1MB in size. This is to discourage creation
of very large secrets which would exhaust apiserver and kubelet memory.
However, creation of many smaller secrets could also exhaust memory. More
comprehensive limits on memory usage due to secrets is a planned feature.
Kubelet only supports use of secrets for Pods it gets from the API server.
This includes any pods created using kubectl, or indirectly via a replication
controller. It does not include pods created via the kubelets
`--manifest-url` flag, its `--config` flag, or its REST API (these are
not common ways to create pods.)
### Secret and Pod Lifetime interaction
When a pod is created via the API, there is no check whether a referenced
secret exists. Once a pod is scheduled, the kubelet will try to fetch the
secret value. If the secret cannot be fetched because it does not exist or
because of a temporary lack of connection to the API server, kubelet will
periodically retry. It will report an event about the pod explaining the
reason it is not started yet. Once the a secret is fetched, the kubelet will
create and mount a volume containing it. None of the pod's containers will
start until all the pod's volumes are mounted.
Once the kubelet has started a pod's containers, its secret volumes will not
change, even if the secret resource is modified. To change the secret used,
the original pod must be deleted, and a new pod (perhaps with an identical
`PodSpec`) must be created. Therefore, updating a secret follows the same
workflow as deploying a new container image. The `kubectl rolling-update`
command can be used ([man page](/docs/user-guide/kubectl/kubectl_rolling-update)).
The [`resourceVersion`](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/docs/devel/api-conventions.md#concurrency-control-and-consistency)
of the secret is not specified when it is referenced.
Therefore, if a secret is updated at about the same time as pods are starting,
then it is not defined which version of the secret will be used for the pod. It
is not possible currently to check what resource version of a secret object was
used when a pod was created. It is planned that pods will report this
information, so that a replication controller restarts ones using an old
`resourceVersion`. In the interim, if this is a concern, it is recommended to not
update the data of existing secrets, but to create new ones with distinct names.
## Use cases
### Use-Case: Pod with ssh keys
Create a secret containing some ssh keys:
```shell
$ kubectl create secret generic my-secret --from-file=ssh-privatekey=/path/to/.ssh/id_rsa --from-file=ssh-publickey=/path/to/.ssh/id_rsa.pub
```
**Security Note:** think carefully before sending your own ssh keys: other users of the cluster may have access to the secret. Use a service account which you want to have accessible to all the users with whom you share the kubernetes cluster, and can revoke if they are compromised.
Now we can create a pod which references the secret with the ssh key and
consumes it in a volume:
```json
{
"kind": "Pod",
"apiVersion": "v1",
"metadata": {
"name": "secret-test-pod",
"labels": {
"name": "secret-test"
}
},
"spec": {
"volumes": [
{
"name": "secret-volume",
"secret": {
"secretName": "ssh-key-secret"
}
}
],
"containers": [
{
"name": "ssh-test-container",
"image": "mySshImage",
"volumeMounts": [
{
"name": "secret-volume",
"readOnly": true,
"mountPath": "/etc/secret-volume"
}
]
}
]
}
}
```
When the container's command runs, the pieces of the key will be available in:
```shell
/etc/secret-volume/id-rsa.pub
/etc/secret-volume/id-rsa
```
The container is then free to use the secret data to establish an ssh connection.
### Use-Case: Pods with prod / test credentials
This example illustrates a pod which consumes a secret containing prod
credentials and another pod which consumes a secret with test environment
credentials.
Make the secrets:
```shell
$ kubectl create secret generic prod-db-password --from-literal=user=produser --from-literal=password=Y4nys7f11
secret "prod-db-password" created
$ kubectl create secret generic test-db-password --from-literal=user=testuser --from-literal=password=iluvtests
secret "test-db-password" created
```
Now make the pods:
```json
{
"apiVersion": "v1",
"kind": "List",
"items":
[{
"kind": "Pod",
"apiVersion": "v1",
"metadata": {
"name": "prod-db-client-pod",
"labels": {
"name": "prod-db-client"
}
},
"spec": {
"volumes": [
{
"name": "secret-volume",
"secret": {
"secretName": "prod-db-secret"
}
}
],
"containers": [
{
"name": "db-client-container",
"image": "myClientImage",
"volumeMounts": [
{
"name": "secret-volume",
"readOnly": true,
"mountPath": "/etc/secret-volume"
}
]
}
]
}
},
{
"kind": "Pod",
"apiVersion": "v1",
"metadata": {
"name": "test-db-client-pod",
"labels": {
"name": "test-db-client"
}
},
"spec": {
"volumes": [
{
"name": "secret-volume",
"secret": {
"secretName": "test-db-secret"
}
}
],
"containers": [
{
"name": "db-client-container",
"image": "myClientImage",
"volumeMounts": [
{
"name": "secret-volume",
"readOnly": true,
"mountPath": "/etc/secret-volume"
}
]
}
]
}
}]
}
```
Both containers will have the following files present on their filesystems:
```shell
/etc/secret-volume/username
/etc/secret-volume/password
```
Note how the specs for the two pods differ only in one field; this facilitates
creating pods with different capabilities from a common pod config template.
You could further simplify the base pod specification by using two Service Accounts:
one called, say, `prod-user` with the `prod-db-secret`, and one called, say,
`test-user` with the `test-db-secret`. Then, the pod spec can be shortened to, for example:
```json
{
"kind": "Pod",
"apiVersion": "v1",
"metadata": {
"name": "prod-db-client-pod",
"labels": {
"name": "prod-db-client"
}
},
"spec": {
"serviceAccount": "prod-db-client",
"containers": [
{
"name": "db-client-container",
"image": "myClientImage"
}
]
}
```
### Use-case: Dotfiles in secret volume
In order to make piece of data 'hidden' (ie, in a file whose name begins with a dot character), simply
make that key begin with a dot. For example, when the following secret secret is mounted into a volume:
```json
{
"kind": "Secret",
"apiVersion": "v1",
"metadata": {
"name": "dotfile-secret"
},
"data": {
".secret-file": "dmFsdWUtMg0KDQo=",
}
}
{
"kind": "Pod",
"apiVersion": "v1",
"metadata": {
"name": "secret-dotfiles-pod",
},
"spec": {
"volumes": [
{
"name": "secret-volume",
"secret": {
"secretName": "dotfile-secret"
}
}
],
"containers": [
{
"name": "dotfile-test-container",
"image": "gcr.io/google_containers/busybox",
"command": "ls -l /etc/secret-volume"
"volumeMounts": [
{
"name": "secret-volume",
"readOnly": true,
"mountPath": "/etc/secret-volume"
}
]
}
]
}
}
```
The `secret-volume` will contain a single file, called `.secret-file`, and
the `dotfile-test-container` will have this file present at the path
`/etc/secret-volume/.secret-file`.
**NOTE**
Files beginning with dot characters are hidden from the output of `ls -l`;
you must use `ls -la` to see them when listing directory contents.
### Use-case: Secret visible to one container in a pod
<a name="use-case-two-containers"></a>
Consider a program that needs to handle HTTP requests, do some complex business
logic, and then sign some messages with an HMAC. Because it has complex
application logic, there might be an unnoticed remote file reading exploit in
the server, which could expose the private key to an attacker.
This could be divided into two processes in two containers: a frontend container
which handles user interaction and business logic, but which cannot see the
private key; and a signer container that can see the private key, and responds
to simple signing requests from the frontend (e.g. over localhost networking).
With this partitioned approach, an attacker now has to trick the application
server into doing something rather arbitrary, which may be harder than getting
it to read a file.
<!-- TODO: explain how to do this while still using automation. -->
## Security Properties
### Protections
Because `secret` objects can be created independently of the `pods` that use
them, there is less risk of the secret being exposed during the workflow of
creating, viewing, and editing pods. The system can also take additional
precautions with `secret` objects, such as avoiding writing them to disk where
possible.
A secret is only sent to a node if a pod on that node requires it. It is not
written to disk. It is stored in a tmpfs. It is deleted once the pod that
depends on it is deleted.
On most Kubernetes-project-maintained distributions, communication between user
to the apiserver, and from apiserver to the kubelets, is protected by SSL/TLS.
Secrets are protected when transmitted over these channels.
Secret data on nodes is stored in tmpfs volumes and thus does not come to rest
on the node.
There may be secrets for several pods on the same node. However, only the
secrets that a pod requests are potentially visible within its containers.
Therefore, one Pod does not have access to the secrets of another pod.
There may be several containers in a pod. However, each container in a pod has
to request the secret volume in its `volumeMounts` for it to be visible within
the container. This can be used to construct useful [security partitions at the
Pod level](#use-case-two-containers).
### Risks
- In the API server secret data is stored as plaintext in etcd; therefore:
- Administrators should limit access to etcd to admin users
- Secret data in the API server is at rest on the disk that etcd uses; admins may want to wipe/shred disks
used by etcd when no longer in use
- Applications still need to protect the value of secret after reading it from the volume,
such as not accidentally logging it or transmitting it to an untrusted party.
- A user who can create a pod that uses a secret can also see the value of that secret. Even
if apiserver policy does not allow that user to read the secret object, the user could
run a pod which exposes the secret.
- If multiple replicas of etcd are run, then the secrets will be shared between them.
By default, etcd does not secure peer-to-peer communication with SSL/TLS, though this can be configured.
- It is not possible currently to control which users of a Kubernetes cluster can
access a secret. Support for this is planned.
- Currently, anyone with root on any node can read any secret from the apiserver,
by impersonating the kubelet. It is a planned feature to only send secrets to
nodes that actually require them, to restrict the impact of a root exploit on a
single node.

59
docs/user-guide/secrets/walkthrough.md Normal file
View File

@ -0,0 +1,59 @@
---
---
Following this example, you will create a secret and a [pod](/docs/user-guide/pods/) that consumes that secret in a [volume](/docs/user-guide/volumes/). See [Secrets design document](https://github.com/kubernetes/kubernetes/blob/{{page.githubbranch}}/docs/design/secrets.md) for more information.
## Step Zero: Prerequisites
This example assumes you have a Kubernetes cluster installed and running, and that you have
installed the `kubectl` command line tool somewhere in your path. Please see the [getting
started](/docs/getting-started-guides/) for installation instructions for your platform.
## Step One: Create the secret
A secret contains a set of named byte arrays.
Use the [`secret.yaml`](/docs/user-guide/secrets/secret.yaml) file to create a secret:
```shell
$ kubectl create -f docs/user-guide/secrets/secret.yaml
```
You can use `kubectl` to see information about the secret:
```shell
$ kubectl get secrets
NAME TYPE DATA
test-secret Opaque 2
$ kubectl describe secret test-secret
Name: test-secret
Labels: <none>
Annotations: <none>
Type: Opaque
Data
====
data-1: 9 bytes
data-2: 11 bytes
```
## Step Two: Create a pod that consumes a secret
Pods consume secrets in volumes. Now that you have created a secret, you can create a pod that
consumes it.
Use the [`secret-pod.yaml`](/docs/user-guide/secrets/secret-pod.yaml) file to create a Pod that consumes the secret.
```shell
$ kubectl create -f docs/user-guide/secrets/secret-pod.yaml
```
This pod runs a binary that displays the content of one of the pieces of secret data in the secret
volume:
```shell
$ kubectl logs secret-test-pod
2015-04-29T21:17:24.712206409Z content of file "/etc/secret-volume/data-1": value-1
```

2
docs/whatisk8s.md
View File

@ -77,7 +77,7 @@ Kubernetes satisfies a number of common needs of applications running in product
* [distributing secrets](/docs/user-guide/secrets/),
* [application health checking](/docs/user-guide/production-pods/#liveness-and-readiness-probes-aka-health-checks),
* [replicating application instances](/docs/user-guide/replication-controller/),
* [horizontal auto-scaling](/docs/user-guide/horizontal-pod-autoscaler/),
* [horizontal auto-scaling](/docs/user-guide/horizontal-pod-autoscaling/),
* [naming and discovery](/docs/user-guide/connecting-applications/),
* [load balancing](/docs/user-guide/services/),
* [rolling updates](/docs/user-guide/update-demo/),