Add Task: Rolling Update a DaemonSet

_data/tasks.yml

@@ -54,3 +54,7 @@ toc:
   - docs/tasks/manage-stateful-set/deleting-a-statefulset.md
   - docs/tasks/manage-stateful-set/debugging-a-statefulset.md
   - docs/tasks/manage-stateful-set/delete-pods.md
+
+- title: Managing Cluster Daemons
+  section:
+  - docs/tasks/manage-daemon/update-daemon-set.md

docs/admin/daemons.md

@@ -121,7 +121,7 @@ the new DaemonSet with the different template will recognize all the existing po
 matching labels.  It will not modify or delete them despite a mismatch in the pod template.
 You will need to force new pod creation by deleting the pod or deleting the node.
 
-You cannot update a DaemonSet.
+Starting from Kubernetes version 1.6, you can [perform a rolling update](/docs/tasks/manage-daemon/update-daemon-set/) on a DaemonSet.
 
 Support for updating DaemonSets and controlled updating of nodes is planned.
 

docs/tasks/index.md

@@ -63,6 +63,9 @@ single thing, typically by giving a short sequence of steps.
 * [Debugging a StatefulSet](/docs/tasks/manage-stateful-set/debugging-a-statefulset/)
 * [Force Deleting StatefulSet Pods](/docs/tasks/manage-stateful-set/delete-pods/)
 
+#### Managing Cluster Daemons
+* [Performing a Rolling Update on a DaemonSet](/docs/tasks/manage-daemon/update-daemon-set/)
+
 ### What's next
 
 If you would like to write a task page, see

docs/tasks/manage-daemon/update-daemon-set.md

@@ -0,0 +1,232 @@
+---
+assignees:
+- janetkuo
+title: Performing a Rolling Update on a DaemonSet
+---
+
+{% capture overview %}
+
+This page shows how to perform a rolling update on a DaemonSet. 
+
+{% endcapture %}
+
+
+{% capture prerequisites %}
+
+* The DaemonSet rolling update feature is only supported in Kubernetes version 1.6 or later.
+
+{% endcapture %}
+
+
+{% capture steps %}
+
+## DaemonSet Update Strategy
+
+DaemonSet has two update strategy types :
+
+* OnDelete: This is the default update strategy for backward-compatibility. With
+  `OnDelete` update strategy, after you update a DaemonSet template, new
+  DaemonSet pods will *only* be created when you manually delete old DaemonSet
+  pods. This is the same behavior of DaemonSet in Kubernetes version 1.5 or
+  before.
+* RollingUpdate: With `RollingUpdate` update strategy, after you update a
+  DaemonSet template, old DaemonSet pods will be killed, and new DaemonSet pods
+  will be created automatically, in a controlled fashion. 
+
+## Limitations 
+
+* DaemonSet rollout history is not supported yet. 
+* DaemonSet rollback is not directly supported in `kubectl` yet. You can rollback
+  by updating DaemonSet template to match the previous version.
+
+## Caveat: Updating DaemonSet created from Kubernetes version 1.5 or before 
+
+If you try to rolling update a DaemonSet that was created from Kubernetes
+version 1.5 or before, a rollout will be triggered when you *first* change the
+DaemonSet update strategy to `RollingUpdate`, even when DaemonSet template isn't
+modified. All existing DaemonSet pods will be restarted. 
+
+To avoid this restart, first find the DaemonSet's current
+`.spec.templateGeneration`:
+
+```shell{% raw %}
+kubectl get ds/<daemonset-name> -o go-template='{{.spec.templateGeneration}}{{"\n"}}'
+{% endraw %}```
+
+The output should be a number *N*. If the output shows `<no value>`, N = 0. 
+
+Then, simply label the existing DaemonSet pods with `pod-template-generation=<N>`
+before changing DaemonSet `.spec.updateStrategy` to `RollingUpdate`:
+
+```shell
+# Replace N with DaemonSet `.spec.templateGeneration`
+# Only run this on pods created from current DaemonSet template
+kubectl label pods -l <daemonset-selector-key>=<daemonset-selector-value> pod-template-generation=<N>
+```
+
+This tells the DaemonSet that the labeled DaemonSet pods are created from current
+DaemonSet template. Therefore, you should only run this command to existing DaemonSet
+pods that are generated from current DaemonSet template. 
+
+Note that you only need to do this when you first change the update strategy of
+a DaemonSet created from Kubernetes version 1.5 or before to `RollingUpdate`,
+but don't want to update its template and start a new rollout yet. 
+
+## Setting DaemonSet update strategy for rolling update
+
+To enable the rolling update feature of a DaemonSet, you must set its
+`.spec.updateStrategy.type` to `RollingUpdate`. 
+
+You may want to set `.spec.updateStrategy.rollingUpdate.maxUnavailable` (default
+to 1) and `.spec.minReadySeconds` (default to 0) as well.
+
+
+### Step 1: Checking DaemonSet `RollingUpdate` update strategy
+
+First, check the update strategy of your DaemonSet, and make sure it's set to 
+RollingUpdate:
+
+```shell{% raw %}
+kubectl get ds/<daemonset-name> -o go-template='{{.spec.updateStrategy.type}}{{"\n"}}'{% endraw %}
+```
+
+If you haven't created the DaemonSet in the system, check your DaemonSet
+manifest with the following command instead:
+
+```shell{% raw %}
+kubectl create -f ds.yaml --dry-run -o go-template='{{.spec.updateStrategy.type}}{{"\n"}}'
+{% endraw %}```
+
+The output from both commands should be:
+
+```shell
+RollingUpdate
+```
+
+If the output isn't `RollingUpdate`, go back and modify the DaemonSet object or
+manifest accordingly.
+
+### Step 2: Creating a DaemonSet with `RollingUpdate` update strategy
+
+If you have already created the DaemonSet, you may skip this step and jump to
+step 3.
+
+After verifying the update strategy of the DaemonSet manifest, create the DaemonSet:
+
+```shell
+kubectl create -f ds.yaml
+```
+
+Alternatively, use `kubectl apply` to create the same DaemonSet if you plan to
+update the DaemonSet with `kubectl apply`. 
+
+```shell
+kubectl apply -f ds.yaml
+```
+
+### Step 3: Updating a DaemonSet template 
+
+Any updates to a `RollingUpdate` DaemonSet `.spec.template` will trigger a rolling
+update. This can be done with several different `kubectl` commands.
+
+#### Declarative commands
+
+If you update DaemonSets using
+[configuration files](/docs/tutorials/object-management-kubectl/declarative-object-management-configuration/),
+use `kubectl apply`:
+
+```shell
+kubectl apply -f ds-v2.yaml
+```
+
+#### Imperative commands
+
+If you update DaemonSets using
+[imperative commands](/docs/tutorials/object-management-kubectl/imperative-object-management-command/),
+use `kubectl edit` or `kubectl patch`:
+
+```shell
+kubectl edit ds/<daemonset-name>
+```
+
+```shell
+kubectl patch ds/<daemonset-name> -p=<strategic-merge-patch>
+```
+
+##### Updating only the container image 
+
+If you just need to update the container image in the DaemonSet template, i.e.
+`.spec.template.spec.containers[*].image`, use `kubectl set image`:
+
+```shell
+kubectl set image ds/<daemonset-name> <container-name>=<container-new-image>
+```
+
+### Step 4: Watching the rolling update status
+
+Finally, watch the rollout status of the latest DaemonSet rolling update:
+
+```shell 
+kubectl rollout status ds/<daemonset-name> 
+```
+
+When the rollout is complete, the output is similar to this:
+
+```shell
+daemon set "<daemonset-name>" successfully rolled out
+```
+
+## Troubleshooting 
+
+### DaemonSet rolling update is stuck
+
+Sometimes, a DaemonSet rolling update may be stuck. Here are some possible
+causes:
+
+#### Some nodes run out of resources
+
+The rollout is stuck because new DaemonSet pods can't be scheduled on at least one
+node. This is possible when the node is 
+[running out of resources](/docs/concepts/cluster-administration/out-of-resource/).
+
+When this happens, find the nodes that don't have the DaemonSet pods scheduled on
+by comparing the output of `kubectl get nodes` and the output of:
+
+```shell
+kubectl get pods -l <daemonset-selector-key>=<daemonset-selector-value> -o wide 
+```
+
+Once you've found those nodes, delete some non-DaemonSet pods from the node to
+make room for new DaemonSet pods. Note that this will cause service disruption 
+if the deleted pods are not controlled by any controllers, or if the pods aren't
+replicated. This doesn't respect 
+[PodDisruptionBudget](/docs/tasks/configure-pod-container/configure-pod-disruption-budget/)
+either.
+
+#### Broken rollout
+
+If the recent DaemonSet template update is broken, for example, the container is
+crash looping, or the container image doesn't exist (often due to a typo),
+DaemonSet rollout won't progress. 
+
+To fix this, just update the DaemonSet template again. New rollout won't be
+blocked by previous unhealthy rollouts. 
+
+#### Clock skew
+
+If `.spec.minReadySeconds` is specified in the DaemonSet, clock skew between 
+master and nodes will make DaemonSet unable to detect the right rollout
+progress. 
+
+
+{% endcapture %}
+
+
+{% capture whatsnext %}
+
+*TODO: Link to "Task: Creating a DaemonSet to adopt existing DaemonSet pods"*
+
+{% endcapture %}
+
+
+{% include templates/task.md %}