From c8abb1c5f84986c77a1c7d1b4f1db79751ed2846 Mon Sep 17 00:00:00 2001 From: Bryan Boreham Date: Wed, 28 Sep 2016 16:30:41 +0100 Subject: [PATCH 01/45] Clarify what kubenet does and does not do --- docs/admin/network-plugins.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/docs/admin/network-plugins.md b/docs/admin/network-plugins.md index c0397016da..0d7f8981e2 100644 --- a/docs/admin/network-plugins.md +++ b/docs/admin/network-plugins.md @@ -40,7 +40,11 @@ The CNI plugin is selected by passing Kubelet the `--network-plugin=cni` command ### kubenet -The Linux-only kubenet plugin provides functionality similar to the `--configure-cbr0` kubelet command-line option. It creates a Linux bridge named `cbr0` and creates a veth pair for each pod with the host end of each pair connected to `cbr0`. The pod end of the pair is assigned an IP address allocated from a range assigned to the node either through configuration or by the controller-manager. `cbr0` is assigned an MTU matching the smallest MTU of an enabled normal interface on the host. The kubenet plugin is currently mutually exclusive with, and will eventually replace, the --configure-cbr0 option. It is also currently incompatible with the flannel experimental overlay. +Kubenet is intended to be a very basic, simple network plugin, on Linux only. It does not, of itself, implement more advanced features like cross-node networking or network policy. It is typically used together with a cloud provider that sets up routing rules for communication between nodes, or in single-node environments. + +Kubenet creates a Linux bridge named `cbr0` and creates a veth pair for each pod with the host end of each pair connected to `cbr0`. The pod end of the pair is assigned an IP address allocated from a range assigned to the node either through configuration or by the controller-manager. `cbr0` is assigned an MTU matching the smallest MTU of an enabled normal interface on the host. + +The kubenet plugin is currently mutually exclusive with, and will eventually replace, the --configure-cbr0 option. It is also currently incompatible with the flannel experimental overlay. The plugin requires a few things: From 8b4e7e8bc611fddc7250a65a8ad4a265a2ebaa33 Mon Sep 17 00:00:00 2001 From: Keith Wansbrough Date: Tue, 18 Oct 2016 15:22:13 +0100 Subject: [PATCH 02/45] Instructions for Windows download of kubectl. The binary name is kubectl.exe in this case, not kubectl, which is not immediately obvious. --- docs/getting-started-guides/minikube.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/getting-started-guides/minikube.md b/docs/getting-started-guides/minikube.md index 020fc5e257..5d3c69f6b1 100644 --- a/docs/getting-started-guides/minikube.md +++ b/docs/getting-started-guides/minikube.md @@ -80,6 +80,8 @@ curl -Lo kubectl http://storage.googleapis.com/kubernetes-release/release/{{page curl -Lo kubectl http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/darwin/amd64/kubectl && chmod +x kubectl && sudo mv kubectl /usr/local/bin/ # OS X/386 curl -Lo kubectl http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/darwin/386/kubectl && chmod +x kubectl && sudo mv kubectl /usr/local/bin/ +# Windows +curl -Lo kubectl.exe http://storage.googleapis.com/kubernetes-release/release/v1.3.0/bin/windows/amd64/kubectl.exe ``` The generic download path is: From 0903054357418e6a93746eee498f12ca7413bda4 Mon Sep 17 00:00:00 2001 From: Keith Wansbrough Date: Wed, 19 Oct 2016 12:03:28 +0100 Subject: [PATCH 03/45] Don't hardcode kubectl version. --- docs/getting-started-guides/minikube.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/getting-started-guides/minikube.md b/docs/getting-started-guides/minikube.md index 5d3c69f6b1..e72a70299a 100644 --- a/docs/getting-started-guides/minikube.md +++ b/docs/getting-started-guides/minikube.md @@ -81,7 +81,7 @@ curl -Lo kubectl http://storage.googleapis.com/kubernetes-release/release/{{page # OS X/386 curl -Lo kubectl http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/darwin/386/kubectl && chmod +x kubectl && sudo mv kubectl /usr/local/bin/ # Windows -curl -Lo kubectl.exe http://storage.googleapis.com/kubernetes-release/release/v1.3.0/bin/windows/amd64/kubectl.exe +curl -Lo kubectl.exe http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/windows/amd64/kubectl.exe ``` The generic download path is: From eb5f7add4b1fa40e5a47f6825cef7c547ab08128 Mon Sep 17 00:00:00 2001 From: Keith Wansbrough Date: Wed, 19 Oct 2016 12:04:25 +0100 Subject: [PATCH 04/45] Suggest alternative to curl for Windows. --- docs/getting-started-guides/minikube.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/getting-started-guides/minikube.md b/docs/getting-started-guides/minikube.md index e72a70299a..e8b75b99d8 100644 --- a/docs/getting-started-guides/minikube.md +++ b/docs/getting-started-guides/minikube.md @@ -80,7 +80,7 @@ curl -Lo kubectl http://storage.googleapis.com/kubernetes-release/release/{{page curl -Lo kubectl http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/darwin/amd64/kubectl && chmod +x kubectl && sudo mv kubectl /usr/local/bin/ # OS X/386 curl -Lo kubectl http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/darwin/386/kubectl && chmod +x kubectl && sudo mv kubectl /usr/local/bin/ -# Windows +# Windows (using curl; or use your browser "save link as...") curl -Lo kubectl.exe http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/windows/amd64/kubectl.exe ``` From 33cdc55903b43bd204bf4e0cdfbab184755714cf Mon Sep 17 00:00:00 2001 From: Keith Wansbrough Date: Wed, 19 Oct 2016 14:01:30 +0100 Subject: [PATCH 05/45] Just give a simple link for Windows kubectl. --- docs/getting-started-guides/minikube.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/getting-started-guides/minikube.md b/docs/getting-started-guides/minikube.md index e8b75b99d8..ee827bab55 100644 --- a/docs/getting-started-guides/minikube.md +++ b/docs/getting-started-guides/minikube.md @@ -80,10 +80,10 @@ curl -Lo kubectl http://storage.googleapis.com/kubernetes-release/release/{{page curl -Lo kubectl http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/darwin/amd64/kubectl && chmod +x kubectl && sudo mv kubectl /usr/local/bin/ # OS X/386 curl -Lo kubectl http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/darwin/386/kubectl && chmod +x kubectl && sudo mv kubectl /usr/local/bin/ -# Windows (using curl; or use your browser "save link as...") -curl -Lo kubectl.exe http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/windows/amd64/kubectl.exe ``` +For Windows, download [kubectl.exe](http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/windows/amd64/kubectl.exe) and save it to a location on your PATH. + The generic download path is: ``` https://storage.googleapis.com/kubernetes-release/release/${K8S_VERSION}/bin/${GOOS}/${GOARCH}/${K8S_BINARY} From 441043a22a59d34b34cbdcdf8dc2437c552ef5c8 Mon Sep 17 00:00:00 2001 From: Justin Santa Barbara Date: Wed, 19 Oct 2016 13:08:21 -0400 Subject: [PATCH 06/45] Instructions for how to download kubectl --- _data/guides.yml | 2 + docs/getting-started-guides/kubectl.md | 63 ++++++++++++++++++++++++++ 2 files changed, 65 insertions(+) create mode 100644 docs/getting-started-guides/kubectl.md diff --git a/_data/guides.yml b/_data/guides.yml index 40d47b08d6..e77e8e1b01 100644 --- a/_data/guides.yml +++ b/_data/guides.yml @@ -12,6 +12,8 @@ toc: path: /docs/getting-started-guides/kubeadm/ - title: Hello World on Google Container Engine path: /docs/hellonode/ + - title: Downloading kubectl + path: /docs/getting-started-guides/kubectl/ - title: Downloading or Building Kubernetes path: /docs/getting-started-guides/binary_release/ - title: Online Training Course diff --git a/docs/getting-started-guides/kubectl.md b/docs/getting-started-guides/kubectl.md new file mode 100644 index 0000000000..6c5043e744 --- /dev/null +++ b/docs/getting-started-guides/kubectl.md @@ -0,0 +1,63 @@ +--- +--- + + + +## Overview + +kubectl is the command line tool you will use when interacting with kubernetes. + +You should use a version of kubectl that is newer than your server. `kubectl version` will print +the server and client versions. + +## Download a release + +Download kubectl from the [official Kubernetes releases](https://console.cloud.google.com/storage/browser/kubernetes-release/release/): + +On MacOS: + +``` +wget https://storage.googleapis.com/kubernetes-release/release/v1.4.3/bin/darwin/amd64/kubectl +chmod +x kubectl +mv kubectl /usr/local/bin/kubectl +``` + +On Linux: + +``` +wget https://storage.googleapis.com/kubernetes-release/release/v1.4.3/bin/linux/amd64/kubectl +chmod +x kubectl +mv kubectl /usr/local/bin/kubectl +``` + + +You may need to `sudo` the `mv`; you can put it anywhere in your `PATH` - some people prefer to install to `~/bin`. + + +## Alternatives + +### Download as part of the Google Cloud SDK + +kubectl can be installed as part of the Google Cloud SDK: + +First install the [Google Cloud SDK](https://cloud.google.com/sdk/). + +After Google Cloud SDK installs, run the following command to install `kubectl`: + +```shell +gcloud components install kubectl +``` + +Do check that the version is sufficiently up-to-date using `kubectl version --client`. + +### Install with brew + +If you are on MacOS and using brew, you can install with: + +```shell +brew install kubectl +``` + +Do check that the version is sufficiently up-to-date using `kubectl version --client`. From 4f0d6c160907044e6ce4c4940b3d69ee0e952c77 Mon Sep 17 00:00:00 2001 From: Bryan Boreham Date: Thu, 20 Oct 2016 09:48:17 +0100 Subject: [PATCH 07/45] Adjust wording per review comments --- docs/admin/network-plugins.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/admin/network-plugins.md b/docs/admin/network-plugins.md index 0d7f8981e2..b20e788b1a 100644 --- a/docs/admin/network-plugins.md +++ b/docs/admin/network-plugins.md @@ -40,11 +40,11 @@ The CNI plugin is selected by passing Kubelet the `--network-plugin=cni` command ### kubenet -Kubenet is intended to be a very basic, simple network plugin, on Linux only. It does not, of itself, implement more advanced features like cross-node networking or network policy. It is typically used together with a cloud provider that sets up routing rules for communication between nodes, or in single-node environments. +Kubenet is a very basic, simple network plugin, on Linux only. It does not, of itself, implement more advanced features like cross-node networking or network policy. It is typically used together with a cloud provider that sets up routing rules for communication between nodes, or in single-node environments. Kubenet creates a Linux bridge named `cbr0` and creates a veth pair for each pod with the host end of each pair connected to `cbr0`. The pod end of the pair is assigned an IP address allocated from a range assigned to the node either through configuration or by the controller-manager. `cbr0` is assigned an MTU matching the smallest MTU of an enabled normal interface on the host. -The kubenet plugin is currently mutually exclusive with, and will eventually replace, the --configure-cbr0 option. It is also currently incompatible with the flannel experimental overlay. +The kubenet plugin is mutually exclusive with the --configure-cbr0 option. The plugin requires a few things: From 0ebd20ccfbb56b1e3730ad5d559de5e7377d12a8 Mon Sep 17 00:00:00 2001 From: Justin Santa Barbara Date: Fri, 21 Oct 2016 21:05:23 -0400 Subject: [PATCH 08/45] Fixes per code review --- _data/guides.yml | 2 +- docs/getting-started-guides/kubectl.md | 21 ++++++++++++--------- 2 files changed, 13 insertions(+), 10 deletions(-) diff --git a/_data/guides.yml b/_data/guides.yml index e77e8e1b01..a661cbfb9a 100644 --- a/_data/guides.yml +++ b/_data/guides.yml @@ -12,7 +12,7 @@ toc: path: /docs/getting-started-guides/kubeadm/ - title: Hello World on Google Container Engine path: /docs/hellonode/ - - title: Downloading kubectl + - title: Installing kubectl path: /docs/getting-started-guides/kubectl/ - title: Downloading or Building Kubernetes path: /docs/getting-started-guides/binary_release/ diff --git a/docs/getting-started-guides/kubectl.md b/docs/getting-started-guides/kubectl.md index 6c5043e744..0498ceb646 100644 --- a/docs/getting-started-guides/kubectl.md +++ b/docs/getting-started-guides/kubectl.md @@ -7,10 +7,12 @@ li>.highlighter-rouge {position:relative; top:3px;} ## Overview -kubectl is the command line tool you will use when interacting with kubernetes. +kubectl is the command line tool you use to interact with Kubernetes clusters. -You should use a version of kubectl that is newer than your server. `kubectl version` will print -the server and client versions. +You should use a version of kubectl that is at least as new as your server. +`kubectl version` will print the server and client versions. Using the same version of kubectl +as your server naturally works; using a newer kubectl than your server also works; but if you use +an older kubectl with a newer server you may see odd validation errors . ## Download a release @@ -18,16 +20,16 @@ Download kubectl from the [official Kubernetes releases](https://console.cloud.g On MacOS: -``` -wget https://storage.googleapis.com/kubernetes-release/release/v1.4.3/bin/darwin/amd64/kubectl +```shell +wget https://storage.googleapis.com/kubernetes-release/release/v1.4.4/bin/darwin/amd64/kubectl chmod +x kubectl mv kubectl /usr/local/bin/kubectl ``` On Linux: -``` -wget https://storage.googleapis.com/kubernetes-release/release/v1.4.3/bin/linux/amd64/kubectl +```shell +wget https://storage.googleapis.com/kubernetes-release/release/v1.4.4/bin/linux/amd64/kubectl chmod +x kubectl mv kubectl /usr/local/bin/kubectl ``` @@ -50,7 +52,7 @@ After Google Cloud SDK installs, run the following command to install `kubectl`: gcloud components install kubectl ``` -Do check that the version is sufficiently up-to-date using `kubectl version --client`. +Do check that the version is sufficiently up-to-date using `kubectl version`. ### Install with brew @@ -60,4 +62,5 @@ If you are on MacOS and using brew, you can install with: brew install kubectl ``` -Do check that the version is sufficiently up-to-date using `kubectl version --client`. +The homebrew project is independent from kubernetes, so do check that the version is +sufficiently up-to-date using `kubectl version`. \ No newline at end of file From 689c4b02b48783991c2e8cdf2e6c2a7446b8546d Mon Sep 17 00:00:00 2001 From: Justin Santa Barbara Date: Fri, 21 Oct 2016 21:05:38 -0400 Subject: [PATCH 09/45] Add kubectl shell autocompletion section --- docs/getting-started-guides/kubectl.md | 43 +++++++++++++++++++++++++- 1 file changed, 42 insertions(+), 1 deletion(-) diff --git a/docs/getting-started-guides/kubectl.md b/docs/getting-started-guides/kubectl.md index 0498ceb646..b346b75c7d 100644 --- a/docs/getting-started-guides/kubectl.md +++ b/docs/getting-started-guides/kubectl.md @@ -63,4 +63,45 @@ brew install kubectl ``` The homebrew project is independent from kubernetes, so do check that the version is -sufficiently up-to-date using `kubectl version`. \ No newline at end of file +sufficiently up-to-date using `kubectl version`. + + +# Enabling shell autocompletion + +kubectl includes autocompletion support, which can save a lot of typing! + +The completion script itself is generated by kubectl, so you typically just need to invoke it from your profile. + +Common examples are provided here, but for more details please consult `kubectl completion -h` + +## On Linux, using bash + +To add it to your current shell: `source <(kubectl completion bash)` + +To add kubectl autocompletion to your profile (so it is automatically loaded in future shells): + +```shell +echo "source <(kubectl completion bash)" >> ~/.bashrc +``` + +## On MacOS, using bash + +On MacOS, you will need to install the bash-completion support first: + +```shell +brew install bash-completion +``` + +To add it to your current shell: + +```shell +source $(brew --prefix)/etc/bash_completion +source <(kubectl completion bash) +``` + +To add kubectl autocompletion to your profile (so it is automatically loaded in future shells): + +```shell +echo "source $(brew --prefix)/etc/bash_completion" >> ~/.bashrc +echo "source <(kubectl completion bash)" >> ~/.bashrc +``` From 500aec5eedf60f574d4022b65f162f6f6dd0344c Mon Sep 17 00:00:00 2001 From: Justin Santa Barbara Date: Fri, 21 Oct 2016 21:39:25 -0400 Subject: [PATCH 10/45] A few updates based on MacOS testing --- docs/getting-started-guides/kubectl.md | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/docs/getting-started-guides/kubectl.md b/docs/getting-started-guides/kubectl.md index b346b75c7d..bd2512707b 100644 --- a/docs/getting-started-guides/kubectl.md +++ b/docs/getting-started-guides/kubectl.md @@ -102,6 +102,9 @@ source <(kubectl completion bash) To add kubectl autocompletion to your profile (so it is automatically loaded in future shells): ```shell -echo "source $(brew --prefix)/etc/bash_completion" >> ~/.bashrc -echo "source <(kubectl completion bash)" >> ~/.bashrc +echo "source $(brew --prefix)/etc/bash_completion" >> ~/.bash_profile +echo "source <(kubectl completion bash)" >> ~/.bash_profile ``` + +Please note that this only appears to work currently if you install using `brew install kubectl`, +and not if you downloaded kubectl directly. \ No newline at end of file From b72518e69c048db1e1f1b76a1001ca33a2eb49eb Mon Sep 17 00:00:00 2001 From: Zihong Zheng Date: Fri, 28 Oct 2016 09:39:32 -0700 Subject: [PATCH 11/45] Add troubleshooting section to kube-dns readme --- docs/admin/dns.md | 143 +++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 140 insertions(+), 3 deletions(-) diff --git a/docs/admin/dns.md b/docs/admin/dns.md index cc132201aa..b87d3196bc 100644 --- a/docs/admin/dns.md +++ b/docs/admin/dns.md @@ -124,7 +124,7 @@ With v1.3, the following annotations are deprecated: `pod.beta.kubernetes.io/hos ## How do I test if it is working? -### Create a simple Pod to use as a test environment. +### Create a simple Pod to use as a test environment Create a file named busybox.yaml with the following contents: @@ -152,7 +152,7 @@ Then create a pod using this file: kubectl create -f busybox.yaml ``` -### Wait for this pod to go into the running state. +### Wait for this pod to go into the running state You can get its status with: ``` @@ -165,7 +165,7 @@ NAME READY STATUS RESTARTS AGE busybox 1/1 Running 0 ``` -### Validate DNS works +### Validate that DNS is working Once that pod is running, you can exec nslookup in that environment: @@ -185,6 +185,115 @@ Address 1: 10.0.0.1 If you see that, DNS is working correctly. +### Troubleshooting Tips + +If the nslookup command fails, check the following: + +#### Check the local DNS configuration first +Take a look inside the resolv.conf file. (See "Inheriting DNS from the node" and "Known issues" below for more information) + +``` +cat /etc/resolv.conf +``` + +Verify that the search path and name server are set up like the following (note that seach path may vary for different cloud providers): + +``` +search default.svc.cluster.local svc.cluster.local cluster.local google.internal c.gce_project_id.internal +nameserver 10.0.0.10 +options ndots:5 +``` + +#### Quick diagnosis + +Errors such as the following indicate a problem with the kube-dns add-on or associated Services: + +``` +$ kubectl exec busybox -- nslookup kubernetes.default +Server: 10.0.0.10 +Address 1: 10.0.0.10 + +nslookup: can't resolve 'kubernetes.default' +``` + +or + +``` +$ kubectl exec busybox -- nslookup kubernetes.default +Server: 10.0.0.10 +Address 1: 10.0.0.10 kube-dns.kube-system.svc.cluster.local + +nslookup: can't resolve 'kubernetes.default' +``` + +#### Check if the DNS pod is running + +Use the kubectl get pods command to verify that the DNS pod is running. + +``` +kubectl get pods --namespace=kube-system -l k8s-app=kube-dns +``` + +You should see something like: + +``` +NAME READY STATUS RESTARTS AGE +... +kube-dns-v19-ezo1y 3/3 Running 0 1h +... +``` + +If you see that no pod is running or that the pod has failed/completed, the dns add-on may not be deployed by default in your current environment and you will have to deploy it manually. + +#### Check for Errors in the DNS pod + +Use `kubectl logs` command to see logs for the DNS daemons. + +``` +kubectl logs --namespace=kube-system $(kubectl get pods --namespace=kube-system -l k8s-app=kube-dns -o name) -c kubedns +kubectl logs --namespace=kube-system $(kubectl get pods --namespace=kube-system -l k8s-app=kube-dns -o name) -c dnsmasq +kubectl logs --namespace=kube-system $(kubectl get pods --namespace=kube-system -l k8s-app=kube-dns -o name) -c healthz +``` + +See if there is any suspicious log. W, E, F letter at the beginning represent Warning, Error and Failure. Please search for entries that have these as the logging level and use [kubernetes issues](https://github.com/kubernetes/kubernetes/issues) to report unexpected errors. + +#### Is dns service up? + +Verify that the DNS service is up by using the `kubectl get service` command. + +``` +kubectl get svc --namespace=kube-system +``` + +You should see: + +``` +NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE +... +kube-dns 10.0.0.10 53/UDP,53/TCP 1h +... +``` + +If you have created the service or in the case it should be created by default but it does not appear, see this [debugging services page](http://kubernetes.io/docs/user-guide/debugging-services/) for more information. + +#### Are dns endpoints exposed? + +You can verify that dns endpoints are exposed by using the `kubectl get endpoints` command. + +``` +kubectl get ep kube-dns --namespace=kube-system +``` + +You should see something like: +``` +NAME ENDPOINTS AGE +kube-dns 10.180.3.17:53,10.180.3.17:53 1h +``` + +If you do not see the endpoints, see endpoints section in the [debugging services documentation](http://kubernetes.io/docs/user-guide/debugging-services/). + +For additional Kubernetes DNS examples, see the [cluster-dns examples](https://github.com/kubernetes/kubernetes/tree/master/examples/cluster-dns) in the Kubernetes GitHub repository. + ## Kubernetes Federation (Multiple Zone support) Release 1.3 introduced Cluster Federation support for multi-site @@ -213,6 +322,34 @@ the flag `--cluster-domain=` The Kubernetes cluster DNS server (based off the [SkyDNS](https://github.com/skynetservices/skydns) library) supports forward lookups (A records), service lookups (SRV records) and reverse IP address lookups (PTR records). +## Inheriting DNS from the node +When running a pod, kubelet will prepend the cluster DNS server and search +paths to the node's own DNS settings. If the node is able to resolve DNS names +specific to the larger environment, pods should be able to, also. See "Known +issues" below for a caveat. + +If you don't want this, or if you want a different DNS config for pods, you can +use the kubelet's `--resolv-conf` flag. Setting it to "" means that pods will +not inherit DNS. Setting it to a valid file path means that kubelet will use +this file instead of `/etc/resolv.conf` for DNS inheritance. + +## Known issues +Kubernetes installs do not configure the nodes' resolv.conf files to use the +cluster DNS by default, because that process is inherently distro-specific. +This should probably be implemented eventually. + +Linux's libc is impossibly stuck ([see this bug from +2005](https://bugzilla.redhat.com/show_bug.cgi?id=168253)) with limits of just +3 DNS `nameserver` records and 6 DNS `search` records. Kubernetes needs to +consume 1 `nameserver` record and 3 `search` records. This means that if a +local installation already uses 3 `nameserver`s or uses more than 3 `search`es, +some of those settings will be lost. As a partial workaround, the node can run +`dnsmasq` which will provide more `nameserver` entries, but not more `search` +entries. You can also use kubelet's `--resolv-conf` flag. + +If you are using Alpine version 3.3 or earlier as your base image, dns may not +work properly owing to a known issue with Alpine. Check [here](https://github.com/kubernetes/kubernetes/issues/30215) +for more information. ## References From 05b949fb8ac729f45b4ed7007de3975ca37cf0e8 Mon Sep 17 00:00:00 2001 From: Casey Davenport Date: Wed, 2 Nov 2016 16:12:56 -0700 Subject: [PATCH 12/45] Fix services partners -> tech partners for Tigera --- _includes/partner-script.js | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_includes/partner-script.js b/_includes/partner-script.js index e4a5362df0..68339345eb 100644 --- a/_includes/partner-script.js +++ b/_includes/partner-script.js @@ -190,7 +190,7 @@ blurb: 'Jetstack is an organisation focused entirely on Kubernetes. They will help you to get the most out of Kubernetes through expert professional services and open source tooling. Get in touch, and accelerate your project.' }, { - type: 1, + type: 0, name: 'Tigera', logo: 'tigera', link: 'http://docs.projectcalico.org/v1.5/getting-started/kubernetes/', From ef0bec049fc3fd0059e124b857cb107cb2aa3327 Mon Sep 17 00:00:00 2001 From: ankurshukla Date: Wed, 2 Nov 2016 16:57:42 -0700 Subject: [PATCH 13/45] adding new tech and services partners to the list --- _includes/partner-script.js | 42 +++++++++++++++++++++++++ images/square-logos/aporeto.png | Bin 0 -> 6955 bytes images/square-logos/cockroach_labs.png | Bin 0 -> 7516 bytes images/square-logos/skippbox.png | Bin 0 -> 10675 bytes images/square-logos/weave_works.png | Bin 0 -> 4339 bytes 5 files changed, 42 insertions(+) create mode 100644 images/square-logos/aporeto.png create mode 100644 images/square-logos/cockroach_labs.png create mode 100644 images/square-logos/skippbox.png create mode 100644 images/square-logos/weave_works.png diff --git a/_includes/partner-script.js b/_includes/partner-script.js index e4a5362df0..c1c639b34c 100644 --- a/_includes/partner-script.js +++ b/_includes/partner-script.js @@ -35,6 +35,27 @@ link: 'http://wercker.com/workflows/partners/kubernetes/', blurb: 'Netscaler CPX gives app developers all the features they need to load balance their microservices and containerized apps with Kubernetes.' }, + { + type: 0, + name: 'Cockroach Labs', + logo: 'cockroach_labs', + link: 'https://www.cockroachlabs.com/blog/running-cockroachdb-on-kubernetes/', + blurb: 'CockroachDB is a distributed SQL database whose built-in replication and survivability model pair with Kubernetes to truly make data easy.' + }, + { + type: 0, + name: 'Skippbox', + logo: 'skippbox', + link: 'http://www.skippbox.com/tag/products/', + blurb: 'Creator of Cabin the first mobile application for Kubernetes, and kompose. Skippbox’s solutions distill all the power of k8s in simple easy to use interfaces.' + }, + { + type: 0, + name: 'Weave Works', + logo: 'weave_works', + link: ' https://weave.works/kubernetes', + blurb: 'Weaveworks enables Developers and Dev/Ops teams to easily connect, deploy, secure, manage, and troubleshoot microservices in Kubernetes.' + }, { type: 0, name: 'Wercker', @@ -140,6 +161,20 @@ link: 'https://www.diamanti.com/products/', blurb: 'Diamanti deploys containers with guaranteed performance using Kubernetes in the first hyperconverged appliance purpose built for containerized applications.' }, + { + type: 0, + name: 'Apprenda', + logo: 'apprenda', + link: 'https://apprenda.com/kubernetes-support/', + blurb: 'Apprenda creates and supports modern, enterprise-ready application platforms for both cloud native and traditional application workloads.' + }, + { + type: 0, + name: 'Aporeto', + logo: 'aporeto', + link: 'https://aporeto.com/trireme', + blurb: 'Aporeto makes cloud-native applications secure by default without impacting developer velocity and works at any scale, on any cloud.' + }, { type: 1, name: 'Apprenda', @@ -195,6 +230,13 @@ logo: 'tigera', link: 'http://docs.projectcalico.org/v1.5/getting-started/kubernetes/', blurb: 'Tigera builds high performance, policy driven, cloud native networking solutions for Kubernetes.' + }, + { + type: 1, + name: 'Skippbox', + logo: 'skippbox', + link: 'http://www.skippbox.com/services/', + blurb: 'Skippbox brings its Kubernetes expertise to help companies embrace Kubernetes on their way to digital transformation. Skippbox offers both professional services and expert training.' } ] diff --git a/images/square-logos/aporeto.png b/images/square-logos/aporeto.png new file mode 100644 index 0000000000000000000000000000000000000000..94e16c7e2a3d86cf1c8d3bdb634b745a6032d561 GIT binary patch literal 6955 zcmc&(^-~dRQrxW+ic=g49PaM!QmlB3ySsh){1NZG zJF}bFUouH{@;uojQcVSf{(U+EoZcL>4-gTa#tb`>`7tRU|liBK4fS)Y`o(J`6H-&11B zHQ)_65! zjQx}SleBj+Uh=+TOs=c?Dsv)@2U>F_I3F>>d;dD||G$gPY$AcZi&Cg8xWiH6W2WY( zMt9G(m8CCI6ne9Vhsm^2e|{5V{N#Q9*>G&ELge<^RwI(xXg+Bk+rGrzV!fLkTe?uVH&o^hR`4K6^%X7)-=T zHR*C0-F zn<&uN*W9W+RnW^4)^@=}i026$)U#|7H_nkr!aVa_wflHAGqVq>qgCYl_qYGi!m@5< ziX4awz=(d)xKiGT-GfH+b#~ZaWNQv{x=!bnk&`Z|aF`7+3F;;qO7rDkRDhxK&vo*n zLUt+AlB>$!0h~Q75`qj(IldXZWj^fIU4FGq>Vq7bM->D(ZJ&I^fTRXB8I=ZL{Us7# zZ8de5Fl0UxH&?X^tb3$SccT!}r3O4LmOF?1N%r+e$KjibWFhAXZX^rO9jl&~W-}Vx zPx{Rpm|)~bJnebm@aR`21+G6bI69sOjyFntuI3KD^rvV-E#&b&R~xXq6=?8_)bn5kWDC z0}V)v+u}%t!mc+#?ao|H3cO+d3_zdY_sTfWaRKzAi1?^$@6+^jAWs$}?)y_pt;0Fd z9mY&fO0ZpYir04R++q_s<@=LF+zB-64;pkMNqWkEP@~Dq#>pOr@*tr}JgSjr01iPT zZ|r?);KEjz1^A^c6y#Aj_5@pU;TEn{$T9prrjnOk5Vr7v$@cVD%hDkW6rq!`xa__h z`|?}YocL*7&6wV7XAW$~%LHOSnH=ONyVdZbtjGM{E4$JudNBSpaIvTu6q4Qc4y+rM z19B;Z(7)-*VuQU%EeXp05h-M09vv7t-p{)eBEe{MU@fO?Bop-MhO~kqfIwV_WYOhO z%uckQ^W$}o(NN=|N?2wp8^qM|ST*(_A6}$2zA<7CA2Ie%oQDi2A{#-xL}ov8YP-pI z&c6$Y0=cC%fcsIk;I^N}U>8G=nZl&PnjJizhrEc7Q>=O+nhxfhqrJ@8C(C*(t0Wyr zVKvssXe#V67QB8Brbd(9;RIV@TlyR6tBkcG#sjpdI&*OnSs;r{*bSa58W5xy^UNa2 zYnx5PbDLHTRTa8E?i0RXwwnaB*GjE@W*(vdyA@*NYP)hcDVRa2dWhewyq%R8G36>b zD#m=mv`v<{lR~!V{ex8@!QW*5bqxsuc-G=*$M?~)xN}HB^v;>Z1G6w0>Wh9+&d9DE zPfhkQH!G-TREL7ED!uS|!lN$$KminIhO-tXXp|hnp178(f|emRaZ>Ot&D`9erv5>Z zZMc#4$ctsl5%b-`FVBzG7JTK8#h^FO;Kzur_j=abeb=^bJ?dAqifhzEDcISW{m`&u zeYM}9t_inVU<31Lo$kY-YSDo4?dB zu$#6i9{f`A=IfAT3I(Go;y-2wp+&@RH}(AbnavWMSjyEZ{572GidRAq%>lMl<`fHS zfG_6MjhE_)Mb7}#aP{1r7_oBW!yc-^{$p>6x=!Cg6Hxg3*Hb-P<2_Ai)S{EbhYOP0 ziAZ#AUf24FtH{ah?AatjT=de%o1&@{swv@GR<_NH=Q-S^q z&qh>}@HCjf#p1b+agwIg?mtvAaMtfA$7)=n8fv^`vYSa~*=JaccT1-Ha#IrO<<>k- z@2VyThi&wx_lsmMXBX%k^SZ^==eV^XF_^8RPCSNjkUOH!Iv3_qmVG(JOym+5#>J7V z6tg>hw!ifG)KmXIQfk$3Q+}VfJu*b#sKLM-Mf_ z*>nS}-e=`8Z6k=Am&~>&!?t4(# z!%hqv;Bsyues@BQ(M$OE(qok|ggDgiIp6cV$I`k_MQg^*>Az`FR{GrTNsjYjy1{%D z>(lp9Zf~d#9lViLwp`;kf>GGajCM368|&+AL^$Co!jy~oVRdzTL0*Shi#-DSqsd6` z2mk0FDo>32Gj{`!OskSI{+saPk_e+#mUQD~_D*LBJ&Elg+l+R{AK`7ELx_QBipgm` zkx1i437JQFg1}?k#-4wr5Ykqbly5C1w9woK&o++q8l#r-#7ATA{{5_7GD3#@a2|3q z;WIQ!opS%ZT-gBjot5TQPhy>DLdlqLPDp36gR!jf;$G0Nqv#d`cP_x`?P{cP_HvEx z-_e^ZIql+dxUP+U_Uz)epJh4D80u#)eEHt2%3++k z9KDeQcLXoD)4{ea5Fpjj8bUL>;#CI7%@Me9fVr;3e&C2*B^b)*>A5V zjw+;CZV{L(v16lHt;mX_G1ZjM-Bu{3Xw$mJdd~!$BSY7hI$~%nB97K@;=8s^`X)<@ z1q`L%4~BX1)^UB;{6{zw`nuTJBIuag#63$lMD04+7E$H;fbCHH#A8FeXDf3GPQul9 zHh7pzBFxqkt8W5+8D^?<{+s$Vf~ZQMHD~yJL~90sD>Ho_`D3cem4h4O;~UMXua?9y z{lE#*5X^9uXYBMkj?M4i1Oi3!NrEF6lWfPvv&Eb|!mnF(MO%(ru07YvJ*lL(usX}# ze#DDcilip!s0-yXJJZPA`+2k##$w&RWBgiG_w^qM9D8s|;ti*4&O*SipA>F9r#;O) zpaHfiq_iQ5mh=wl8vc`_87o%vgAs+WxuhgRh>qxXs4KS#=A%pSPd0@l2ip}jUA<*f zo6$R#NTUku#mXv1A;)H`{@H(+ct&KT$)-w6-x!kf?(3gphOU~@sV0k})@chrSxj)@ za1jL)q0sRLmRvQedRsoXD0&@ba-(wE^$|Te7c6tIKwdl$QoIlVQROS?y^KCheNi!>XOCVp`G#96FX zA0jd_(C|-^loukno)DKi3OOONMvxP->Y+HF+jlI;D99Y?;%?bi59ffW9x=r^D!g$6 z;i??Q$IieXIOHM;j->*}gna)7NVzlit7me0%WOA~k8S|=zSyzYzxc1t=3Yw~&N`O1sp}CA_L|UtL>ah| z-8KLwqBwsxb2Ywxdbmgc;r61!Y)^2uM=<`Bb}*%`mo7H;D=>%UMiJyYhsix5Rlq9e z$!6^gk7nfGSR(__Ty11n{+6zcsyc=7E?EMUy{X$k8m7P6rDNFCie*sIfsk8XCJ?HU z;!c~X3>|B>prty@`2!Q77BY?pJXqbh9}n~!Uwb|a{Xy4EUuGS*V8rY-hQ|Q^k7`U7 zi&KaAzrWqr^O)PLolgG-v}y{-E_xu96vRYyww$nH{jPuqQ@gDTR~yhec+Om=H=V$0 zBE|rTg9;i($!x#*ElkaEG7~xzb@aP-ZVh0V%(kwJ=YV4-tQ}&A@Vi@uFHC$MFFP|- zD1nhCz=p)Nd>rD@B)$To2Mz|1Qx8pW(jBP%q+BG~w9K4at#7j^Ydf2byWE-2GLT)%cmnRm1C196ev^ki7v-ssqT~+3)#pxtKpY=lI!3 z(c_K|8w07^YyKp3z0PV`ErVN#a{|W94k z-U{jaTkE1X+qkzqIuZpvWI2J&oTJmxTL<-I&-5F-RbGNJ{uK%ys@P?Xo|`k?J0Q7| zMA}UN3 zd%@QYgI9kfYc+QITRQ9p8}myA{BGQCA}Wg*>%PvS()!t|SHd4Q7g}kn^*kcU(L951 zXtuJXw*oFKEO)c)-+h4XB8jGFZlMc_J1(t>xn3>rsy-Ezy6rJB|5EYlC-e^q^)gaN zmRXnPclOd7ZqMwGt*N`}D|ZtZA4FIvu^pkNcC(~y)W2GeoM2_#Is>OUJgY%;eDM<8 z@4{F8*8ehT#OY&#weV;2wAD)qZ;71umfuH#84386)a?yOV-FW*8XQm$!mI!Cep->W zsuYE`zmAd$jX^-`LbqnPAD(H7ps%K~L8fX53l7qj7o&yrqNJo^Y0CQs{-+N%MGo&6 zJg@!)*Rg)Bpd5_mfFA<)gIrmM4heko{A|w^jr7!1ETrAstgJLuI6qf+VZ#T*Xb=uX~(4-ZEbl4)WKn7r^&<1VEqwWjmc1>^vjp8;$2F#p7u zsi%sMuxQ$5jkWtOm-m?_b2wlH=Ca|5<7%GL(~igJ1GwnCZrzCWs-%I~!>aOf-A;7$ zw`g!2gsXSP9=$wf)_L!M&kr@`=GKZQGY-9P4uu%-M)pd9(n-H@UmC8vM5$BAr$t?k zl^Drh1zm|cKx;fc3Dy=t_~+c*T(3*D{;x^C;U8DHHa4U%P6SL$awE{H;u&l86%TjN zKRK=ifYBQ$^6gS(gLsrfoOJ=|nJabkV(*jp znzgwef}!xI6*|fM-uJR=uJ;m@h5-7=`}OqQI#F6ud>Sn>Fo^J|$Hc(k^PejLKNspi z<3d9FX_Kmf96D)+#sxw>g!iHZ2kn!?!yk;jhMWCs<~vfbFNV-UhtP<*B&lko!j9Ai zl#5ur@9jDl2F~M(dK0Dsdolb7P1~+n>q5?BH%NnZ5wY z?$tgwUmIinK&pDyBU!~lv*?t%Q@%XaFmJ*vi#Cu$GGb`%+&*~z?4$j>l~*_52fP-$ zn+V6v@;VPxlP&I(c8~2t#nTK8p76OGhs1p&Yjpt0Ca;cS4l>?SRGf*N z@PHTzqR$EZ<;|W?<|Q#$UhHra8b=pWvwL?P%gJ1xo9A#k^=#x8?ZMw7c_9J^7T8T{ zglKQ4h8@S+FE*t)EjD<{zQA8>7!YR7pDSR8n`6d*%}|%VPO=oUx3&M-*q(bHJT|uS z_@d!A;B_1z+yQ*tgBG$9BmTL1a&HN+MMQ$_$Xb< z30@?S3UGY!hkU4#`v&m=?yW=mYBU5Or*y(8r)m1SPrCs}N#SXQnUZ3=kcVj!*q0P) z4RkKpw4$7s*dj4wPg-)Dpy9vOM0^U}*gQJIC|KPV`+Tdz@ZSBI2pS^t<#0(XT?jeR z)OAZ086sPgnU1z(d_!6%boMQPKkiJ*bl0>7%L#7^z;Y~|JQ?s9fE$3Rndr|SXjVqE zxW5=KZnGX4(y2U!3#Hq0!@CnsfFOHM7lXc1b(i!n1X?VRyukA(=cL38#2r`X?;9&3 zi+#W0y<-3_1Xlfim1WR@)A?h&83 zzbmWdYJu>~d`1ug1UDMHdgD7m&#Yofa)ig|J_yivmnhzwgYxT|Y)EuCR|?aYv(+No zQeF1&pntc4&&X^jy6CfT>e+)IOg=hx@3#_&w<(z;`Z-)^VZNU-y0pyrF~LoWHwVNvU>W3_luR1GtrH$NM3geX}gX6ySq`S53r^pDL5%R4IK?0T%|c zh$E9Hr~O>%&lmd83egil9!vfQdUJgMaG1@hkOA(>x1b)-PKjXT{en zYnAAS$G@}H@UDqA?1&Q=9h$z9fJ{R7IqWMAtgm>RC=kCgBOwGi_qg928z?4ZYUt9H z!Yov4E>A^WlgNpuMoUKjYLF9sIrft-5SPY-iZ$R9ao7C@r(WW(sfaQK-B{K)Uq=%v zQ^&hrT$N9#kxAG`fH}3TBp|FSUUf5ik6si!>yHwZSfi|6d;yE71m19y*bKd7UBrNq zKg751)}#B#KCn;`^$g-Lj3@$rR8q;$ykvGI62l8i=FIk@Bpwu}URmR!_1<@l)@8u6 zitDUwlIPgrHtV1yo~s3!GsKDSlB>Kf*pILRq_+F(FnzHW9U{XQv~l^%WZPq-kRc@E z2!YQx#DQEg;WhPuC{pkMwb?}m#XX?H#OBAID zx{YqR#Yeyg@@pgxn#SBgoEVyMz)p7wpndnT>r(s*>nCBn7eIoPcqxfCBgfZFEaxxE zc7gK@SkRs5R(gaLUPU#Y?c1xJ0!pAH8;~d+LjU>dM_?_^KO@-SUpa4k|Wy1O`a+nr!;%iYCMc_K^)^EJ{ zQ=CPwvcHT$gg*4$02`)mG|_pgMtUcqZm$DEnz`{$t>_uy^2{J2!mF!F>_VBBo ziKO(45Agnp4d)eY&vTCjr8J-vIq`-uXuGBeX!)GvKZY~6ARDn!+*1hPyO*ux+gE;? zQE-*Zlzr+G2kx^2_c#X!g%71nNHdkE?D|BbX{q~bflEGp2!Z=uH1I;KOPdzhXx7u4 z1fhKyfJiP0lip1E97Af2A%MhlCn!g*T8GC9=WA2SRP`|14I2J%>W-*ngi`A}xlIQbQ^NgIMUKAZ97ZQ2DV MFQXz|A!!oyKhBI~=>Px# literal 0 HcmV?d00001 diff --git a/images/square-logos/cockroach_labs.png b/images/square-logos/cockroach_labs.png new file mode 100644 index 0000000000000000000000000000000000000000..85750b1d7fcc99de83dcd997598e91deccf9e5a8 GIT binary patch literal 7516 zcmb7|RZyH=m$o0=-8DD_4IbRxy>YkT7DCV_I0Sc>;I6@GI=IUlplJwh0RlnNB-mtX zs{WdK=R5fi*0pxs>tyeBwCjr3)mFj7rp5*U08dR-QU7nR{9D0Bjvs1qEGQ7cYM=KNl}=1~ml*25(<4Cs%hz0C=}vWZ-OQa6&GH z_;w?&5f_oE;iXT8$)GQv7(trG#>t39s2;~y1|!oOCQ?#DrI{#=LrG4KNFvkY#!bLp z!`NpmPySF5moRm|6I|i8(vLWted=75K5DowYM;k|VB%)03hD_*;MU616K#JO9~s-( z=amIxvv>o9nC%XXeviyZz+IT6Bo`9|V-P@kw~T`cK(q=VuV`bR3BJqcSs+D3Awm4p zUTa`RkOH#rljW-bS!JY%qFgoupcECbnYOpv19W)+8}{(SWgw#HE-x4fFwbNpLn=)J z7)TuB6ag0rpkdxPK?wl80!W=SM#O+M9^jRliIXbO@)>~45#qD~n7F_zy}0O?0BSH` zGr`RK4v5MFNR_WlB!6->5gu{=-BeziWDkp&64)G*#|Oj2gqNOUPK|<#RKyn9HeZ$( z8k9#R02U(HzMcVq(loNauf2YJH;3OeHz$(Zg>TLCeF*)T$0rLzTNVIz{KFQX zxcSD7%fr6qSe1IL{As%SBAG@cqudz4=q?1UhH z$Swi^uKK)&|KY$yiE#O_GavYTA@itG$_hj{sipe>fR!Q(ui;d?%s3_hD3*dbTjXeN zhAFur=(NLV8^bscHX>2-EF(~PLV0Y52wGp8mkp8f95J&kbeuMP|H#q_LG)Z=GQ9D5 zp+?>KQr-lAY%z<6Ir_m^sB#mSB({v}aY%LvAm(IDioEDM#$i=d;zSmP$v9FyCXF;< zRet>hLk1&Nx+^JPWU*Ka<=!-j2|yt7A*Nr2KNH-hC-o0*r$SGSD0P%_InIKEKYxA- zd1=9dZwIwhl1SOnLcI&YdWy8d_$$aUC5GUL`zzkDFY>Lgxv?_rr(2mCaO=mYTZ|h} z`r>cx=UW3&iNQ(`ED_W|c?Mg0IaQD*NPA7ShJltvh-e*46bmIv56aGzQKQkp)=Ttz zgwjrcKNh6Q&AdXKLc&KF9H%fM%0e8kAi&;}imcZ9$#98yNqNarpZSElwm?XgnmJ=~ z-NCI5LoAbu{9(@0r_IHAIQ}-DjUnQmZ@DRToA6!$Ns%Un_2YQISVFB z(>SF@H!Xs?0v#^l2 zoPD{EbbX0Zb+u(;3WDl_ChhvynNBhod$~S$h5p)DM;z5T*I^0YNH1RQzND!V((K`P zv~a7-&dCPVtJdq)tDFvqG@Vx$RShWhncjo(<;K<7E!b6@Q|8c$jtpQ1o7LFWsgen_^SL-nIOq^%wmsa&tuGEtbz@ljgd`5 z>$}#Ee0{B+s+Uy~ni87hvoo{hvnzSR1ycO00!IZ$1*-*HpKZ+EnQ?d3bagwbwe6c# znf~bN_l!0ZGpz&}nOU|gHx@OruJkqbRTEYBXlH2G7q2*fY5CbT*;V_Qs^wq}9p2RC z(G_azY)9ezd9>-vkBotg(B9CGk5U+Tu`v94Iy1U3zYbqgabxjs`E!LoWI{)8vQPM& zLGy(3Yl3@SB-1ZPT#F0}mQ|c7oO}E}T*oEGYeY`-=1B7v(2Q+P4D74>-}+eYykEL4 zDrhf&p4Y&ee~A2W{SmuSG9y&-ve;D6Z{f>?!nWSQq6|(+)+@KIhAo4Y2T7P={-r&Zvx*;t!G3bpC>*~>wNJlwvOIX2xoqFcz*x9 z2vkR~A~~bPgXO^H$oF^eLl}pWWyIl*zuI8ua!I8CaUkkN{v6@J#SI0&EyX{I*^1u^ocVRc znt9ZPIi>4(-$<^D_6qgNjPW{|&U^(Gfz;eJXvKIrn?yX_2e%&1uVZ(Mc5(KCDRTKH zEWLxu=Abvx)#SCj#N6W9vYK()uQM;@2svxWlUT1v0$G!>gP?R!XC}Tyk;S+*1q)7k z?JHGIt@r6?Y5l@METe*z+y^~T?6s-=ns}O<1>(iz`EMCK1f;|V_&=nlrTWO%F=}OW zIe&LPbb348;27QZH0qi7nU6jTW74fXwE@kP2fItY>e$eXW0e!5&Dqh!X=pZnN-Vp+ zm}~ZYrhN*YDq46^@_bGVCEiKzs*KSv;h_T^4m7+GYm@W|f@ANIp>f3+qc=!)Cv`Y>uB@mvfkd?}-On@4(wK?l`~w zr3B~8ndN}@edto?sTh*%?su%7oST)9G5lnBWBA8V~CpYWM93nWp%Zh!5ovPrEzZiTfx(MXK6Tvh3SU9%J?5!BIe}cf;LeS%hufI(SZrH(Ce({#hnU3#` zeEwCwa)Ckqy}6xvug;A0;>!1L?;lj)F9-^SBu#_VuMH<%*H0psGn}vHlZvv6Lc@4} zZ`i=gyMmAG=eFJucwO`o`tK|$JtrPX^h?ZHTy^_jwZRWaOu9|-o5ig?zWaZde9BnX z8Sg9X%WvwjaO?T;exNhh32{PGEIPM&GZ6HX{CVYTv6$56!1a4C#3te87<7z&aPplXztxTL{rPaiZ0;jErYgl-%nF zdqE!f-~T3qy{f(z0K8`ffQSzO@ca31KLLP%R{-$c1^~o!0f53Q-Fj3R0ElDN6y*%x zt)CSJ{Ys_RSXOX*#u~M{%>DLXFFXFk`An30Wfk__t!R8htV>HU5;)dC+QH zZ!9N^Cg(DOXI`gNGs>aC!U&FFVZ?$>h})){3fvb3XFvIbu?HRPZ@>hwh{8&adgeF$ zF2s(cj{3I{h^K5sG8rmTg0H0QDYLF&W@mSKeL8|?Cu$Aw-lQr=G50W-=74ja~41%;CAqC;J zk!tb6JkLfEv^(kmqxcS5LeNXq`i{>v_WrQPu&HmmZ>NB9z%W_Mw8vhs7UUYT2WgH7 zNeewf%0b2fpFnb%?gM}Vurv5M&5!fKUjxemEo6P|8-WmTE-Tlt>9UP#)70NYQUCju z_lpIlaVxA1)Ith3`w|cwAA&J8@KRUhb0xnGZw(%#xLa3E^d-1lL`wF`StSp; zyoF$ZX%q{$G9S&rtiC(Vw|;}NAzwv&u(EM?>8p@SP6!~%J}eaIZ7tFT=SiN#%wUOT&b zyMGTUiT^3K8!S!zvMd(tw*N>>wC}!672l!a>;GgjgDu6I{f89mA95OkIt1nKfODpq z>_A=jC8P(JII3KGwyAKmrfft>Dk5ikv>02G4QWaB5)CVX=?B0|BG-Uog-Q=t@jraE zbHFmN&SUJjnBZr{+RCDfb}ryrA|HFq!fny%zo>XzR_IPX&b_fB^8 z%`8NnwoapJygSB@+^3pVEhdP0->Pb_lvJX+c0}9*=DrMrKJ0n-px+g@K8g~bv+D7# z8Zn;K(fdrj$V%oOvxmffgiNHnL$`kU&9K>2VTRGYQj&g7#zfjKu|uiR67RKWLt{fs zFywlKbjfXy=pcWV_|rP21+h+WN!6T~N6#f{N)JouHr|c0%WjedVZzYdGm(-py~}4f z*aogPR>?JrQ~n{nHa&T#Lw`0Zz@&pb2au~WEOFe22XP!qf=ye`bI>mXMKsK6RGeZl z>?0qtFr7dCgpv7L6NSGZ+2?|ous$vqymHJ_eA%osiW-Dr%gD&LJER@uf!_sd^*2tD zJW|?)t|@oz_d&v`<;Gm7`fuAFgA`KDBU0j#U^H&a0?3mW)S+J)1UFYlsa2T#h<^mN zg$yztT01*<(|$b(&X=5-nE&Hk@!@^t$2+BY^e5!*@LZ4N3Ny6>t7}i$pIQJ; z*8CD?cp{iDVLrfZ#u5bhPyGsMC3jR>KO;)K_G;V23eiWGSNx2d?JD5GZ1?DAQ;%CW z@mVtwGTShY>>~v{%}o%kOp6WTOR{J8B)lScMEOG33sbeO96|o(88@g@ylDn&NQ}7{IaralwS!K(j0$GE- z6oBl#5=MjLDccS>F^NOy8TfYGVK3D1kb6Zgj&PUx{&wvb^A)FvYSl>Bd=BIw8mZ|k zv30vjm1ENOSW?^SXtD%@r{PE2M{c4G7T55YeGip!RC+8wb#8;UmtFJNL3!W`wfEiP z68d^`w^D9SMxhiHV*zcqqwkksyHH%o0{;Mv+AI3rz!kIXJ!>INy)mVqyik1?V&SOU zAQNOd87H4dxegP4WWlLrNZu}!1M7Cr9cGf8Za)j}{%L;&kZl8x52 z)Grx+3L!=sdEn9(({>0Ua*U~J>O|z5{V86g*38w5)C<# zy;83_jx+n&yvrm4%OfP4r`)}jW}kZurzx>yp^D$}W*E*Nzy5R>I>5N{M8EfBoS^Y@ zkSQ+_g`aWmE|lniHX=WsF4H8K`(cf`tr9I{UcDt9zhW+|C_TRM(^OxpAR1;vo`UMH zeD}=$YO2?pmircdNv6Xj7#o_Xe5$IJN@qD7!dCPHo)M;Fi2=uNu*?RX6u;{UxwF%{ znN`a}e{ZF#uFVvUIpZ&f2U`{EiHm?P$VvkI@LyiU9Earn3RB)vqOwkB*u{k=WKv;5J1FhMCmT9!t`~>FC@l3U{;0ub$9viCE`9L>z#<2JTUwyCWy7MfZXiHnR`Y z%(oQaPsD|p;kuat6hn%zB=mRStfiDkfxJjGJ=oLW!O7~Oz zvdX&tI)+MQlG;}E)H-te9Q-5PEs?>a?UpA~(-RjB%LZeV_Nwfv`>t|}v4P_bYk5Rk zE!oGk^oS}nJbr~L%Ps$DNV*B4KDt?V}9J-*| z3W(Rq&DwjPbRn*!7=ZP`w%PxV_)pVZJn=f0!vZgVf+_{Z_;)3U~YIxL8|8~>Q<C=E zDD%E61T{#NZpjBeORB#e%iabBrx9DbwSk%jh|4|`g6^6=9K0h0?K~OVm~EUa`S37x zG%EY{t(9lICGzGx>_TPAo(jWN3Y@Xzr&0kO)Med&OHVC()g0Q?(LQ;ZQ*Ylh=JLu` zbH4pkqi4?%kFU&0R0Mx-GdVwgh5Z1fP*Zrh>xh9odB?kAts^jZm!$f9^t#%)q0Vus zk2zUwhlZy<3D{-4ZFsu+C(cWTsbU8tpA>-ohq-sIih$@bP*4U-m zkMDwxUd;<$G8iq+t|6Ms9Y;0s8;WCAix@3oN(WT%n%9mJCwgV4M@Mx1P2AdQ^jrrd zsN@IjZw;h_1NURL1DsAg8lZu6Q&j_4*(oJ~BX$a~j)B)EywyRl$(u9!k3~MSCl+GK ziak)Z``-fvDvof3V{OgsUeETuu2s?aTjav@Z_-5+cjTKAZp;_ECwAllmS2@-A8vPA z<$PTwF5Qt|)XfJKZQMEa27auvo5NqK=*6-)Ev;!m)<4)p1qXQl70Fv2Ss7fq0-Z!S zmzEgF>>{ePzI$ereJ*God?l)<8p}VgUCEAoIHt6^NE%$c=8A>_0_dDAdr5TkYu<9c zZUEfHq@xdg-~al@M#75Xawy*JR7>yeijzP^-(_!xu@e8sOBD}3;>((nzzHpva)+z1 z$Vj?)epWGI%!X!k@By7~^zWM24|8TzhHtG7%T#mJfmk>4PamyeoK(PSz$~V3x78?ePX+Q?*-n~ED-s=gn_O4 zL??mSKS!4Y^X$DG%_mUp&Vps&+lqa(wN*7 z8r_=cQ-Vy;$tX9eo= z`d%jMV&>1nK-3U-NK%Uff(NQTmKB9#dyyXMwX@&_Bj|IId0`x;&UR7#)Rwi+CbV;8 zlW!DwOlGtF?!&pPytz_}KytvroOxQD)r)Y&#K)bU2C(V@4hXeRg9aKci;07HYT z2~F5R=ha}ow@)Nh*e2_+?#=S{aX9@fNy!x(luVQ`o9v#7zU=k61^ADKiJmVstY|6) zMNj^&9pzSgGLZP`tn;dl3{Td9@jaZP=Ly6sMrp6K>$(1YQKKzR{+`c;|NT8=lyyMc)v5 zW{cSes*@QNbP;}XG4lT*L;gpP{BI@l|IYu>=lq@i hU!QZOz4MF#$Wf&2MiY$H{MA>0nv%9+le|sje*s`vQb7O! literal 0 HcmV?d00001 diff --git a/images/square-logos/skippbox.png b/images/square-logos/skippbox.png new file mode 100644 index 0000000000000000000000000000000000000000..2ec4fa46e02c2b39e711b14d86c4706985df5c24 GIT binary patch literal 10675 zcmZvCWl$YJm*vCVAwY0FRTTbf3EI9igl!je__U5dZ+7$N(i(Ke6P~QsH4g^PWNA>L(yL0<~QL03@vc7z7|Q z8xH_LRQfI;OgLF<={vvBOyWR=+G!^sL(aa8&+zI}rx7|9QZij4(FH6kv{Ug6JqU0xM6YWCuN*uW8zAZ( zD_#l^m4X20q*JQ_@}U7np-)DZ>W>pqosF1CgHqAjL8dmjqY| z0DewuM1BKku>!CyIK^<`Mw3}!ZSI+X=*YP(@x3YGGKEXLmFp_GXM6d{1-0Qo0tnzxSs(dJu0eB0{k;laU*?0~qg@tCIHyX9}g zUd=b#XMe%h$NQb`J<33K{Xl8xr=8yuck(5K(}@VdCL4d^q+eT*K3<9D$p>Ui+O+7g z_f;^RVkI-9Z`n)8Bg7Nw2C3#g%=We!|3L|Fa{+!?3q$xMFqP|@LOsaQpliAI?mYqk zHysXr3$*Z1V5^|rX^)R9kvHjlDgf9*CeaB1Fp#8V28}g{48sEelKFx3zr+ac`tTTg zVF~+SHu{iWjktrwDF=Rwql+V&feD?B=zjhXrwyI_MMQ7JvLH&t*{fm|n&gPe{9C;h zRmc(T#TY)fkG3-q0a|PX9?O_)EgZrmQi~!M9ycT8iL4I@jTuErIvS3xLN1rU1!Pl= z1d*x(iEf0PA^F1eq}me%MgZ(TUPC*j*^&e6RfHChnW>o1L;P#qEX zrqgvE(3pYWdJ(vxJ;X_kiN%0g@>+^3z;aSTN=}S51ReyaV3prA}+9LKt_rs%s4C-bf0jE0Uk=s2nlwYws=UHx?<2A=A6`=y(+~CV?`Dx@GC{q=$e^L zJse*$BPqj3$U)9Q{{hDV!2#KgE@F;}s8jKi+8(Xyn6)haA^IWbA*^{iyO?@Wrs`&S z0SJRDPDi}Dpjow6C9E8a(=C&4GU$tpTuF6d%KTR=T&u_Lwa2dQ7@H}?ifJ>NbK@t{ z$7#nmRv=eMPrNM^MTs-^OOHXH%J) zma0_+tWv3xKI`JHxhTsi?fTZC{VxzzY*>~?k4D-uZVD#nNNrVZvkbB9i-HjUeGPM^ zO$k9khonc3ec5GtZnptASKgs)%d6|?h0u@X4fJOBQg$zmtOeVEObNS=JcTDX$mQf9 zNW?O{eyt**-!CdrVw_@}wi}$#HY-@PC>0}x7X5ulZ=Z7CZpdz^h#Hofj9O0NRN*pH zRiUkvu2eu?Kz?{~VzO{@DT6CZh;5nuDC;O|IcvMcNXJu$vAMjt)m*0jK&Mpuvbobf zM2AniL`z*qzd@=xr`M{9!MoHG7E7&Z^yw9j z{bsCjx`FRGYFYErmbjK}EX%%jWI=5U$mACcHmIh++i%1dFj+5YLTB%u^&*>ps~V#<=BS-t$L zrNpE1(fA5i_(9k?Z(n#{kY|g1OWSaQr?+LKWn8J>A=fZuTOxqs!|cQRV-`>brh>49 ziU<@7)P?-_zL$TT5gA)28gv5w(dUuW6UxIAVzBwV&7UnEl{x3VnRl=y0yWVz^t zE19hY*Ld1F+eL<$Ewm>VwQ{s%Y~={~nCWY{?QOfaUoURM_Hy=+_I+^ESw{37y$Ys& z--VQYsbI!rITb$Di+b}H_>lPa+X={+3*z!J$hor8z8-c zcj9?MCzluHAV&$~VOuawMSQ0`iu`64f9{t|cTzicA^tA5pu~iDC-F)$g`{T7Q_DXV z?&CkrL+alL?W0;)h*RK3Z5rZ#!e}!gHj9@YgLG(@>EY@v%{480CL_l9Qmb+qCNCx$ z#@6F`vaj+!E--&%?j|;ugvx0$5o!JD`f1HqFX-g8j<}Bl!w{+g`%|zrvYFm)QK{-e zGp)wIi`rYkCeWa7>_=Y{gv}X}I^to7-`U3Ug}#y&i}rSt@k?k9fiO-)WDWK@ZW>O& z3hjpHAwihKSWm`Aa$`l+K%3dL#hk^-pDm_bUDVXdv_EO6_9&}sjVRk1jTiS1-_}{u zCTZ6@f|@1Tih2ZFJI%{RrzQnNK*gXt&YrPoQ&vN;8{y<~4equKps@J{qXY*k`2Y*9q_ePiWnBfz@XBmZ`GL;4XmBxw3Q;XV#c0wyjaZafqv^emTG zn2&Ey#QE*ywM=5#Y??Y3_r36ayax9Ri~rN&+49tE`fNekO!^)PYyPNJB@!AeI6SgL{~-IXi5jZqye-aR?l_!YBOd3lJJv~KA@q}Z(mJ0nASh{ zo1~^dRRsXRn+gB`2LS-jAD?&v0JyON0H;O(0Dn3Ffa{QGI4A`G5O2vyih(@W&U3ud z_0)47MUI%8+`VPWWW>eWu^_|I(V?|rBzK7%CGYN2#Il{BEe&vpM9n=_;8MONU`Qt8 z4zb1d^Tr|uLH2`@;0FMhq?Dqhax`SRWl2+Wnh2Dw&N*F!%aU#qq#BV;cr|9NM@<$X-``3G34+3wgFx4U{Dk*caYxaJ>lf^k<&J)DNH0g7ODWW{jWaWBY|Ff@44>YpXH_#??UE#W zou1En8Po>^G*t}-L>z>rO|9?u^IL33zP~gV=uGwV_A=Ut4$E^kB^t|3b<-b!P7+?2 zr)7u{EyOs0C8Q-p599>5hfMWl8U)&jDI?tlwqx-_q2a<@74^fji_TgO(rb&m6*7FW zYXo^7?M=cQjAX@!CBk3h9YO_BlY>rpPuT$XP^!be2(_jGAA~2y_*lPp*1tWC$vy+G zqOxhMvgo072!Ioax_J$qOb!#8xWd06m`(xjho#3A#+2Sz(HNX~bA$3`nDfBJ2C_TYdn15GPSi z*j#gM;KRf7`B}PzQVlGfJ~W1SFg%SUX&y{2jDNn}avT(bgjRO#M=IntBym%5w)dw7 zJ-y~blPsiRvS?cfuiXoyDhK<#5ce)0S*u};gG!AQlHg=iRc~Mgbmp0xih5IK=Bc{m z97SHOnZsFvtNUbektyn?P?M|2GG?eOD-uR0efD3(z)qxA#3!SPHh!olmg=>hGLlSt zxiKgLOPk@t9}t}fc~BlRBF!sNr1D9VoK6Ej@>KM@LRs%3$ZHSoWa#4A#X#vswLuc6 z27qGKf5wrQm-vlWRb_sLP!|^{u8;CG$Z8Ct2F#MYn&HQ{;*xoh3AcoAkCpzO{%$2JO}z+~PzgXfbO^Ls8ilnLS;=ySUZbMYKC$xPhIyYgd$#AHO z^UVbj%sVt|>;8Vfk|<7g!@)2pJtk8HN>&jW^v*KSGaYeawv55CZ+GiQqS!Lp!zd#b zJZdU+2@;kf3V0UBqv~^8Tfe^Vmqyi$9#-1BWpB_la%ec{9V0SrUey%~IyxHLzWz*Q ztVLnk)DvvUDvEo-;97J*53MlB!We2{nrCS55%Y;=k$cXwhY0g#dT0 z=48WfAcwETo<`tFmFU~l%s6xnYs*(UZ(FyU_O2jj3Y1o&?-lY{RSvTO%X1ZV@L#_nXI36FnUds*L5k; z0Ie<9WdZ+yvW1WFJ zw{`i_^%yBEtvz&pJFB)*&$9a-(oeety7H+%@dg0SD76tgYzX@{_I!_~lCUX{O*OWk z222ay4&6UcP%ED0-DCU=bEvYfwT7mxa+s2my8j@5um+)vc{ySR9=Ywb3jt5S^w3^? zya(8_mZgkz`2A;o z_Qkk=Hx!$%cw@L?(jcaaG1X)P_y5Q|*^P9sH=(YugWF7GwErkhSu9|NVD!8BSgrX} z+Yqn}V(fmpq9@2-!mj#sw1Z>Va6Tu{QuOjW69|@50$zID{#*X*W+E^@dNlI{VH0W;!m=+WO z6%Df^hhRLDMv|uT;e@R`lIHV`xOF{*mbygy$_vk$Z#cOZB*)PWmA>S|5>+yKOf!0B z~K9KlGI?etC2`{JV7Pwq#q$g&`Zlzwq=Q5#1Wo5fmM+lI~47>_EGwOLPM7k)OrA_ zLV1N@DV`YKmQ3rXfkh)d>dyW;Hc>NuOYy!UI#iBL0fnh!nD!#Nne zoNZ@1G9L3sC)6*P(dFzD_A??n^LX6gg5K%`PlvRFTT~VJt!vET6lgwa;3}@}{boGZ zv-J{r(@Y9rUAZYGWZRd!5`HDIe!|GTdZn0Z)M#l<8~v@mO{V1f)Pk8bQarPznkr#J zS(hg+_Y&2e+abTO{!5RE@vX{94vCQ8P%2y&w}xhOlchFm0Ea*3)|!^I@KK$S7#6%% zk3V-V?u>E|7QzuiSdW8^LT-y!zx`(9mi#8_3LN(EOT4-|KT0 zkor(x)fo%yXiS_4*QMxej9I2+BJ2=<1HHgs#4I;<6R6~bF*KfBZo%lJsJ^~M)K#3! zZZdsoXI9TO!CM^2I?_;Pj#P7$ludnuF(=LL^6&Cw_0Jz23!>&EdGg^UB97xzStlzA z4sjqi*Cb`a2%FEAV)1|n4402UJ;zs`Niy9{nZ;;(_a=oRr+LEC=?%H|`k0H3viSwh zpmqse^oYwyU|Ma5=r?DUc}%Dj}+tXF!wiO+H zT#)CsmQoBlZ%r6_7o9KybS&0xd%-UH;4oPYLB@v?ZScUYcyOv~EEb8h_s^dZTS-*K zo%WT9KH+AwOu7<===%Apu}lA$DLSLNVT_+EOzwg{&rEG(O(zz=_ozCuz|})f(4VsW z&|c`s-IL9PA{H05D{WIwRuK(*_>IK7mRR5EAEvF@8ttFMi0Z2Gn=He^>|+bsqWYMv zT&tSO9N)-5URJOlTT;Iip)5dSl+yejq`a#Zl02-u;ch;dY+4+==E{+qM6)z%j`M31 zFYj|)60@q=R%7Eb$m3g<@wa6qFUx>B<%<-u7+|u^KUv;c&o0;%zC|FuY`N?u*_p>s z4v|T*Ua)nDoeWT9K}P67o7H^=J~UqWkmxB4Y!)~z{<-a|=TUN=<;Nt?Tr^-X`cnjF%@GCJ z8J`z!T4dycaC-wW9ZT5N??zXF?qjl(3PMlwQWO!4cvrH-R0d&@;u%gbb`smKh+O9+ySQ`;0$pe22R>fCA_ z?(+R-5uqPk6RYtmndvIqDBSvj6@&J(`!z^N-!6l6Bw=)klrXI(whM@#IJyM*F%>Js z`xzG%Vx*deem5veJs;&^qle;!TmBN5+h{(EZL;cL-y-A*27`EXV$B9))qt&U{1C9% zSB?+NehyG_W21d{D`(HhMTYUEgMxHLaSrP@)R28YiWJ2{A#}OBrTBG6UqfhS7gwi- zh7KpvS9{O-q}_PQpFEZ^G;JNUubNt#fBh|2yM9B?YN@5pJ2TmxEra-Rp=&Vsm^dMF zL-}Jy_w_C9-PmpI>R=;udp?Jy8Aq;*dl&H-LFh~EE2RE2BV8?rKagq=N44VZiX}WD z^G0oLmJRAq*qWncms}ZhoDN&B{KPGCICh^DlrkVpiap{Ni_RM>3Lhc5$LlTsQFnCk zo7K6^{N&=b^jch%;@D*e{;$x}EQ99t@8$+M49uNd?5nGWV=#i)d+{^9s?i zLG0R#cUJ4+kQ-GQMKA^VrQQQd?waT2uW&kWFj|k^J=wShz13CRkHxiLQf)Sz5(qh^w!D12C3xPfllU5&B@tUG+qz+kTbDxJfC+YQ1Q4NI>)3nDdKykJ#h7M1wS3 z&x?)0W>(kDqnzzVIC!w~g%xTUqP1+UId|&`;SS=mR%x%O3_(O{f#n z9PDDz?@!C?PHWb4tN4pn9y<_V9<2$Hw~{xda6moU%t+JqkE>Xqo9MK3?ROpg6Qwns zB$?!m%z}3gNpjx6L_!?q#(P;-{0;1y%i^o=B95)KKz8GD#MOc9s?}AA=b{Ni<>6iT z8#NhGp6l7yQZ6R3$m_BhkS$9W!4ne5vR=0TdQ}wPKK?*}ul)xj4^vkVtFu#1_)n1? z@`=^8SWdP0JV`sjc9M}}2+W?ckV+;cQnO&}8mdFq>qu0jw++uL;)TQrg~JDPeT*IW zz$;`O#ym*>NNYO%Uj1?fAk9Yq7k1Lc_E7n8+1K1nUa_NKbZ7dc`kg02IA85JwE5hV z)$*?cTiYx}d&*MS-+=h|`@fW_lE2k@rm(9{A|moAO1-nO1;5l2BOT1$=eg6*8 zBV~8QvpdJu=75DW89-j=(C}jSJBL+V(M=W7R~@SXy}=jmuiXwuCjWXcdfZg5Px8Wq zI4^V9B1zExi=+yWJfYCP58-K)Q#I5lP+wIFZDDCq6SO!ndlc=2_4j+Uc`Ft<*(UTV(RjJ7H*7JXCYsNTwAfg}5d%xB>1v?l z#Aem48cdVu<~uvU=-SEyfPgp1j(X7Wd-6i`=X0kBQUCPMtM-R>dM5K?w;sN z#p@H-P+)u*;KLpr6C@pdO;hX`|647Lf(( z;QRHkP3o08(qqey-a<1rBU(I^^3jgG>qy}>wz~BhwW&o3o4!R(eyS7;Zk_s$$I7y7 z=cyAb<@#AK!*9BWwGcLc?fvb-lb3t$8eKKf)=;V09A@+(rm~}IpI)6w%-Yhf-tkw0 zs6Wu=nVGAPdidOJVQ@}Ae2AvdJ`hL($Eu4-*f}zl;uLDYwJTK z4x7t>S6)yj)Z?n^y3%DLH*4LmMFrd}Qs(kolJ{u1K|>l+0cBM5GzLme_q^rwFFaBo z4bXEkpAAAY^I)cNPJ&xd;9HSFunt>i0Mi_gXH%wkF3}i55ffyk<&e(bAg<=HV6{a+ zU54_eCgpdnkFPj{)I;t0^u}PT`V>7ybz*01)~Yn!BXp$|@$T4%rX>44aalhP46B&i zUoYM#UhCLx8?rV+#L4Wwe%;-J&Zz{u4`|+uG**{6k0(XO@AW40r6vz*@ma)qaGq-L z?mY26yVJXO%UfS%s@xdvK<8LjRPj2d6)h|CqV|fVzmnSjGXI;d0**t=57LqX}(Gr=_)3yW-zM0^hztkU<}*D-L3`& z#i6RUf;c}SY{$oW+uR*}G1@ASRNltoB&r9W2m4+|6u;0hfJyMl($|L_tDk%S1$LG9 zdMNQ>sg}IvmYNJlNyb(Mt?`zBGYTP^_YYt_OBf|DS=dd3q?6W79bj6#^z39YsxF>{h6?wHwDEK?rK-_uj>FRJ&W#JU4)xGKysyvgm;{ zm~Sn@SLfp)&=H#6AbWA3ClSNFgCOeVonWCx7(W{z;AJnI2~WPb~T$GA^&ys>8*Py&<`~@B`pN| zw#VjvzYkhX)r8Vi@xHp0xR3GwvRLIz%P5Y+wypIe9%k+h-&)_RB8N-oUZP2K`Wgqj z<%T;!oa$Tb@-TBzYkhz?)nQ%v!%)a@Cw|l4Q!XYli-I}xBkL3H%quq%Km8#c<;9(n z$~qUe6TLBfYaaP1qF(DahnCzCiu*~19i2CV4_`34nlLrSvK7usX^%wC`&$F@=!=cQ z%RjLhzJ2IKJA%;6Fe#w`fxAbMZ`-zf1(uqw-mB zX@O3EdMim2^s|mow;EFM(2CRY%ra^j!lD!Jied3;;{%w;qD;w{^4FRC1OmTB`|j z3#{U^za5!!e8Oi<-tSa{8M)Ej*E}cB)7HC=4*;GI;-GjhBK8iN2W6t7ZJ!pgpTnFP z3cgq=*;?g2!$iGsrkJa?`HobB9r+5aFT%nW^VK9ft<-PHanRdr3lcQe5Q~XUu9vc0 z7-&sG&jyiM(tdt`6x_7TlIRb89drKyPgVvTHpx3c57b(L{D0h22a$QD3= zWv55`H6W{1G-)yU-D9@r+vHiMo=x;)x6Tg&pC$y5qYiUn?E_~JJ$VHyXFZ#nV+%Is0GLD{8r|9(%EKlKFXC51;JO#!2z1;@zSCaDTpnH^2!1xf32L_--Ts#zyCwV$bb zu_TXdt}pk-L__RUc67dK0*WCcLq&HNo=XvHO(s`n@i%3mT|1{V``<3urigZ%$U#Df zEiohyY2M^CNYlfen|FA%k6#D7mXrV$Sx~&u>=#z5uB*N$u5u2vD&zQc*f zlJ}<8T(+aK{pWZt`sY?MZ3^pbrMK7ac5*)@xvcnYptpC0hFS^TomQ+mYe<{PUTNd! zAKV}~hIa)rm*og%VN8P0jh`;6TfWw!;gi#&Z|7MF>@QH-GIAfP@k=`SF>)#wUDTH~ zJ@PJ~#IrRyAy*RJZOBlXGylnm8|GJ^?*m5P-XmrtsV<{Qqokg~ft+v*MXUXQ<> z9-l+mFOnrF4cRJyKZ;ECBr1B8hQ(QEY~Tvq6M@<|9V)xLdK?5(bQp)pcJUn|c zUk$E-HLGkRPMQUkZwfSvixw+&NeW|-Ys%H}O3T*k$mu;R6x`&RKdDz-$T&zOpNKmSCm2B@wl=3=OOBBwx{a{5M{jmIQgO`msbg`2dt5~bA9 z?nu8iL845=Rkv^5XU^+nm{+_MV7SOnZ#718kjAKd^?TF<38LbhP?7q6(WIYYiQTZ_$uH%oRMGB-=IJ8d zqQ3*Q?{Pizb31rK@42czws;hD4q}IG0g+>ouBW3uW!yyawI4c>kK1%1eB~<}ac&Fw zGH3e|D*qmR$xAhGgG8Y>pNJHoV5X;+j9b1aD8s2s{}o=iiEG4Ydtxj(W5~L5=u(fH zUcg8YPBqlyccYvCYxpFJ?$D4I`etmaXvTmsees)SU?m?Ao zY|2a*VI@8_R3SXhW4$M$F_6Ndd4Hc@qu$W}frr!Y$*WOtQ-!DW?+EUhSz_>`*dGXs zI}6$aZGYvg5q5{0c1x-*d%v)Yzo@?aeQ37Oh{`OC>}$i~eh6*X_cnxZXUNUK176$vW42W?S| z=ezukDDC;8K5gx`gi}`cH2!syW+)j|?PAh{@x>-d;pi`f6m-w%1iQa&D%1TLnY7Z! za3$&=B5|e~^+!4dmP?KQlxdT8ze;U0WJ7 z*)U35`LBmtdbb>WOLd#Iz3*~G*MogCm1J><3OI-=9jpvjYRm@Bwu;&A_f926ma_JOX5z8g z^Ex7EXLC$H3ihMOiCYSE@+S%g7h}w`D*=hL%JtjRUeilJTLqupa%w}?g%2|w{^9QH z!OnHO=5sq!xB(YBkgPhDy`p2Gdtbn32w1XfloPu2+dc$l>o zKlkdc$6ytGM4Flpl4xqr9wE$iGM=?1CjI8*Ty^rv$M^yvovG~&4~}mBnJGgz9HGoo z{ZiLI{#J|Zd`}hLBL!=9?|L-STMq!eojb--?N|R2dea&eEi`NPy*>xX5XE9X@Wot& zzo2$rFP6~1JQq&>b?mSx_w`DFTJTB*u`lVw-Z`wi>te{jYI9!H*ny!aHL7B!v9`(7 zcGc5Rt}@e1B-tXVeL)sS2DKyBl-2>spTeit(eKosS_fX*D7&qq!G-~ua;{m`ElGLt z?GRPC7IJD~90>v&`^lWlyBK#_4d8dKRlxDU2{k9I*AD=dkFwBn;7Z_U0;hr-4HoZ2 z0A90|B7WfN^Hb?KS2^C8nPQj>zZ;N`eFm77B1)l~#~@!lcOm!R5gwnfT}0QNujUO+ zns_wYinCvzGzn4Z(uQji+h0D~rO~uvPyLx8iSejN3E(e4l5gXzzna!hRfSMZj<7f% zmXH^|AerZnpZ2Nn(mOGp0E1A$Pz7RwTBr-}F~Arw$L=oC`Wybqp8P(3F6_N z7nj^tOI`ioNur^^&aU{gF9LG*ANDnJAY(tUons?dPxPj`u4X6!st0$BQa#b|!$67v zQlKGOzgL>K)+&uasikrXA&FDzpqMcRMNT`J-|-i~41`8cb%->JOpWYdvGZ21-iQA(`_?tr@;mU-*QNf)@`qeA=gu;r!vJQd)|Xy~DyN zHpLh;U>@+O2CKE{`(?C^RvsxdP^a3S801Z-Nf(2voF5gdQ|sw;lG*&B)Co;l@ z+N60jQX|_3b@7=~Xk_wiYEP@;#hT{rr-J(0KT31Yngxl6Kq2^B3^C(gV3mX!Psx7q zFmt-E88fv#%si9Eb?t$1$~g-8UL!*QUruH(wsy!U6`ZL%`=@uvfaF$@Bs&dTBDVgRI$R{Mw{yXZgdO8IC$;3%ya^Lkh(0St%_i)kMLnBY43ZE z)qvA#O!8va*@xuSkFVgN(n$rZ%v0aBIBC2a3ZdmATkx1~erWn7BN};N&nBIEjGftt zQqX|u%-S1FEE#+#9W@D#<@o>MBwsvnw>{6#FPyWvX!{VhG-Fh@MtbSD({p~ExZhayDSS8V2c7xm`8||@oX!>(CQrt@WcFj@ z$>T){*8`&w_DBfA0eD4c^vU$8>Z*Ot&8N%J!8Y2gvc9ND0hXzm4q3ck1u6{sqPJw> zMP?+P`NNgA_6aj!!;M$v-B6cDVo6+cfk|ahmn}b5EIKZ~0A>AI`$w=gi320)(j*n@ ze_Fh-5epiMII{s>kI`1O)CwM3p&X_{|M^5uH*J#4x-JFQcqm2fekbEWDkDs6@+TjR zMf}c0l!Q+~qDfAgQOn6FnKBnb;WKZzCfvak*VX@=a5AcG95-n1&?!1b%sxx|&~@aP zc!K0v??7}tPz0)+#|T3#7Wzaw)19-8>#3=oFN?UjzPo8lOSneMZHIt1TP*Q@#ir56ViGTce49T{_pXfn*;5HL`zH_h@KZTO#4RJ+Cg*f_kwAd@6LVGSXXUrf4T(LE zY$U$W_by1L{L{SJrT^B5@$hO0zFA2Gn`37^&4^Jc(Fignlb7VCs45Z#_bqR^QPPf3 z?)8o~o>MgB2iJS;_9R?s!(wT%U+`LZgrzmo$lv^XChK6*_NA{p*bt$5QHLXfGtcEJ zD|@qR0T`^a3zZ9M)WdUCZ0Hc8Kyh-?1n&(m!bzfghL0w)rHn<=r)#r?NCL{*N`OiduJ!z~H&CwX(M&Et86=s_$m2Tl0fj zu2&F^Sop!ZEZC(vP`^sOTlKt3TlCez5tbkSkq50gu$5RMb*khN6Z`4iwsE~mXz^bM zKgCC;2=k4CqtD(Q4J6eUIoWCgizLdyyhsbnFu%FtJE89No1Cjbd$UnVe!U@)Z!H(% z7B}z$xLUchJKVYNJ064X8tu*^GBE*IlzE^RyU+`m4Nr)oq}LQ%*eHMTsth>!GxzC5_uY9LV$pbJF3yIYy8elxr+??x`JZ#zcS0&lvp9!h z%G?#9y;GgxUfAhZJXNLTV4+3-+@hj~wAPqL$OvIKJsh+V)ZnPVO~rlbtiTJbw?hjT zGkhGedf&7?|1N*=ardb#u>O3*9l3(^a9ORB3E;)=rpWR2B50vH@8I*11nza`wZcG& z6cBB??+{HcApyXjZzCN%fKnH@UCl;CW!1kei+vd#ze1I#=I-ByKWJZa>3itHW1J15 zv5>RJyM3m*$X;b*6(4y^ue-p*SxSuLgpW%wYVH)xjU7N^JjRbEk3&`%AKo`79Hg;@ zW6MF?17(2Ns*0a~h^Li_T(BRz+i}rRYNn+ncRm_QU}bI$e4RzoX!)P}Ax_K7xkuhu z9f1+sZ6i?Nz}{c@651_^LQnBRfsmD=dp>;6WsJ^2T~GBkfZ$HJ|Mj#z**8YqaU|@y zCpGfDbYyD>l3<+1*K{wd=c9C%2RbPj!&26k&e0JjB4IWrLWZX<1tA#|Ao~)`n=qA= z`eArlu22yUz%FjN4?%pUAMy2uf%7b{7L{`;GUf&BsO(12+Jqhwb8Oly+4sk-PXoSi zFI`>NpJ=h!>dRluwj>T4vTlaIw)ZfLelJ>d4ws+#Ufd%`-qNDes6M@XDMv8|L$zpQ zc)MGJg6c^4T!W4?g)lVFZJE5-Ew&RX*(eRTMS7XoBka?L>ia{xv22$+=M;m%2qAPk zQe~XY4HYWTrciK$CUSI1OCe=e#{O9eWoEde2q)pCk|!ujk{`k3=%3R&dw3b=#|^-`F;l4Hz}W@CFzU-+LsRAp0vF>QQ8E3 zUMfPIxui1Ctt&v2sr{wrv*B&ldU%w~`uetIFK02eJLAcFV3r;(-z06XVHMYy#vClm zyYcsfqNfKyne8xVx>a1+gtJw;i2Q|m(4O8}F;Q-`7oyeNxbzeoFy43PDUYzelz z=XSM`M$u(s+Ummdr-zJeV6jW6RaFrebBKsa z5|nD!AA?>h>FYSRVbVH|EWf8pXz9gv(thyK&C|-wk|dq9rd|(+Jd}xn%4X1^_ZR(l zbY(N7b=A9wH~P+uw(`-LbhtFReKbQmoDlCJQZTgV9>?c+ZO@|m5YlIt40a6Mvm!2d zR#)h)ORZWf$@Psv?jbL%oXLvq=K zHR@W0zxNfsv%YZJwFZUg+g)v6}L}>Diq7tbf3$o?D`~TBw57O^kd=kvm>8 z_xk)}bkS2EJ%Ro>OY9@;&S>}GKZ6MK!bws z>vGBur9MFXQG1A)`$1sC^Ix~tZwKO_m8yfir_2?8o6l?i^3fCZWy=mP%iH5Ek&p(> zuErr*aRy5)=SBu7j$kJCd;Y%9N}NKtF zp85Q8)!IL>XlH?F$Bu6?s0R#VdG^=OAmj07bJL`u@1BoaNW57VD=!AQT!`+vZAIqO&S9YtE0mgQ~pm6u5D9G5~qOqjz~b@%f@EYbY*92DazEdNnry oXO=ONa{Yh(?Ee!7NAH<1N#!=ZHyH$@|6~Th)*5A1XW^Chf0A}UlmGw# literal 0 HcmV?d00001 From 4f6dea72b913083b598b95d4bb63222181adddcc Mon Sep 17 00:00:00 2001 From: fen4o Date: Thu, 3 Nov 2016 12:22:29 +0200 Subject: [PATCH 14/45] Fix broken rkt networking link --- docs/getting-started-guides/scratch.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/getting-started-guides/scratch.md b/docs/getting-started-guides/scratch.md index 72a9ee3125..84b0ffb5af 100644 --- a/docs/getting-started-guides/scratch.md +++ b/docs/getting-started-guides/scratch.md @@ -381,7 +381,7 @@ The minimum version required is [v0.5.6](https://github.com/coreos/rkt/releases/ minimum version required to match rkt v0.5.6 is [systemd 215](http://lists.freedesktop.org/archives/systemd-devel/2014-July/020903.html). -[rkt metadata service](https://github.com/coreos/rkt/blob/master/Documentation/networking.md) is also required +[rkt metadata service](https://github.com/coreos/rkt/blob/master/Documentation/networking/overview.md) is also required for rkt networking support. You can start rkt metadata service by using command like `sudo systemd-run rkt metadata-service` From 35a14cda34c5949320ba1d20e6a1b184e58d71e2 Mon Sep 17 00:00:00 2001 From: devin-donnelly Date: Thu, 3 Nov 2016 15:06:08 -0700 Subject: [PATCH 15/45] WIP: New Docs Landing Page and ToC Organization. (#1468) New Docs Landing Page and ToC Organization. --- _data/concepts.yml | 1 + _data/docs-home.yml | 5 ++ _data/globals.yml | 1 + _data/guides.yml | 4 +- _data/tasks.yml | 1 + _data/tools.yml | 1 + _data/tutorials.yml | 1 + _layouts/docwithnav.html | 11 ++- docs/index.md | 156 +++++++++------------------------------ docs/user-guide/index.md | 23 +++--- 10 files changed, 61 insertions(+), 143 deletions(-) create mode 100644 _data/docs-home.yml diff --git a/_data/concepts.yml b/_data/concepts.yml index a69276201e..f9422daa98 100644 --- a/_data/concepts.yml +++ b/_data/concepts.yml @@ -1,4 +1,5 @@ bigheader: "Concepts" +abstract: "Detailed explanations of Kubernetes system concepts and abstractions." toc: - title: Concepts path: /docs/concepts/ diff --git a/_data/docs-home.yml b/_data/docs-home.yml new file mode 100644 index 0000000000..1cb8c9b05a --- /dev/null +++ b/_data/docs-home.yml @@ -0,0 +1,5 @@ +bigheader: "Kubernetes Documentation" +abstract: "Documentation for using and learning about Kubernetes." +toc: +- title: Kubernetes Documentation + path: /docs/ diff --git a/_data/globals.yml b/_data/globals.yml index 73978ea750..2994e8b519 100644 --- a/_data/globals.yml +++ b/_data/globals.yml @@ -1,4 +1,5 @@ tocs: +- docs-home - guides - tutorials - tasks diff --git a/_data/guides.yml b/_data/guides.yml index e31411ea48..85c5dd4103 100644 --- a/_data/guides.yml +++ b/_data/guides.yml @@ -1,8 +1,8 @@ bigheader: "Guides" -abstract: "How to get started, and achieve tasks, using Kubernetes" +abstract: "How to get started, and accomplish tasks, using Kubernetes." toc: - title: Guides - path: /docs/ + path: /docs/user-guide/ - title: Getting Started section: diff --git a/_data/tasks.yml b/_data/tasks.yml index 38e1a89c68..4c6cd1b709 100644 --- a/_data/tasks.yml +++ b/_data/tasks.yml @@ -1,4 +1,5 @@ bigheader: "Tasks" +abstract: "Step-by-step instructions for performing operations with Kuberentes." toc: - title: Tasks path: /docs/tasks/ diff --git a/_data/tools.yml b/_data/tools.yml index 8993e091bb..cf2afca34c 100644 --- a/_data/tools.yml +++ b/_data/tools.yml @@ -1,4 +1,5 @@ bigheader: "Tools" +abstract: "Tools to help you use and enhance Kubernetes." toc: - title: Tools path: /docs/tools/ diff --git a/_data/tutorials.yml b/_data/tutorials.yml index 01440b09d7..ae9401c918 100644 --- a/_data/tutorials.yml +++ b/_data/tutorials.yml @@ -1,4 +1,5 @@ bigheader: "Tutorials" +abstract: "Detailed walkthroughs of common Kubernetes operations and workflows." toc: - title: Tutorials path: /docs/tutorials/ diff --git a/_layouts/docwithnav.html b/_layouts/docwithnav.html index c2f74c1bb4..877ccc4e63 100755 --- a/_layouts/docwithnav.html +++ b/_layouts/docwithnav.html @@ -11,13 +11,14 @@
{{ site.data[foundTOC].abstract }}
From e3c87235a548a6a698f6dc73a40485f691fefd78 Mon Sep 17 00:00:00 2001 From: Janet Kuo Date: Mon, 14 Nov 2016 10:08:00 -0800 Subject: [PATCH 27/45] Fix example test failure: rename ScheduledJob to CronJob --- test/examples_test.go | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/test/examples_test.go b/test/examples_test.go index 1853cdaf0a..7e5660c4f9 100644 --- a/test/examples_test.go +++ b/test/examples_test.go @@ -127,11 +127,11 @@ func validateObject(obj runtime.Object) (errors field.ErrorList) { t.Namespace = api.NamespaceDefault } errors = expvalidation.ValidateDaemonSet(t) - case *batch.ScheduledJob: + case *batch.CronJob: if t.Namespace == "" { t.Namespace = api.NamespaceDefault } - errors = batch_validation.ValidateScheduledJob(t) + errors = batch_validation.ValidateCronJob(t) default: errors = field.ErrorList{} errors = append(errors, field.InternalError(field.NewPath(""), fmt.Errorf("no validation defined for %#v", obj))) @@ -242,7 +242,7 @@ func TestExampleObjectSchemas(t *testing.T) { "redis-resource-deployment": &extensions.Deployment{}, "redis-secret-deployment": &extensions.Deployment{}, "run-my-nginx": &extensions.Deployment{}, - "sj": &batch.ScheduledJob{}, + "sj": &batch.CronJob{}, }, "../docs/admin": { "daemon": &extensions.DaemonSet{}, @@ -272,7 +272,7 @@ func TestExampleObjectSchemas(t *testing.T) { "../docs/user-guide/node-selection": { "pod": &api.Pod{}, "pod-with-node-affinity": &api.Pod{}, - "pod-with-pod-affinity": &api.Pod{}, + "pod-with-pod-affinity": &api.Pod{}, }, "../docs/admin/resourcequota": { "best-effort": &api.ResourceQuota{}, From 27840ef8fe81b16da93f01af1456ea250dfbd51a Mon Sep 17 00:00:00 2001 From: Quinton Hoole Date: Mon, 10 Oct 2016 16:27:54 -0700 Subject: [PATCH 28/45] Improvements to Federated Ingress User Guide. --- .../federation/federated-ingress.md | 74 ++++++++++++++++--- 1 file changed, 62 insertions(+), 12 deletions(-) diff --git a/docs/user-guide/federation/federated-ingress.md b/docs/user-guide/federation/federated-ingress.md index 87965a3fc7..6198de1817 100644 --- a/docs/user-guide/federation/federated-ingress.md +++ b/docs/user-guide/federation/federated-ingress.md @@ -64,12 +64,12 @@ healthy backend service endpoint at all times, even in the event of pod, cluster, availability zone or regional outages. -Note that in the - case of Google Cloud, the logical L7 load balancer is not a single physical device (which - would present both a single point of failure, and a single global - network routing choke point), but rather a [truly global, highly available - load balancing managed service](https://cloud.google.com/load-balancing/), - globally reachable via a single, static IP address. +Note that in the case of Google Cloud, the logical L7 load balancer is +not a single physical device (which would present both a single point +of failure, and a single global network routing choke point), but +rather a +[truly global, highly available load balancing managed service](https://cloud.google.com/load-balancing/), +globally reachable via a single, static IP address. Clients inside your federated Kubernetes clusters (i.e. Pods) will be automatically routed to the cluster-local shard of the Federated Service @@ -86,13 +86,13 @@ You can create a federated ingress in any of the usual ways, for example using k ``` shell kubectl --context=federation-cluster create -f myingress.yaml ``` - +For example ingress YAML configurations, see the [Ingress User Guide](/docs/user-guide/ingress/) The '--context=federation-cluster' flag tells kubectl to submit the request to the Federation API endpoint, with the appropriate credentials. If you have not yet configured such a context, visit the [federation admin guide](/docs/admin/federation/) or one of the [administration tutorials](https://github.com/kelseyhightower/kubernetes-cluster-federation) -to find out how to do so. TODO: Update links +to find out how to do so. As described above, the Federated Ingress will automatically create and maintain matching Kubernetes ingresses in all of the clusters @@ -147,17 +147,28 @@ Events: 2m 2m 1 {loadbalancer-controller } Normal CREATE ip: 130.211.5.194 ``` -Note the address of your Federated Ingress +Note that: + +1. the address of your Federated Ingress corresponds with the address of all of the underlying Kubernetes ingresses (once these have been allocated - this may take up to a few minutes). - -Note also that we have not yet provisioned any backend Pods to receive +2. we have not yet provisioned any backend Pods to receive the network traffic directed to this ingress (i.e. 'Service Endpoints' behind the service backing the Ingress), so the Federated Ingress does not yet consider these to be healthy shards and will not direct traffic to any of these clusters. +3. the federation control system will +automatically reconfigure the load balancer controllers in all of the +clusters in your federation to make them consistent, and allow +them to share global load balancers. But this reconfiguration can +only complete successfully if there are no pre-existing Ingresses in +those clusters (this is a safety feature to prevent accidental +breakage of existing ingresses). So to ensure that your federated +ingresses function correctly, either start with new, empty clusters, or make +sure that you delete (and recreate if necessary) all pre-existing +Ingresses in the clusters comprising your federation. -## Adding backend services and pods +#Adding backend services and pods To render the underlying ingress shards healthy, we need to add backend Pods behind the service upon which the Ingress is based. There are several ways to achieve this, but @@ -175,6 +186,16 @@ kubectl --context=federation-cluster create -f services/nginx.yaml kubectl --context=federation-cluster create -f myreplicaset.yaml ``` +Note that in order for your federated ingress to work correctly on +Google Cloud, the node ports of all of the underlying cluster-local +services need to be identical. If you're using a federated service +this is easy to do. Simply pick a node port that is not already +being used in any of your clusters, and add that to the spec of your +federated service. If you do not specify a node port for your +federated service, each cluster will choose it's own node port for +its cluster-local shard of the service, and these will probably end +up being different, which is not what you want. + You can verify this by checking in each of the underlying clusters, for example: ``` shell @@ -258,6 +279,35 @@ Check that: `service-controller` or `replicaset-controller`, errors in the output of `kubectl logs federation-controller-manager --namespace federation`). +#### I can create a federated ingress successfully, but request load is not correctly distributed across the underlying clusters + +Check that: + +1. the services underlying your federated ingress in each cluster have + identical node ports. See [above](#creating_a_federated_ingress) for further explanation. +2. the load balancer controllers in each of your clusters are of the + correct type ("GLBC") and have been correctly reconfigured by the + federation control plane to share a global GCE load balancer (this + should happen automatically). If they of the correct type, and + have been correctly reconfigured, the UID data item in the GLBC + configmap in each cluster will be identical across all clusters. + See + [the GLBC docs](https://github.com/kubernetes/contrib/blob/master/ingress/controllers/gce/BETA_LIMITATIONS.md#changing-the-cluster-uid) + for further details. + If this is not the case, check the logs of your federation + controller manager to determine why this automated reconfiguration + might be failing. +3. no ingresses have been manually created in any of your clusters before the above + reconfiguration of the load balancer controller completed + successfully. Ingresses created before the reconfiguration of + your GLBC will interfere with the behavior of your federated + ingresses created after the reconfiguration (see + [the GLBC docs](https://github.com/kubernetes/contrib/blob/master/ingress/controllers/gce/BETA_LIMITATIONS.md#changing-the-cluster-uid) + for further information. To remedy this, + delete any ingresses created before the cluster joined the + federation (and had it's GLBC reconfigured), and recreate them if + necessary. + #### This troubleshooting guide did not help me solve my problem Please use one of our [support channels](http://kubernetes.io/docs/troubleshooting/) to seek assistance. From 7f8a38b7432a059ee9b0116469fadeca6ca9b017 Mon Sep 17 00:00:00 2001 From: Steve Perry Date: Tue, 15 Nov 2016 12:26:42 -0800 Subject: [PATCH 29/45] Write style guide. (#1619) --- _data/support.yml | 2 + docs/contribute/style-guide.md | 203 +++++++++++++++++++++++++++++++++ 2 files changed, 205 insertions(+) create mode 100644 docs/contribute/style-guide.md diff --git a/_data/support.yml b/_data/support.yml index e280c827d7..1b8e80699a 100644 --- a/_data/support.yml +++ b/_data/support.yml @@ -14,6 +14,8 @@ toc: path: /docs/contribute/stage-documentation-changes/ - title: Using Page Templates path: /docs/contribute/page-templates/ + - title: Documentation Style Guide + path: /docs/contribute/style-guide/ - title: Troubleshooting section: diff --git a/docs/contribute/style-guide.md b/docs/contribute/style-guide.md new file mode 100644 index 0000000000..219d62560d --- /dev/null +++ b/docs/contribute/style-guide.md @@ -0,0 +1,203 @@ +--- +--- + +{% capture overview %} +This page gives writing style guidelines for the Kubernetes documentation. +These are guidelines, not rules. Use your best judgment, and feel free to +propose changes to this document in a pull request. + +For additional information on creating new content for the Kubernetes +docs, follow the instructions on +[using page templates](/docs/contribute/page-templates/) and +[creating a documentation pull request](/docs/contribute/create-pull-request/). +{% endcapture %} + +{% capture body %} + +## Documentation formatting standards + +### Capitalize API objects + +Capitalize the names of API objects. Refer to API objects without saying +"object." + + + + + +
DoDon't
The Pod has two Containers.The pod has two containers.
The Deployment is responsible for ...The Deployment object is responsible for ...
+ +### Use angle brackets for placeholders + +Use angle brackets for placeholders. Tell the reader what a placeholder +represents. + +1. Display information about a pod: + + kubectl describe pod + + where `` is the name of one of your pods. + +### Use bold for user interface elements + + + + + +
DoDon't
Click Fork.Click "Fork".
Select Other.Select 'Other'.
+ +### Use italics to define or introduce new terms + + + + + +
DoDon't
A cluster is a set of nodes ...A "cluster" is a set of nodes ...
These components form the control plane.These components form the control plane.
+ +### Use code style for filenames, directories, and paths + + + + + + +
DoDon't
Open the envars.yaml file.Open the envars.yaml file.
Go to the /docs/tutorials directory.Go to the /docs/tutorials directory.
Open the /_data/concepts.yaml file.Open the /_data/concepts.yaml file.
+ +## Code snippet formatting + +### Use code style for inline code and commands + +For inline code in an HTML document, use the `` tag. In a Markdown +document, use the backtick (`). + + + + + +
DoDon't
Set the value of the replicas field in the configuration file.Set the value of the "replicas" field in the configuration file.
The kubectl run command creates a Deployment.The "kubectl run" command creates a Deployment.
+ +### Don't include the command prompt + + + + +
DoDon't
kubectl get pods$ kubectl get pods
+ +### Separate commands from output + +Verify that the pod is running on your chosen node: + + kubectl get pods --output=wide + +The output is similar to this: + + NAME READY STATUS RESTARTS AGE IP NODE + nginx 1/1 Running 0 13s 10.200.0.4 worker0 + + +{% comment %}## Kubernetes.io word list + +A list of Kubernetes-specific terms and words to be used consistently across the site. + + + + +
TermUseage
TBDTBD
{% endcomment %} + + +## Content best practices + +This section contains suggested best practices for clear, concise, and consistent content. + +### Use present tense + + + + +
DoDon't
This command starts a proxy.This command will start a proxy.
+ +Exception: Use future or past tense if it is required to convey the correct +meaning. + +### Use active voice + + + + + +
DoDon't
You can explore the API using a browser.The API can be explored using a browser.
The YAML file specifies the replica count.The replica count is specified in the YAML file.
+ +Exception: Use passive voice if active voice leads to an awkward construction. + +### Use simple and direct language + +Use simple and direct language. Avoid using unnecessary phrases, such as saying "please." + + + + + + + +
DoDon't
To create a ReplicaSet, ...In order to create a ReplicaSet, ...
See the configuration file.Please see the configuration file.
View the Pods.With this next command, we'll view the Pods.
+ +### Address the reader as "you" + + + + + +
DoDon't
You can create a Deployment by ...We'll create a Deployment by ...
In the preceding output, you can see...In the preceding output, we can see ...
+ +## Patterns to avoid + +### Avoid using "we" + +Using "we" in a sentence can be confusing, because the reader might not know +whether they're part of the "we" you're describing. + + + + + + +
DoDon't
Version 1.4 includes ...In version 1.4, we have added ...
Kubernetes provides a new feature for ...We provide a new feature ...
This page teaches you how to use pods.In this page, we are going to learn about pods.
+ +### Avoid jargon and idioms + +Some readers speak English as a second language. Avoid jargon and idioms to help make their understanding easier. + + + + + +
DoDon't
Internally, ...Under the hood, ...
Create a new cluster.Turn up a new cluster.
+ +### Avoid statements about the future + +Avoid making promises or giving hints about the future. If you need to talk about +an alpha feature, put the text under a heading that identifies it as alpha +information. + +### Avoid statements that will soon be out of date + +Avoid words like "currently" and "new." A feature that is new today might not be +considered new in a few months. + + + + + +
DoDon't
In version 1.4, ...In the current version, ...
The Federation feature provides ...The new Federation feature provides ...
+ +{% endcapture %} + + +{% capture whatsnext %} +* Learn about [writing a new topic](/docs/contribute/write-new-topic/). +* Learn about [using page templates](/docs/contribute/page-templates/). +* Learn about [staging your changes](/docs/contribute/stage-documentation-changes/) +* Learn about [creating a pull request](/docs/contribute/create-pull-request/). +{% endcapture %} + +{% include templates/concept.md %} From d10fed8566acd1c407ea869a9a33238695a20fd5 Mon Sep 17 00:00:00 2001 From: ankurshukla Date: Tue, 15 Nov 2016 13:03:34 -0800 Subject: [PATCH 30/45] Added new set of Tech and Services Partners --- README.md | 9 +-------- _includes/partner-script.js | 23 ++++++++++++++++++++++- images/square-logos/datadog.png | Bin 13224 -> 12214 bytes images/square-logos/endocode.png | Bin 0 -> 5231 bytes images/square-logos/giant_swarm.png | Bin 0 -> 9772 bytes images/square-logos/mirantis.png | Bin 0 -> 17311 bytes 6 files changed, 23 insertions(+), 9 deletions(-) create mode 100644 images/square-logos/endocode.png create mode 100644 images/square-logos/giant_swarm.png create mode 100644 images/square-logos/mirantis.png diff --git a/README.md b/README.md index 381cd46567..2801eaead0 100644 --- a/README.md +++ b/README.md @@ -174,14 +174,7 @@ example. If creating an image for a doc, follow the section on "Docker images" from the Kubernetes repository. ## Partners -Kubernetes partners refers to the companies who contribute to the Kubernetes core codebase and/or extend their platform to support Kubernetes. Partners can get their logos added to the partner section of the [community page](http://k8s.io/community) by following the below steps and meeting the below logo specifications. Partners will also need to have a URL that is specific to integrating with Kubernetes ready; this URL will be the destination when the logo is clicked. - -* The partner product logo should be a transparent png image centered in a 215x125 px frame. (look at the existing logos for reference) -* The logo must link to a URL that is specific to integrating with Kubernetes, hosted on the partner's site. -* The logo should be named *product-name*_logo.png and placed in the `/images/community_logos` folder. -* The image reference (including the link to the partner URL) should be added in `community.html` under `
...
`. -* Please do not change the order of the existing partner images. Append your logo to the end of the list. -* Once completed and tested the look and feel, submit the pull request. +Kubernetes partners refers to the companies who contribute to the Kubernetes core codebase, extend their platform to support Kubernetes or provide managed services to users centered around the Kubernetes platform. Partners can get their services and offerings added to the [partner page](https://k8s.io/partners) by completing and submitting the [partner request form](https://goo.gl/qcSnZF). Once the information and assets are verified, the partner product/services will be listed in the partner page. This would typically take 7-10 days. ## Thank you! diff --git a/_includes/partner-script.js b/_includes/partner-script.js index f7447c63b7..5763047efc 100644 --- a/_includes/partner-script.js +++ b/_includes/partner-script.js @@ -175,6 +175,20 @@ link: 'https://aporeto.com/trireme', blurb: 'Aporeto makes cloud-native applications secure by default without impacting developer velocity and works at any scale, on any cloud.' }, + { + type: 0, + name: 'Giant Swarm', + logo: 'giant_swarm', + link: 'https://giantswarm.io', + blurb: 'Giant Swarm provides fully-managed Kubernetes Clusters in your location of choice, so you can focus on your product.' + }, + { + type: 0, + name: 'Mirantis', + logo: 'mirantis', + link: 'https://content.mirantis.com/Containerizing-OpenStack-on-Kubernetes-Video-Landing-Page.html', + blurb: 'Mirantis builds and manages private clouds with open source software such as OpenStack, deployed as containers orchestrated by Kubernetes.' + }, { type: 1, name: 'Apprenda', @@ -237,7 +251,14 @@ logo: 'skippbox', link: 'http://www.skippbox.com/services/', blurb: 'Skippbox brings its Kubernetes expertise to help companies embrace Kubernetes on their way to digital transformation. Skippbox offers both professional services and expert training.' - } + }, + { + type: 1, + name: 'Endocode', + logo: 'endocode', + link: 'https://endocode.com/kubernetes/', + blurb: 'Endocode practices and teaches the open source way. Kernel to cluster - Dev to Ops. We offer Kubernetes trainings, services and support.' + } ] var isvContainer = document.getElementById('isvContainer') diff --git a/images/square-logos/datadog.png b/images/square-logos/datadog.png index 82d6d11aaf93168cddb9ba48dac2458cd077562a..aeab0f227f42eeb6fcacd2635bb3835ae2dc0451 100644 GIT binary patch literal 12214 zcmZvCbx<5n)a?=yG`MSUcXxs>?(Xis1h?Q4U>A1?u0evky9W#I*F#^(Id@Ksnu;763NZ=*06>!mN^8Kz64;f7gaEtu3;|bR1F{=X-xB~p z#rf}o17zh80sttg_EJ)6YId$5S5G@vHwt+vDGE0aR~vgLYXHFKcdn+bmL?Qm^ltN7 zQZYItMbT9Q4~arUGA;xcOvgltjHwV!S-6U)-iIY4^PXh1AR0a)AtWA8o#kUJ$_nB> z<@bcJqUhN1ryc(y$ED7@!>QM%WwE2`tK7yJ#9pM2X+Umu-jI(KlH^$1VIzaXJNs;( zLs4kl0GLRPR+OGE)Np|NAQ2H}s$Rry0G!VPDiWYqIlGsGH0llGuVkhXTu3-vF9g}u(a3%sIdaf=z|Xz03o^enf`D9!xTz9xB@VM z0>?U98ek_3sGiY@l>z8+0B~&-2L%ButN;#qT^k^vt_9FLjfvU-K>7&aP>+sa1ibeL zn2l0X`vAf-0l2c4x*`|MwU|dtu$9Ve5NV?klnFINVs%H<)ny}Rn3gAC#^p1AXP)($ zt=~73h%1x_WBY0n04MkrYa>&b)V8{Mu)n{eFeqtg zKCT=1YTIwxtNY^g5G3+^ceCBSOB2dv94hzze!GA2TB(F|27(-Jv3{5+_uPW|_Dr@w zH7IY`4z3qqOd zBuK9N2w8d&Nc%pl_n|(U@r6s$4E9T6N}^bWkb0OgR)3Xbh@7e;V=`m^^_h&PSKTf$ z*$thoU%M4u)D7d&94W7lp)(Zuy~HRIjydJ;XgG^lJ?aD`g3O3}%0A$G>^K^Vv1nX% zDn&3akW(X8i$WVnb}8!dPB6+ywjC@y3gG(s9N8(ynG)KdF8b?ZlT=$dKXJJ1chp%c z5NB2*enIxE$4_F>c)r4;*(y7X--%*UBOJYd2obplojBNr2PEqSSdfz~C+exHKUNJB z*XdNlcf{OS&eVIq#}1Y0Mdo|&ElFWcE&H$=0}DPSMR5%N-kIY zN!O0`Fi2>@#TlgsWT9TdPQ+oy^pBPra9nOL`|?U#vBcJuf@&qCpL1smSI5 z5>qFS{kC#!Kom@2p|T7Yxyzt2?w~+)fO)`kfMA`$ zC81rErLj?7sD;IwXdqcr*sSqGJ*pgs$16*4DhyvH@JHf!&?`k3XPxtc_H1 zmGoKN`3b1ppY%WQ*WMq|BJsjTqHP(7vI!9hZHc@Ie`F|?5-r3Ljc9T3a~@|LWnyHy zYU?pIGX>y*$5B!hQt47jnAo){DoZOjD^aztv?sI@D@-&4Yh|^$HTNozf2CDvSLUi6 zYm#W%RtT3zRoa%jYKa#I>A%Dv=np6js7e*97u1{Xbq-UYS`sz-uuwXpRYiJ*@($9~ zPpt-gVd}_uA?t_}t*j^%l;T$4)@{_dO0f|~+{Zj_;v z;VX*@-PE#GI+l z9VMNj@?Yh%EDhzhQpQqYC1ItJsmZDDQ%jk=*`l1wTu0eQ*~{5mEoKHj1}x3x&8^n* z4f_VA`e)6ZE)fQT`XzeW2F8uDHMupkOC2>GWmsixD#Lkq z=5KPd8?*aQ%h!IK@txV9MXl#g^5iq->2rI|4vb1|s~^mXqvoe_IBr!3sY`FmSNTP9QoT=PsLwxoio->lyJ z-sS*hA+&I|@G+qhp@#3C?)?HNdlJNj)~s(ER!=433)&>`Boe>Yd_Cx^p|l%4Nvff| zJIa^_PurtMs0IC5nng4}G>;ny`bZxU6`>m`p5PX(M?A-w&g(t5F6Fr;{V1I#y_s#x zsVew`Re_gDtdh+^YhJA z2K%V7n{VNC|8+zeegzvgi%{BUrDzp_lz$SKOy&6Tw3j&EvNRP1wnbI~hO zMoi=?mp~?EKgbEVlh>DKh`WU4pe>xfBB@geO=%-rC=WmD3xzY6s8APY7zCW;E?G&b zoZM{t*Y?on%S5$xM8oTlOI!;(c`D+VV`EbF2YptQX35e&S_TZuOo$D(*19%5Q!(R$ zX;pd5Q>T-S<7-L$Ip_Irr`Y}2JCNp*NJU*%GQGpDY6rmv5qIA;ls&u;%#k_>)grC2 z%}maVs#T}D>9s*!bbiW?p(cIfU;CQi9FI}8Q4S&kkJpz^jaBv7^|zYLA0u-~#PDKb zYjM{I((!^<7}k9bNTOWFdotHkepXZswpq>C%-cW@H(B!x(bFo^57W_I&{lu{MBCE& zd3tjzv&No2#jw^9)-2Ul)FaZ`XZ>;dx8J9w3sI-y3uvz=Xy7Ece91R5#&Mt zGCk3;>P>%E)2Z5Ndj0uSVpVD*YEj(qb$#W0J=mewyWnbWUG5GcB5dXre3OVF^&v4c zaUv2e@;FaGOi*xF+~WoMTqZSRHA9z2@cR9AqLu)kJ?MV%czJp*W3Dj$GCjG+IiTd_ zZ|^bDkG0|cVe;-VA8t;QYn`XlKJRsJvh$)X4AuPrieQE}yNCK4&DpZ2WTNEcxA*WF zSI?H*tZOj7Nnr`pPzC_}XaRtbFaY4;4K_jn051*z;IA0~Ad~?B5V%52hhzZ&njLv* z2`!)BC%N7kMz(?H;$=$ttE*fcRjdYHH7xoN)-i~4o37J32j9wjq{wYdws6dks^hf$ z>Fa|=5KN?Sh?!C&nB+@1iASoFlRjjz3T2gGSt z4Ff_f6$yb)YH*WyR#kvQfc57JpYwMqlxsb{bn?ENisRy+SKfm=znbBFPNguezc-~4 zW6#+ay@=i`hD~S`yB~f(`xn%x&}8C0mg1lz0Yz@8=Ree&O-JzMbCeY3G%GMoyXq=D zC7W7|9j{nxdfpcy{BH};ms|jYyif=A{c2_8iif^G8wHY(^`G{Hw7EH6GJSATsy%YZ zeFSc^)42X$kqqZnY)Yue5%dygkF0AO-TtZNWomw0VovYD7^Ej7Mbc zpwajXlENLA9fJ1NX^3u7NDkQMNetDB{HpOO|M?LOS9c%F>D=;3aisM+%|Yz(fLXt; zGu*cO4jpQRuqbBV2uhF{&m1BMT`O%*Rx+SY9xs3jHLzGVG6NexvZ_YQAQ0KvbE6_- z?I?Uf!QjLk8QMO9zg}}KJ?w(D>QO+rlfF7g#j%EFl`pARgl~2N}ke00-k9GJf3LsjllA>9z|?0RHFWPw@t6W&j<*EgNUm<0J zT1CS5h>J$206slW(&nr$NS6!8I!vr~v*4?ku23CJ7LY|-Sm_L>BmwJAAWXF#-gpru zo9e-a-33W4dBTIc+SUR8LcLO^*!QQ(?iP#|WSG*V>lw9l2pZ2Khm7jF9WdMQWXMxG zkW-!z2DRl~`T(xosIH-Lcc!9!sopicQ1WPlhq!ek?=6|z5zb@@#8+C-UgBFXoE zdV%NPpy5D>^x_BFNkg4}Jvx2!igY^xiGdiELpJ2P!`~6r-41Z{cf6YZb>I30#n`%S zDEXd#aYt2HMQTRFUik(VeRaC;NV|x@tfs4b~PD4=l@h%R7kJu z+Udp@J~B;zF8uyyf*jB_G~2(&1CM{RCCexvJXS{L#P6V0{+)u>kBV*he4~NK>@+1* zN;s`-$vIN?AL32<%lYf7La7q!)@&xcu!RsS62Tl)@8~Jm~hDF7%X?!e-1! zT*lhh$|F5(2%m8VT&G5tT3F1KKjC7`C(*>D*+$blt*gSJIe|RZFqj+M9$Iy-Kjzx~ zguhWkiK{()rd1(6KTRFn5;2@?9Eswsw6gz0;1jSSkDz$+kI_vbDxz;)o5ftE$CsW9 ze6p#o6s>z(Cl3B#3$!txtYz}8um#KW7V2yj&}Dw`3pRmiN3;)(ysaj?!Wp?w8w{Fc zHtcST7aYq1=Jv6Puh+KdiW&9=PdhqWVy^Jp373_2 zUUa%BK4UIy3($Aw`X%|I`gcxmev!N6Yqbuadd@Cj6?S;pKq!TSymQV2b_9d|EfV+~ zJy5gg8xQ}jB7i6rm$e^L#Vh#3&IYYfNb{VeWQ9)_L2x?`E4RCS(Xr5&UTnhfZ2ZUK zB0=fB3v`EDzN`Hnmp$*E^{L0~s&Cf^zrqiEIdyHDo1BSwb?m(>b5=G6t(6VDIYJ~9Kx zz(f2LP1KGkSIJ=D)VtyD_y?a$kL-V~f5#(Ed>#nfT?BY9fL}Lj8A2~b4357tXlAQx zS;&ujMyEhn3M;X*gRj|KUk|rig(s{)&CeG671B&)bra1;==D%TL;I?8i6$b*G_*Oi zcENyP(id}LNvG~I_QPPm`~28PXo_sBaO|2mYoibIf2d77LY8FVQ=c7t&o&eAMz%Zm ztDMPap~q?O-|Yen?XpTW)wujB3|r}km&)#_ndb{6(d7APHf{+L_Jii|Vmf-<1Q$sv z87ju<)pq1`tk=bk$~BqkLiF<;v}`FEx#JgNBz3Hu8Yb6f$ct}3GcT_PiLnZ>$ZPBwojM}#{y?(_ zH&)DRvi8$e2&nG@_Bb4nJl$7Aqn6)xTeYh>KD==4l9z}{Z}<=q?r#A*e}&c{r!T3} zED;vTZ7+R_jD{S3C#yEP>dq*Bvd8&w700r&jG4ea){NGL>>8cB`;Sec!7_|%d1;5SWaL3YJ_u5H8^qxz-s%Z24;412mnLZ-6x`As@oTEQT zNUI~*&Rw)~DR4i@?^4>rELqMye}03X^2m^FnzR&jjq%AZ)QV;C$1Z)yyLfx7#%B06rxRZj{y8*|aXzr{ zWTn>Gcaw7w<9}IE1lu{#& z3)lWfk&$u?qV7emjOCPgEi77cg<@B8n)k=t#>=pGwY+;I7hViEmPymQJk?BYzdHZ9 zmURTVInupV3kG(49bcq)!$7Wa@e2Htlxg;!R6Ud(87yV$flf$IeWV74p5^8*}_& z8b5Mwb(OeP^k79F%T!d(m6;MrKQnf7_k%pe;^PO)Wc~f(UKqV7a{o>O#QsIP+1x0dtPFB;#(A%H;q)OymO1luO=4kmR8rn%abYMgC*PN`oRr_8pj6g zfgth^x$jVIwe3J)&h0DnnV!VM*BE3NkXpq@cypo*&izs347|}TSi#DdX#tio}W|-wX;W4UXWmU zXXaVCkJmQFeygE(Smb0*sjV1iUwfM1^t#hgUb*`@E0A+yRbkwz+QWlZoQlo|^knaF z%hO<|E$Df8aP$*e(xSC@z2b9{zqu{!PrT}Lgu=I3AJBi`$ct#;3)2LD?EVxsv5nrQ zCtw{b&g*R3amLC-RIg&Sdbi?yQ- zOl4li-av>I*dr3NFIroryFuq2Q4CNu$J<1peCc1bHt!TAgAC?2~ckXs{tA-L{ygnww!(E1A6hSurCT0`zKH!f#)( zyRTn5d|t)3ESIb3eO@j^JMa3j(yA-W7V%Is42_jc3Q6?Avpl)oX7LMhYraoyQoqO? zq#%7~8jMU6MFoS31@o#y->DQ1AIF%l_t%93X?&hl(g&Q=_l8q;B>#iEVdzTC_i<25 z1k?#dy-k2$ju)2SaWwMu3r3H9*5U1zq+5%hnZV|6O9r>DRYQi`{igd+dh1qUg;rq{ ziyE@TtueXROD23%EwREVbEV{MGZN3r%ZrEH%7qf_Um)Cpr!Qt8!g`=@Bqyu$d_#$I zVJ%X;)VPxD(dv26z*yH5jRoVQpC!$A_{E)As$C)>X+H~Te?$j?#t8=cJ^Od5L1%A& z7g{*eLsIN!SSI>q(Vl|( zVm0_SgW$T1)nW1q#1k>goBO#fakuSZA5SS*-G?c2C-(>=S;Y!S(iIKz&ZDM|jqCiH zfd=EgB41&D|6FkfT=wb|yoI|Y@K2(NPFR?jH{BmL-M8tBODWU6IcrU&J-TT z`R>3M@?+MSo7dw9;C3pFgn7n7BQj%fa=37ohb(X>7uX=pKuDcAnR64D zG-lWN)DM3B{td>7LQXJz0)7J+Z5OYjSCz)Qp-8B-mW*uc0>LYf(M5&jlTlN(*vPce zH?%X4G6J1Np_)$yb%MAcW$IsKyp!?Ri4K218rg$AYzi!u;xanPGRL>sFFY1TmiIfq z4LA?v9?=lT;-KXUMtxapjPf~)I#2jrFu@_befJjF+A|eb3Y8AmH}_t_25K~ifOM?A zY#NovwBo&$tM`1)O^rDJDzs4doqBCw2-^#9C{y{_09O~W4;M%7Fr82}y<2nF#5zKU zsv@uc4sO~w-16QXjm$yWPt=?|B-O1+gqY?}vdAfCJTz38Sc^5#{N*Uv7&NHrcz# zT>NF57_&0AG`X_RZk*^3!UrTXe@52!MZGR+5m`$r8dvuFUD`%XlXW!n3|YjhETMXr41#Q!W=) zpvoFdKQFBm`g{QCt^2xG$iQ|NN`r+yJj)Zv<#66u zQ=5E&yD1V?Tke5fL^!~v<&!Hj&Yf6CaWJn=YsE7ZL2~52rCWQJpFu01^vP|H-2)t5 z`o{^`-Nzr^Ros^6+|+vjyY^v}2_;sJ?b8*a$Vx6#U>C9l7?(Uha2($E=0fzn;lN8= zlnMfb2KLig4dM_NPX;WIXm%wN;uphWk*nE|={e*~tHs5*`}-}UxT#-NMoo-ibk8## z1>sH0tfesR=cszJ*QD&f+Hr$>7vyVoE<#dua{LVo`Ylm$T$lfpnpmaIV!((AA2AZ| zZ{GS=f30r9UWNs7HVsj#Qr|67XspP(nCjbuN@5@d(UsqN(JQT9mE8(c0T`6(TL%J zDbUaI5&7fRL)9y4G6(YjUl|vZXK1bY^wj<*&D_m`of)4i<4VRiV;k@#i{G|^!GPiR z?isra>CaAJtzO{4JBvVhzL%?Pk*jmpJ!ZH1-T6v54ZqWSco*UXulUTx7&%RmyHDrQ z5N$-LbV&VgFa>X}-6(~%dBw8*crYTYPem(VVGZhhP+2l+@fjEHhVBt) z!fNaEuu1c?M_pnPy+x|%DRoqY14dK*X(s$iZqZsyER(7NB$1x65$_zmEK3s7P@)vk zco3A2QG7C3PWX-@d78sEh_b}9Y?Ga#XJr8IQ#+gtmXT3I4m_`{`Q{P>01otp+7 zj{WPC+?xEb=4dRBC4uC!*Uz6Su4U$$OkZf}m_0PY6BIZtQva(*t9m8wmOQ!NW1)tl zHxQC^%<_LY$BGRv5k)WgGZo|f@BVBIB-6Nxc!jiw)mSQwLDd$ZR&;B4VOmt*XykEq z4>Yx0M3PimL@gWdj(SjSQ5k0t<>=$V6?!SW_Zk;a$A_aEVf`R|JeV9N_3r_9tG#p0 zgYVEB+G&Pm8gOtyAiQYatuwhs^)#PP@~_aJELowNLPyz{tZHCupvv zC0l9nd?No=#Ioz4<<&d#p);)mE0%9olMWylHzz8dg21SGmSa1eNTNRrOhdt)JBcZo zat!0VHKc-=q;x%zWWX_zI0nW12?NknO-bjGC+uW{4Ebdc7tuE*11ZVsaYxO6c)my} z2kKg9MXlp@DoLihFr*^ZN(u$O1tXmmrS_4EkV3lnA){-LJ@RFB9%D>Xsy@h3Ls&%xCPM@Qm7Vc-Wl9j{BlpAPr)N^=^x zvMBQDit}k;aLE_@E-04yl7Zz)y`JIo!`F8r54D9As>VR|3aFO1$T}W)jYO*X(eiZ1 z%yDII4X-hxol1cT+$@~KH@3f#t`E6n@!jU@=&wpVl~K2LX??ROW_%=e`4I*|32t}+ z&5F?+LtyRa=IXXtjWyoM`Z=$z;CesfWv~^pOzOqGF>dMu?9H*tuerVLL%wr;>V9`O z8=r^{EHqVZeM%UYsJ1q5iL$8 z0_6rlZ9U6z(|j0m_DZZ={?oscRcyn_=fqit%MtL@yi1@V$y}_qu0maBfw|S?4Q!97 z4od}BU1{Z^1z^Me^qHx;RR^{AlNW3TjWmpX<)!Wd+Rb8JpX4E}$DSmzr&k%MMkY~G z(d$yZal3ms!0KbWzf%B(vGo9^G+fR4@}&Mt5hen}3~u&MuwHD-45!hf*9ez$Sy;WB z>SW-fom^*Av+x2zyq~`iwE|B{5=dFsO6FD(QhyR#4V*^CcX}UzvqJZ3_<@K!8a=*y zv$ha>0oq)DQxS-(duXq_ULP<|$ziisZRs zSgWvpGeNARrck#=dHgZ7qeFpDS^BF6T786fT>=2d5Fs2tRSHF0oOLal83?PV*1j! z-N;ch4Fzd`9F=$-Mh#b8p2aUh87IdvZK`Ari9eSSk-bMcyNt1Di4{f*CK_0dY)kUL zkE$$kTj0;+M)w5vGP3A)Aj5$X5)QKV}_U2q*wQ#0l#4KqYHjh4E( zeEgOYU8xzLN3oO6JTeC#LpzGXC|dtl&N$TpK%@5+D=eepBdo*3WEgSvsdq$XlFjxP zshu(}>A9BE7d*4XRi`98MjZ6vlovzCCueb)RNO_Ib3SnG0L>0@IL}OTE_A3IwaG0H z?4dL;Av~+)K>v@%?M=FPpE>b#38d`#bzNXk{|_LDbpqn#U0L>i#~}iWiCv6p$kPZE z{Agg4MDgfBSCcv(8(@rnp?Er!-<4mnx^{wVKd@z?q{Hpsl3!=X)0Omwi6Ce-RNH*k z+$~K0(6Qw#dK-k3rT(D`1FP2W(^Q z^!be7Jo|JKxCVt;*S_#}Yc&_xnIAwtH#nJxZrEtrxqd8xk*g9|EfU-V7h`3lMFTJZ zy!h5E8Y;#LgDgE-1f@K5M3tA94C}AfzUSWHhpkP)#4I8|s}gxjPK|OsN7VzpZ~^+vfpd z(G%dOIiM#D)W5-$9j`;6-b`m^gJa`3-D{)VS_?G{jhRdq82$Gy5@p+O;c3evJVG`M z@t&fnjL(v=eiGlk(?}i3h-RAJpOrU0e%3_EGtHwH+l(lTpmxS=&;Wx&wlV4|CZ%U| zhkt|c+g?v1_y_qyA41U3_7W>e1gjn0NFoQ*56_`HS&D- zHzc=@_sIXEIT&Rn1^+GgMiI8{MMPrtzJ2)TtOQ-c`p3B~A#pZmv)W`vL-M*Cpt}Vn zrR{6#PEotGRc}`6N4y9ULeIjqXDX|L}H<9VJC;5MR^pyUB{&`01u)upD5h%N3A2K-){! zyC<0NkRL9sH5kT)SdQSrDSu%YdXCa>+Z7~UC;!YB*Al|oi{(B|@U8{b1KtVns+g1C z_Tis*sA|^c9e^FFW|ksBpK17ubppoXPs#b4%x_v z6e52h)T2=6I}`ACppDkHUZY7o&i#YiroDI!Z{2w$r0Yj_N5|UsmC)eWODY2SYwz9gPehE~L@$L)7mqfa9h*I<* zc89C)MJLgv9|;`Jm9{R$U1Sm9-zVL>!oQAQKV0>`S?-U27x*WGpsSP_rz++`WyS1v zCxEVFBp{*#1ey_qH8NC11(>3&+j`*U!EN%%r!y4}m+kF5bV6T#gi8nYU%%Lv?Fi)d zYL?y|557J0zFRmFC$$xKH)qdso;w*ZYsLtk;$f$~kNT`24eWOUWnW++=*<7_?+viI zQhw^^S(##1HZP!~tiZfASn_QTgJz~QOr0{_Rd9u}A;(lVj>)LRZMy8Nah14XLD z<`Xt{|_$AtvJBi(6K6aQuA6or|NJ5SeBH@ zi!ZC63IpC4N1)}HQ^UKkfA_@v*Yu4PK6GN=;+sgWWsUD9& z6JytoH9S3I8W9UTYEx!{n?*n;^GL53q>p7eEdr8xIn^-S(YFx z$m~Q&S!j=yPrSRRY$uh~fNvIB9ouj#zBEc?!{rkXIJxnhQ zSvc8O9%&XEGDhp<*jCS+EpluXdVWsEf=58w9JPpraiarO4j3e29YEi$J^7&Td z3<<^g?nz~nzm`~ZAV&QB)P7DfvZ~rhF~v+BaMvQK5v0G)-$F5?U6g_R=0a6Zq>6HN zWWE639nuHq$DI?5RIC*5zD_6`I*etm`7&uG>^wN?RmS15t$O5EE&7$iJ7cstGxU3W z3N&XDeaU!NSQ!Ypn2iE=OyhrBj{y6tM)%B-=i$oIm8q0J=UWz&$_*?JK8RD~^nTbi zQ0RAY0&LYv@%r%nh{fu%-V$9-#l9 zALxHQK>vAz{x|-=JV5{Xfna8$|MdnZ!; zTS z?H3Gu6qA5{P}~kWr4sN4{q~-A0cJ$Nfa^^&QPV63W*kr-_Ui8123oIjd?;{1TqNOf zF&a)rKr%*Ob_uB(FsqN^q*}uHG0Gl?UQyC)#f?opL8n-v48fLsBUW5>Nq=_@Dh@|26mxm#;pE|{8sB~^Q9@ zb;*Av?wEs9f?J7XqMT4SGFUAL8CQBX#E};ONR$b}#_VO*5>EKZH3A9{90EXth8dqN z=<#n4Wu5-hnIK8OB}z$##ljS)@qaVGI+Q`hvVpr&?Mi|dgfJu$+KmtE2k{S%e?Y+T`%NK+f zoib8R-y+He0E{5E3cA!pkyuPp?LQ9|-k~kZycQ6H)N^fpJJIqPI zCbC2$#i$@qV~XuOwQ!6Hg&gFezsq}!2KG;X)bDOISS+AU6=fmn%k@9m%e`x{{Fa?Y zMVxiS#%btID2i0&yJ8qX#d3HkFH-sQ6e>fN5R37C!9kc%|;8Jx{#>BtrE z(WN9?s-W^}_N0MMLZS>!XQT$boW`)PX8W%mSQgIIf|6TX0K7oTy&{Bm7G#5p2;R_W zqIbo!%74Wqav=n2-McWFX(-Urd(SUPf!th#zE#UpXq@$384h^g({-?=PJO}D?6rsA z*Z5Cz+>nC`b*K?a4tpzB{xg9N;c$R5s?3MgyL+=}ac*V&~zRDnI%M!BefBx>~E zevw#zkj>?y?raYqI1}22QsI0jKxe*xhgRmq zk5#JQ+6l)JYkL`rkSpa13Kf+Nrgk#{+$O`RH>hf$adK*GclQ{=8|}_?TOkVOj~s9T zHx?1Vo`7sIRE5^#W7>cB;c$rXUFJ(kxKI-=Giz*Nd`p}Oh7Z9P8&QED{WmHov=)hg zeE=+Kr5P>JSWdRhD)nbRPFiWIPLe$5KAHO}QE7xgq8ED6W zau=tN2c}i$#C3ew5`A$ZPSH&Z-AKSvG*k(#d3#=eJk#dMJF^y%W~Twb3hNH&9 z2?paRGv8c$C8_TtJ*CL%-nH8n;2BS22iTLZywGYOs$IWS#eeSj=6P_~Lcf zY~xzWGE3md%aKH51XhkqOOC)~ZFFdzwD1WPh9634xJ_Fz*U3~lvH2E3BkGdo8)Rdu zAWt*FVHHnq`?=sW4z`1m(bgLdg|ArDfJ?#fe0}RRfgnBwy6uz1E(AXwQi+pc$K~eG zaGn;E{T*0AD4SPAoC1yaXKG!-Y$RU}+tHwWy-PR?XJ{$N1mo+yF^~BT2ZGCs3_(0; z%X2W?@{=)mW>y3sOr7@eSOUT_531&eYTA`;AO0@^K^Q8tw%|4*^uYZWz00N4ooS$s z%0g!`COY9=kwbPcA#Tft0Qtr1>lcls@G0+{3P}J-j*XOu1mQw7AZ>#mPr>Q;v(;uK z{q>=vg4uK$XUSB}(CZ1Y%fc(|>$_iel6AC6-`yl2NcLYM-ovEibYjFd@1FBqMltZN z9}2g=qQ%GaCs6run^(Jp`N+5(%1ZoJ;ovs$0r_%BP+~jEs<-)fy=J)%VW9=3(FM3F zS3hj9sA2jBUYyK1Y@m-6?98Tw307CkRFAb5jA}|e9nN7f%xGf{7B>P*80*FgWp@lG z7~a3zL@gkgWbeHY;_V*7Vie=*iJ=@yvA}|Of3_Zlzh{>z8YuW2C!NNHQJJyhqTG=> zDf1+HdL;~)9HSrWS70XiZXV<20mTO935_0~$P7H*a33Yc;9A~N-PA)?6ppc~@3+D? z&-$Nb5_=&8AkL(rMBx`^;mZX}oqz{amAak1!oLoQx??zF%y^Tcj*{^wclfu-KC`R5 zKQd|y{&gHJw~@+1iOW$MoDo@YlQg6@J4)=s8TdV*K0;sf{jMkx4l`4g0{4kc7%#f# zD~V7=%VBk1^x3Hbxogj(-!3Dhc5fI1t{!U4AWj%CxUv`ol!yi^s(-v43f)Rl*~_F@ zk`(L~^hU#~+Y362eoent@8EbC>yVk6n6HzX!%k2ha0y4#Ku2jJ)NaoXIK|RK^^$8{ zZechEn+xcCOo(|)d|{p(4jx#i*z+nL^u^8rI`j>mLun`bScjC5N*U@@t~<458Zw;T9+_bxC5ho4FKU(Ql{3 z#+VmJE46P~Ux2O<@rzXBaG>JV*pr%(HUN5rOyda%>KH6)e*QF z6|YPlQ9?Z9fFqWR(cma+GW`0?(VQ5x)$TRje_z@@3aA{k1K^Qh@MuIwa|OdCnBI-L}x=WM+C6 zC&$M_sDwqnw{ARhZ_U^?(vsKN0=^$B?!Uj@4L>EH(t1@x?Avt;f1>sbj3I%wi9f@p zNoo6re*-7Ub(YweVA{=OLYRThg%gt{3fKx7>Q#a7PcuS=XFudi4Y4JXotK`e1;rF! z{0gyi%!JQzEj0)|;GHnqh)NG=ArFsBP#KAgI1)1up$Yr>Fu{M&>}d zUhH(;1=yG2`i0KTU@bVM<3@ZI(UxYwiWpyUy(u+4&U?LW(m(Nwj}(X6nZQ?{1l%F>q9$hl9#S9uri@*1OTK?8#5s3#T<=m3wiQW>2MPadXE_X`mm%TjN`5D@5FA?yA zCn1R1zx^JMC#i|49_KMOf&<$wY;MsLC@{4^$DO|d!eOE8rCq7?1aZ_%iL2>ZiDHZe zYamFJml3s`LR>Plmj7-Z;K(ZxJ9wg;en-hu4faSJBua^ zCoP~-pDt(*P#$a`v?t0u*S%99{<&`?xqR+MU>rc$fJxD6`b4QhKHI=u2_{N0Y$_eE zCnr*sb^l3^PJy669GgW7jXN2c@e3KJ984!Ca z0;{Wen(M0)HXw|xDB=cbuAUe4O^ysoI~hb*NgrA|`Y(V9w`Ybt#up(0N_{wFl2G4X z=N#GxI)REez?v|$OSf@?x5Xk}B;D|2?drs-J7QxBsAwjZ2a66>PlMM-{29C)2(wuL z;u0A)_EeeTHaY>p!jfcci-?7=B|5jO;k>&9zk}ZSsvD&PPPBTfM_WlStsIh!h*|Lq8exS<=HoGk zS+b^e`SE*q+t(Fxb=P}%jYW?YraD==kFu)BmTQtfUUl&9r&>i%S>~_reb$L$Fx~G5 zE8HGy`hWn7*UAx{g45QeU$S|74%B|v1RX0@7Ap>On24j^`}u~;EoZ#QRQ!mPtv3*q zI*WzfI*7)CAOkgQoc?Uon6Di4;U~rS(AW6>eY6f}P?o}l^~qsy2XDwvTvESaKX}{- z9_XY=jpH}GN`j}8FjG1D*Dz2`z<+*c)Yal|Z0UPJAVV(Qn3xl~+(_#9^IM1Al($VK zTPU$Q-%oH=^thu-Z1QVZ(4gVPnUItr{8y^uqb{)Fzli-_H@O2#J=Kcb?mf^%xHOnN ziV@v*V$`|;2w{L(_&|i|n-WXizt-tI_nt!`{OA-ep(%cQUeajCSy)j=m3Z|Dyh(pR@woz{F&b)` zO2j=|$L>wO^u>g{Ib!-JAl`mR=w>UvCReaJ22emmtdlZ`VDWS~fncq6r^{_dkVjZ+ zanE3-Wsi3i<956G1wXzjWL^V@_01k50-+(P6S}ehZOsHkiU7U0{Sbwy5v>w}T|q@j zr|}2%d}9NTd7mp$YaSA#ynL~&9f2ZjDz!4l?Yfa`3J^3Z*S>dV{EL{#&tG^_B13dz zE$5SIFS%Riv5w^q1ooEpyr56${qr8SleC5Qp0=v)Yv@@HC;ym4$!M;{XFAV}XNa|> zxU_2d)ogA)&LMNz&=UxU#)2go0=4^>IKqtvm0ZsBW7zyMvg>jbxk6lco*9usm?6ky zCRR3{{AyHe(Dp6KG8o?tKKIR*!N)}3m|4g`5md|c&1dj4HbuUh<`XBB!i`QB_v(ZM zrFQg<`4vLl)_LESNk~HR_RhkEJ7=!KrKT?L;Z zOIMFnYnP>9uY$I6RQl-%?`p=&^BggJk0g z>c5}=fH&kaIppqy%0$ufku+?uW-uRAoQtq&E%>gl`g%Z)4Gl;rwe|3?4KTWs3IHpa$lGxOx6VcA2HzW!CF+p^5%yzRU)E zMEQ&u-=UCz+exFTGj(2=Z-zFzC^XKg5*|v395r%z)ilp(y0TUS^~{}@mqt-W=zq?$ z=thKtL)dm3Y&Gjy3F!CYPT_LDL$jGmwKap^F~!{|)zLenp#?CY-&`}sC!XMK$3PIuf`ZO(yaQhFmc zrdTt2o`Qv2ucDCfxbv9nGipMik2a?~W_d#{X~|3`8=5tatOe&1jpn*K8U)-M{(Y7| z;6}L8Z@&}i7BK;Zzywm+oz=A|=b3P;`rr5ipt}2XKNWf#kOPYEXb6}t9k-{98sVBt za!X<%MG#yQRL|APJh@_P1XS08`>G-*45(jmIo`PYRgza3o75BGyUd6NxS6Dx19lyUm zlOpfRxWiJz<-Wf&PU!LhZ zE+1mpx>s*k8@%ZcUjln@aM|TZd>M!trMp&KNbHBDw1yK~DIB1;8~U5a=ky=HUPZ-a zX$h}TdC_}aN{4A=6yW%|CD_dKs%b-%1Dx>iE3Jh~Z~Q{A3*rNSLh~p}9Qg(&j}`xL zBM!(`qZb|45g%!Q{4%LdHh(Z zLNOyGH2U=TjZxK^D$})`kFI9@G1q%C`mB!)K;&N>s2WKJS-JLvX0q*?%Zhcu>K>m1&Fm=OZzV zS?~0>>}|nKnI_ClUj4I?5PM2Z$gA9{W@QW~!Opnu)h#T6Jh)W48 zY)VBMq;VGdqvglxq+u<63Jjm%bxpQPqq{fsm$6vTIE_Kb0y{Ds9&7aq%nF@o(wKTi zDwT6a=)4IO0GE$qHO-QW!=O_6exKTqgY~6;4o5q+X#>U6$R{||Va+f`d%m%NA@|Zc z=wDKVT#DJ_&TbP{yI`CRsQ}ZNr zL><-3YHBsjz8!XmNfcN4KM{vRe6p7o7HcQcFtGOdI~aUTni8zd3;PBH{;5}oW4Z7( zW5H82EU@*I0bqKe1 zP2KsZR=B@X)2CY%BLe5dd}{%`CIK}Dn26-l#q-{qb&Un-m8W|-T70HbM^|zxIzF+M zh7H5bzXS-pgG0-rLJO3cYbghNq0$`lD~1!5zrPHp!1~aNvyp54<05HN!utLYs=WPz zAsEm4c57JXz>LD7cM>@$S@iN%I* zXIH+JjQHV-VJr6dDe$;OoqrSj^%KNWAKmb2>6~pj&q>dnl)3rP;fi5-0<7`B{U_wtmCM87 zAjZ;tN&e?5kvtGv)^DU!pkmg8Olki!kFva5&9UBmi}== zLRM>?Ejc#cGXK=>%jnnMf1D3>bhi=$RD^`0g8Oi4lD8_f<#?^L6ppwHhV(OX-aq|p1-Wg7m7-NSUSzOyZ>*qM zGzx1?{d4z|&IdTu`<|!6`3?My0{1@O#|2j=a5eEXDMrV#+wW!E!TVK49||C-yj6>% zp1G$aJ6<<>z4+ZyIXk$o9!RK5QgYV&4c%K>3=s+f#Y#t|dLmV}8f(&@c&sNoM_PGH9 zC2p$KObkE&14hnV+~iRdPgvdSl#TcEfxuPAM-!H64|#`|2W7U_XSo6dZQN4_+}hy3H^W=Cz6l=x%_>z8O-TILfUbn9+}0u?(N;Y*~D`3Xe$K<1Qw2 z07ie4ktB|higOlBhsMN-f$~yqsrg5elFv6QWNu}I*sCD5uq*rm>LNj{+%g010^r$k zzosy6GupWOST@h^ZRZuHeuA_}?}T1#M0q%rk0;jbtCfpql@ENB!KwL>p+6;uHA1c` zzE4;Fu&F`UxN&S+jD+Zb=f{WUlza2gsG;#XCGfWyHF(suh)F;zln7BA^5Et#V^; z;fk?pj1{e<>%-=98nK1IYJ`e}(P_o9fa+yR}VhM7n{{%^-M6-NLZ$!p3- z`7*JwDB$4Tf3+b8rtEx=Am*bGPzCUW04 z1@f1_HdSflBS_6hzc?!_=s!ex6M9_;3!*8Cj!~3313KC&CHaueQB%`m-+nuMKT7mT zkr(}ncE3Lmau#$DJQZyEIvzT`6MRwGsQphk7P4U`?;;32&8CY#Dd}c0-3S+bvGf(0 z=~MH2;OMGiKnQZX5o%yN$>Pl-7*d@B)(Q#_U4>gRy5bgx=+f_bXfc;>+;@d)J#=fd z_Xtd)CGMI%y(fDWX+t6!eA;yW@@CH1ik0L+V+0qfP^*?zJY+Wg;sGlZFfy4fLU1^e zWyWIJ8N3Jns??707>IB+%O%6b#;=h%af%HzM7E7u<{&lm zRZS#uA~q_`?x`G!%oTKINY;b$g2E8|f4Q1n zRy`W^UHi2{0l>5y;*(&wcS8-x#!-S?^d{9jYl$@;_RlNK4_oCP)-viBewq55yo}g+ z_>{U&Z^=Xg~XdX!}_m2HuUkPANG+KUNT3)X-OLUUDsp{gsouF+@orb4%rn0h2XUpv)4K4$M|ChjGy< zDXyFEdHCk(mjoD(v4Y31^5p}BAjoP2%RtvkW;j+M&%Z6w=xIdD5HUWYKR0MH+0dbN zk|x$SbCK-rj<)%Ffj0Xgk~XWIS!%T7igL}A<6r2ZJi9BX2(PHvj&|!;KWz!(saDjz zh5^D*FdZfK!`9ZK(IE{?ltI>*D?#kIekwf*bFL;l{lS*FoXcUa>1Q89!^5jH zR*zk$r$^|9W;P3~-eyIV@Xlxz$v)a_N)a`2Ggs74-9n5$eDi`Lmu-s>wL1@1y#><= zG2YJV={gl#MZPZ<_m@tzp%(KF!%sdPpL#J&Uhg+GK6_81sZitl0$4_Y9(JvjkLTT0 zvEJx?7Zak2B>0Qn?^isv6^U90=035->K?Jz8E z5K^xI*V&dgO1{z4Ye)2gK>`Mum^u7;idkYC1SUFedui zbw3!4K@d~i>adcW#Uu4+5SXic)Uy}r#o8oJ#N0r zW*2u2e;dcvc@r#UerACxi-DJCLmT6Rqb8I%Z}QnU(ChYB@1b6{rN$2Ys~8PY({FeT zz^-+^sL$iRvyZ_7*cqXX4>tIrgT6{#b=QV(oPl#A`pUOn7FM`wdXh-Uyc=A$pQK2;1Wq9fZmpP${$s z_{%4s8j%gTrZSMGujoGB} zOxR0(!^#eJEo7TlU!!B_*Jj^Q4_R}x#cwR=qnD{*P)_f;>{xG5@aY=k zCJk083gz0M0yJaGqyL_F(|I0r|&0sR*9YJnDpouhlWo>YWuL}O^~?Juxqi=pJ-o53mj3=i z<8)iQX_d{g`IT63;YdkL6RSx5N8fq#u7(u8y=oeZH~{P!<)Hh^oFgHf*JA@MewwAt z=eWfu_VV&q%fLnV;HhLHRvD^(3#M`RD)L0#V$`FPXc#Mrv( zxpfmJ{j>^zGD>reZ2qQZFRwr2MNCVoIN;^3k@9m2GoN0#-8b9)Bn~Qa5_T)6{XZ5l z22aFKkYt8CPjZ@nklZ_+>lJ&$g+ev@e{5;$8}a*IR6R_*umL!~##VP6)DQ%&G!rrG za#gtn9_R2F+3J%zUSd@GZk`76xC@cSJPUCt&HCrp$|%q&l}UA(LdYKi<3MI|MjMR# z?t4vOeGvh3Q!^n#_hFqOBGbIoWngi$&1&oRRA!=93v6y7H$*ZMxp{I2j9!v{y5fDB zrkyM3>{U;% zT)YkRI6S!+dzI{m8(%!D-83FAu?%o4{_blR(oCtlN)e% z^_Ht&U+{D#Iw+vSXdPMIVRuexv^CRyMrfhxxZ2+A-~nR9VC^tJ9~kYP7&)c!AUv8L zp#RBK)^u^)aWIqfu6vWNywYkS1<8GXBwzb*ycmZ3d2kf=9PdBT&smh`=1tIO&Nl%N zx)Fk$T;_Zu&3pJfi^wP4=uG=OSm<}r0&X}Qasze|SOY&U&)?zJmOKJp7oSI#nhJDp zZdpA8tCAQ-8weK8eU}Kdgg{QZ-ydRo{za&Xf&bZM)$aJa1+?#XoP0c-*sTdS?mgZ6 z5p1&W9f^J-XpP(MbNaj6lJOetK*5Q=QQLvSQb54p+X|Q}{X{XAm8d~~>r72F$Qvyp zC61@>!7KMdcTs9pLVjie4}~`*C1j5V8?_b{N5LucdGPOANfnki84T$WUzO7>XK6Bx zEc?(EmBgQWd%yobw?%B5V9XDoPx;3x!>y)E|wKKLdJ zIlxv1x(tKhC-Q!3JvC8}MVP=fYn_MXjp@tI!Rqd|z34TO_QObZ0ec0m6ndnDf13Z`q1iokd=ut;InF}MvMqR#^))@{7M{Km z@^|YHa8>Uz>NB|N_7TYzuCZ;=sx=|TCxnrQg8@oB-@g!i=g}#)K;hHmtMB^YAEMhn z%nAP4R(XsTA=I|;!1`*+ye*(2`zNEz9UFghIYe&G{F4K2F?j#Ttws9znb1Y`vf;tK z_4bbgM)O}8SRa#xps6!u4Bd);9{5<(%!xQNYYuIRUd48&ldr$&6U>)r~sg}SccSX@+Tf{Un%4X4t2+` zlM8Ne3pV+Dd);arx|`ljhI3o+kNwcSmzO?CgYb}g7h2shWFvs!(P8l$^48$Z(8LC@ zEtUQA{9zHe$vucm7f$nm;g-ue+wUQ~Gsd#Lrk@_j(Q!mCWlf@Z8X|sWMx2fV0F9%9 zazwBqyZ`N~T_qa=vQW3B-@t^OcEbXHE33vJqZ3T{^hP?xh?;FsK9$si^uTK`){l3u zkIro?k(b<)#YS?%&Sz|qzi!{4y*4eCo9*rBMd{ra1{VodohPf*d+ia@hF}4;4%uAO zF>7ckh0AYx2P}~rhsJoH4UIK{)0Mf)+<(5J;g)bI#*$q=52+pe*!HQ?Z{OVHEIc*_ ze5wRVZ#&r~X-FBNbwX{CFK#JDKHoR#81|g+s#3_y9Z=o9n$cMx-m}%&Kp}5;?djL0 z&j>1PmeIi)bLz+3Q^Ai?PIe4OOX*SEJ)?3_S--3>rtH`ftuqi zE7#8m94IOU*!#WY?mTh0Nbz;&P{S#%_ixp47HC58Hp8rW&TRh7ved&D3>M|by}M0A z&T8Er4aG=BChpnY&zzRRd7t`tTt!Jd+gx<(nx;#O@fk4BBW9^ zcEWkieZmtD=NlH`pQV`DIyOX!|4Plucw$RdTokj2q%;frtDdG7pPJ1j?xUWp`)ks_ z>#-KY4*eny-jva79?c-*-4QIEuj-e3xbrXM!mj!?y8Ye)}CEZGbfl3w% z=Z6`vV4Ta^Y-6Lv`cbZ0%)ELfU#x z{eLY>acr{482xU`mlNBG38=evQmPi8UA0wjVD=~;$!4FB$qJ_7;dIIh^q(W=9it??sy^cf zHST|&Cp$kD==4O6b((=nu;eU7x*e>i3}pXi{i>>oh2=kfeOYo6`7GCIHnBkE+KW2~ z&%Kw$d+*ht3vl=-s~q*tlB)eY&qlBE?R=W`ZU0s(I$l3lz;$!#&aj0c#=54{?_C1> z;_%&-YpaeZw|(@-d-nP72Z+Slb9&3!MQ+_$c*1n|4Gv286k%xbO0olAEsOpaT#f6? z!xQ=G|0$~&sT#jG+~@wu47glCL4!uAay0PGx5xI4d+Mrhx)3&bg)hQ4&!2ZH|6sG? zK!fW^f2f(Po4<&VL_Z1Z)P3blXLweZC=P;vCOS%Nz{f8K0h8oO=zu-<#*_1$cV{_| z?;m%U`yWncg)bZ0>g3<8K&K?T^iB1d;@)|5Bs5_rS7T|~7f)mH#>inm#nf9=N7x(| zuZLMt1i6f!Bu->TgM|l-=y8HRO`v1=k9_g} zuyG|-8&LJhstC8ys25ra{S#2bA zhZ5_U_4-I+w|0Ig2x|Q_W3}`nAU|=1h5Y5#&-aNIIzIba{onqIXcGK?j`RZvX%Q diff --git a/images/square-logos/endocode.png b/images/square-logos/endocode.png new file mode 100644 index 0000000000000000000000000000000000000000..a90189d6f99f6467accc7809ba304fdfb8a0437e GIT binary patch literal 5231 zcmai$WmMG7+sA)|bT+Gz%gwB})j>E!`#Eoze>u(j~c+E}aS@NdCF+ zbDs0RpMEj(Idjfj-mNGuhV;leg@Kse5bRSs!FjBEF9_F3_m8Az@yQrAB z0{|}RUjqfm%%%bW9BoH1`1x}Os0Y;D0qR1p3I@}=xIyh5U)uqI*K&>?L|<=@Lh54e zm%MsJK(abimmG^;S3Wv`EP<7i0h>rIf+2s2T&I^v7iMq38=TP76X(jNrkVuWLM11MgzxL5#OGYif~8-7o?FQ4%OB_IR^?vWs- zjuk)#$oj;}mjJR#C;>U?ta?BmI$$|!V~qr!a|4!a@3&`xfSk(=UlhPBnSmT7F9Dz@ zwTn;y93+9NNy8{bz?cspvsdpI2Nt;jK2;-o6`-yafKL$NHUL<70H01oCHxp;~3i^7MIU zPzwYK5pJA~0YF{?`NM0^ZoMW5Y9=N`Vp|9-xc7T7?isDEk#}2@l`gUX@Xf=2`tBKT z00H)@iLLBYUV6bX@M?(dT)lE<+Ppmnq4s0xy24;!ze*Kr6-av|X9 zGx$UxwJYm?;*EG9&bop;rZ+B$}CiID$-vQ9VIe zgkO;V zIwH?)ChKA7B!P->Y!P&rJiXNuITd3KW35G%GJ0BOA>w6hQEapjojx|kq%!p;)^_4Q z{Z!Th{Nctb&zR;(;z)Ukd?Ud9qRb?bU;(zacvRK;BK;YX8Ks#wx=ee|%Cm%29y28k zFWWjbV2UR{qv!e(x|y^2c~fwcW|QI63@680*0tzT?>oEh$SXDKEut-o$xYJp|_dhc!OsDYK6qC><^wYj-``iC0seXr)aV z&5Z6T@22gd{et1)hmi*lML^i8v#2nsAk;9b+H{Q)>e*=OA$>kk{)6$@Vgs$aL4sY!5B$9rn_+GykYHvJ;N)95f|D8g2Y` zFPth;(^8EqRVsBVl@Gc^YK}^CO1cy~Os)e7nFzkoM%wx(rpKc?_GtylSLgS7)8%{xrIHoOT& z)eCj4#juJCs}oA;-_mX$k)BI_|u;~@B`?|uFLaM@A7KC40yp~%{ zm{Yof=M`vKv|XrOW{}6;WNgkj$5{2XI;}VlXN}04*WK$kM`z*RbH3vueJRsL@6mm<@X-dR9OqKiK`F(KYnzJftBsRgnR z*|zr_t+ESkxEpvA-OBqU1#{S`F}~`N2{%rQe95l9DftoVVoPzDx)C>>@pf0$EAa-9t2Y7%N&bi{%n+jr^eL)^(|Jz2 z^2jKy#=nczN7E_rW$#E(Z!?P10j>egR;b^>%EFPkwlS~Cdb8D4SPqRed1O=#8GlaV(C6+y7e5<};&OY^|gV6E5lFvNN zRLauJyf(8oYu6r=o}S-d^SMz*JSEH)0-1QLo#_udF7E}+CP7XoV{%e*{QPJig5+-be39c1ljXIQ`^t+JM+1HTq||ItIPW*#r^zTuDH}{*O?FWV&LL@onqDBjqP?~v||az zc3j=5-D&Yl_DF6Cyc#|yV|KT)c(U^T6&#j#Hoc;Jfe{)!d6#e=M+kltmk~D_h97p2 zD<&;2{$0lHcJHPXJZU@0noD_Ca5q{*Nx|!XId`xyF`Yi0pLUv-)Z_fF_;w$DKwXO% z>>GU2J?tgO|MHjN^-(Wu1x9yLxK60O`HucQ`@O@T`g6Uh((5GZq@;UvwDhwZ8$oWw zgKyH?sOV|}fDa1*1Ox-XpZf>y0l-^60NA$#0Eu(}poAt`3@AOQX{M@zoW9rcVU};a zp&i|jjIWJRjZI!O^3ecao0y7~szWRv-)&K9CQ8)( zyYN5w{|Ef1p#L-ApEntBO7*b_ZkdJnV~DXSAHKB2|Dn9$vw@yRGJCuZSKBC_T3^E) zi)*Sxxx;<;$N@DK^+WE(du%TZFWO7esobAbC5!di7@2_*(QRmpB$V!AQ>J>Oip}_6 zjZ2xbamG+fOf&*+;Ubn?KbWU73ZT%s?i@$geI~zaK^L)a?|IR9e)pd=_)FOh{qd0R zn1)jZBHhK@>L7^1XhC)C6(DAW!;g0g?hDzp~~^}Gn@_$pKh zkn_6Lt@xyZhBc-O&5t9pzK_n7wF4`f)-*rpUE=7Rk|_=le`JH%CLV93rX$T!H6~q- z+XFs`c!Q|CoiTp=$@cd^i9TEDw&3O?g_XXGBSb^W_dNaGRJ*x5$}CwP2awW^WIrjSAR z1z_xv|DzTC?>Sdf3C4Pcx;R1vm$GVmoZ`h*rxccSd;;6eDezMFeh{WHVJD9jyc#2R zn7EbY*V)2%`;_FNjwfO*8+kRSCLO+a7R(Y)8#BrUuug%L<5bFJRlRy{gCj?2IV`@N*2Gx~&d+^g8FZ<#1RH3hCeL{%D$|OeJ3C&EMn5WCa zWxtOwEEVJtxZH-dOly8{!ozHCU@dFvO$dIfU+zKu;)q&#(r$sSG|J8LEbFMR5b(5B zSMKHVVL$2$l=3<;q-FN-I!mf?E+s!owsO2mm8Cp#RSx?|GuPHxTn?XdgJGis{T{~f zLoVr#4K7OOdS~bKGxm9QW56gKwRfX|;){hQpF4k>;rZQ=7QsMF&_W^krBf=dvHr-x zVNw688yDP&u8=u$R5v-~-N9y9{2~pVt_WfQUAc3}=Bja;VT*mkLwoF==qoW&jhRq^ z=a;SadV%m4d;$(-v{EmO+q$VqUSk=Egqn4Cr5 zshyrHdtOIW9V#^!E97UbJD?NqSMuIvBhnjNWPVQ{#f&euu5#3vK*P>1vcV(B9}DW{ z=Ln-Xg#!1*#wU;Q2)vHTY+M(J5G3|Oltbe4Z`0!)`yJGX*__suAR5dT9-Sf}fkyjc zq>73-+3xR^{VdQ}IGG$NZ7W?z5?$m4n&^iMP2Sg|mY0Um-Cj zUGZJw*JLH4%Gt}P!9q@J)1yH035My26I`Qe#2?5LMVd<33Qd*$nAZW?C62k2zEJxRx+Wn!28$C|4a$yUK z{&uLyM(o2z#;s{qeTnx~%WRPcizJd;D|o=oJdK+JdOse!HJjT|+!)Bd(n}R+$>_nHSLGQ)6 zWnF$=%cf|5Th{Ep+Re+hX#Cv_=4slg2Rb@-)zIKw)Ghg{P|B)L5#c|VLEjx*iP^2( z67+_;DV*o>B=UV9iPXGM@Fman9pO!!C5;qg4%SVmWq1d#m8RbQAJ?#heH+qVaKIqh zT-;SiZ3$C?U+M5z7{b}|Grf9SYImtrIJU?%#TFz1X>3?$JM&r)5k+$KJ34XlrzSNz z>u9DM`&W)i%;r}_>+Z0zqpMgvK~>u80L7HM**zQdkD?=^m(SOad5CtrH}qI?dnB=? zQ~BxfiogHi4TNBR^YcVGDS#Tm(&JENMu^?BSeQB(B`53Oo_`R(empTjxlQICe(nrW z-8u2!AVu>IoKZ2c-SH@d_&g;xK~`%KH%dc-=Iwt@zGap_LPrczENAj*ju{v-E~~jY zADYr&nd$X|Y$F%sH4f+x{y+8VY?+f_ao;}}DbIZsw#_|KE$ z@>9Q#l%5ZH$D)yzNTuq573W(oDwveOm_$(K;U9Eg*3zlgq+LMH*}xm9%el)W~sMWy*tN+AjcsQk#MH^EU8zVDZQ7QfketXnSZ&&zZKvg z?(omAKlE?;c#x6*A|Zbl{tp5F-D3XN0Ik4(ocXu#{BvtmVfH;fK(##2mrJfL_h58@ Ns-l)cjl5;hzW_Vd&9wjk literal 0 HcmV?d00001 diff --git a/images/square-logos/giant_swarm.png b/images/square-logos/giant_swarm.png new file mode 100644 index 0000000000000000000000000000000000000000..6434f98735d8106dd8c799ea4349d072a750c399 GIT binary patch literal 9772 zcmcIqWm6nXvt1x~@ZcIOxVr{-hs7nq-B}23!7aEhE{nTMNN@|XxVyXS<@s=b!rN6d z{b{C8cg>vc>gotpWf?RSA`}1sfF>s^ss65u-d73|!u#IcC%f>jKDdg@X(GLg50Y6p z06+ndlN8hR%sR>T^3;Sb^!T53f;e=M+bIN5#2RKJi5;B9V45eG-76^`H-O7}w)Rk; zQo{yd6wprXa|?|^EOO-PTH=*ZdVSa4 z@?FkJ4!qF+RQ$dUf|z^mVh;8Bp`nbJvlfUg%EcPcL?^1%s; zr#zw9O(lu>x_J+RYTDcjt>3Eo!4BszUk64ZmVC|cKnYHV-QX&DSM8q1D){>@RS+?p z!JkB%8fXAfPrP>eM5|#!&MOWWi%dg8PvHlo5Fy!;@#jt~JTb zU@ISb5VM|GWwaBC9%j_-()xW*@~7RJ>j9TLn#{x&>*8*odf;Mz#ViogyI7)re(kxR zoK4cdI#3RHgY|Z!nD=jgB4C_Gmexx#R%z-{%|#3l*TqO}botV#X1&@&k2C>Noxy0I zb1hv~@XOS;lZlT{Rg)zq;p zY4{-d(#MKJ*t5?rvQx+qgaB+wX}f8af&<%<#I3g3sPOApwp_ zx(~!pe8Efz5&J5(5lT+t&4P4Q58S?RDfR4&zLvS6!ly{efYP>OuW}O+ScyyV|jr`Jb(l)JrzB8^L0}ovMg4Fs7XqF)^)# z{-&l$##s;FLLxZ4>%>ut%AARR>!Rh z*`+$`mvPMs8L{V5zgb48u+HhS<`R&pow7YkeRhwNZ`~=Ss3XaX6Wq9?a#cG#jL`Xk zgREf~Pq=8HlQ^>XlwwEwW1FE=)e`?NN7l!$)k;UKiC!(tJo+VQ0=?WF0kS+0mD?`I z7yhNlyj_fA({zKdiSh_9MkjJMOCH3UfZxE)MZ{gmHv2x@wT&gjew~p9ct5^$ze)E!N-gaCDVEvI+w(mV#c8)@I*$lV&a5H%g#0zI&8!^tkRv}o zrxmhxHoWi6Y(S06>x$~60b3H)zSb*Io1)K3=v)YSQk&4VEwBl_d|TO4HPx-Ani80^ z6ug&?>EpE%rLH7i)RAzKU=N&l3!ViX;F10X$2wO3aY8r^Ei@0EVtEMZiR7I$(I}VE z(i&y=eSPS&P1xWTZSec-k%06Ss$^gtLRfnX+da}1D(e2jLLkCWoW9&|Vn|wN{--Ni zjpN#yQrcgK{$HEU#u<6?wUEiHo;HxMyXrDC~@on2{HHjQ;p{QN1tWk&;^d{jJmh6 zbi10}hL9_AzJ!XXx2aQ02Q)6#r)yHKPr?iIUYF!P*y(#aRAq!GkCncMDl^^Ktq8g$ zw_YYYa2YDik+j8tr^qS62G?MYKu{LN1U!k6IQ5Szt(%i`act<6X6DQEYtN(dkmi@m zPN4kQ7hl`pn#5)d9iQ2|yotL>>wCtj z3de+$RBl>T1Ks^avgS3~P#0cG*hix~0oRaag)I-uQq=h}IMPl;{}ua>va@fq zL~o)_8h-g=C1i8_jZY`LFzKF3eB!&eu-ICIW1C)ar~Aq^o0S{V9gaE z?7+*Z_>W(~utaVp36AlZ-r0A&skA)?=24~Xd8=1h<2^z}kRWpybBUWugPEMS$~m_u znv7f2=BuXbogP%?EKf4W z-?6s_S)1}vAFev(tbB&7(jdGE+xSqMlE`_H0~*nP>zFWQt=xWnwst22Ch=%o7&wwWOEe2H8qt^=86g#mP`9KGsx0*ZxnJlxM%N2-tA%ar@(woW4{ zoh$#ajn7qGj8&w~K3Uq+7-vsdZMn!?X)aG{f!$9AeM{)fp-NeCBe)*|Db|1G#R&bf zLI#d$`GPc*e5DQ7EX$NQnwFyLHY1Hsc`7RG?wV(mT4iGVYuf0kEq!Rqi?0;`T7oiA zacJ)$CtY3^ZvER(@tT3$i@qZdSUR5I;$cW=PvAe)W?9ZOK8e@=Xftf&Y9{q`$ zNhlxG(qfdU8q(pjo&z)Ihn7(Il>be$+#E0P-y3iZ820xppNIQj&9fmKR~x6 zM{f6TVl4;9*7KwPj}PPPok&%TtHnqj6DzWU&xPnwWMT8U%r3T~0ju>Rkaya}o{yEH z4kIWkwOb6J`my?S^H`~k#y^U7F8;3wz$aC*S-wBS5Yo&@Wc}5wdp1{Z{`tVe0qS^P zVeaf^IaS2HGh4o|+>@^I@51`)_>s{#eakJ-0MP(3+L56@HO%?3agg%f;)cNxq+abk zQtq~x^BQn5fw&fMV(?Xenlg%ka6`n1B3{*K?X=Q0`FD(Q{mm<(-TlK*(nL#ek2a0- z>vXJ8<4RZI+drJ1)eDeuu!C(!zfCr`T8TsZFoGpr;0%05+_Y(bTclAVNtmth#{hEF zt>f4uLE^1U5&4sCT1IAvJCS#;AlsCs?J;^$lAmT3hec_|) z`tvj1pN9`BYESZ?7iU6vW;8mTM4+rpCg-Xw=-~Yc{ZmjOLm7E7y>NM-R+H8 z!vlLwj7`(kqATNkg2OB=&cX!kG{ze$!ZukIC)o=ce7V<|S&X@!KSvGc077u88zaw+ zZNGj|{C!HAyKh^wwMaG2d}P*{&@8;boXKi5gl!N@9n(lLgg!~fGcu*kclwQ3?i_Ub zM_rcV^~pwy1ZK2$S|s5gHVD7yANm|&E#wK)`vR-a<*M5WdL8OhOw7$1 z3CEdPO-IJM^DeGtLuKK>h9N7$KSGywZO407lJ0He?$!6T&-FZqYc8ha9&RDGB_UZb zaD?aQoRxSGySC#GiqKiS%s7z$W;QrQd1#0ur89U<%v}Nu1!E*q2I*HHxYt1patG6w z&W%4q{Esh`77Tabyko^G{qvq{=c1?DL5i`!w!(*1mxXoq!(y;0w~lH1oM8E|Z!ijX znqO4g!2*$DfqkN1Y#X5rz^>|oG_U`(psFu6?ee#G9H2%M_;c^^BUguZlB!4ZcgEzn zQH!mj#Qaa#J@QfHwND7A3%ebK)BaX7Bu$0Xr9Rjc^pjU5O64N`nn20Fm0-A1Mk-K) z&rBnt17X|UZ-4{Ei4~5bXnnTj%t{L(DCva9(|N+=K}KXtbmqe}3v(uqr<;1ZLJ*5} zm%TfYuhLDcZaq(Oic!-5PALg#=S8mlhdT*C(D;Uqpiu%b`x7jqze7FPUURz{n(zk#Q5#6-%^jJwz_r$9Rr%ewG==*+f=X& zahANtC40F?YWtrH7{R7_7$UbRHX+W5GVMwLM#wYw~qD|ZB$ z{BEY}Cm#`d&GR0m-z-<@C(BHI)@EJA_mZ&a0gGnvjlq^|vip4LJAy2djE0b;em+F^ zhJQ@%5>&{hj>m+e&b5*qMWfA&PU@Q>oZ>Vlv!jQVGluhv;0681?2E| z_GbxdLjjets%Z_wuvh|Z-#mH>bAFpiLTRTenEkioOqFlc46BeE6m~WJ1$rOzuTJXhY`1h0dq=(E0cRg z{RfSPgv$u;-qEfnW#mf8L=&Hr!kM&NM?o781@?44?Ulw@lJThZgdb|UBbQ>$AQiLA zE|B}lOyiTjv8i1xCpP?@PAM}wr?&c4&|$HB$uV4PEK^g_!OfDY7c^@QAinC)DYGG; z=O5_P!F^#+NQG$;+e?cTVa0K4a%)*%%u`5Kk3Or6&B6QJ57InX2s>E?8L6Q$)U%0O z5(F>Ti=4$bth__-B6wOHgvTka`#DAIuk^CyNSodK*lbi%Wu@C5#SU9Orz(6_TeyW* zVfZKIs02&rPU|`!y|EZwr=_ChI_Xbc^rHVsg^L>1-B3PvOs2A^NejW=zgSUL@289) z`+r692MFdKVQU7B*~PgAaPevr%>9jBr-h3yD#(v-=xin%<^I}K%`z+*Z0=}DFV+G( zpf|#1wNoo&Z1(;S?Cy=4A_AFX-8s@uS1J2^1`lJW z6Rh4OxxbysRJT5!W^vTc79(7H0OsU7fzSl>*}6lX1=q#V8IP$Q#tGgm4>K6bM1fDT zjgxg|-78>bz1G*irs%yJ++vI5C|?3gm5FQ4q1|v5q6hcHf%;p|eCTC}r2qZ=b2Q@E zH_N|stFgQD5Z}H-NV8ij%sRSkBM=@n5M#U|7zvBn%JJta#K@8kZSxk zFL1OxF%#^Sq)&Y`wHK-s$y9aj@p>L{VmnDj+Nu*N--i3Mnx3MnyNpE~3_R@Hyb!5N zUYyjNnMn5EcvN;Pt!&mD2j=O$-MCmMR#i3J8&SF?1e}}`4(r4SiD?HraFvdfg=)ig z5Z*g0k#6#pUQ6q#cxxY2|GO>5^bLIjMRIX%3|eGWMgjU@dvrz$)SStsIQkOX4VPdH z65n>u7hoi1?4HY)(lXxNEnZA7QwX5?g|EUJ<+RjueVoqWxO(ddTtpCb-%5)m+*#qiBAjm_S3V ztd?d=L}$`#$YaJh8c=$UU>qMHReXi!DHw+v3gEMDrLlM22tVxr zN=pAd?1>TOG)TQ!G+4i^ry9S~rc=;tDgQuB2p0Gi9UM#2a@_~DZ>^pXHdomyPyzvx zxFw8q=Wo4EvPVHKc2T35dy7@vy1~G&hQvX$uv?2#hl^Lx#x94Zc6WUgWVZbsDX_^l zvs#oH+RuSzTuYt)oe{X;)Yl`8;2X)hB{BiCzj}O52MTA@_VC;rgTe8t=8VO){ke1>OG{+5A14pu$dCOH+inX3+l?A? z)L55j19O9RprQ9v{9I1vsvpxjr$MP$xu)TJ>#7UsjlAKA+p5y)#h#h;au}5~qnDuV zI+V3>SN_@(T2ZqlK zdv~5MNseH);MYRh;WK~28m$i~ct>JK;p5&%#vdLXCGkU(J7HMQ|1edJ{CB_6oYhvx z)?o1LuZNJu+yv?U7^vah?QCYeJNy4e`!ADr=`{)Ev>Dd1##o+xk)UgUPY{sEjE44p z7*Z7XS7~uKy4y)0f7)t+<;${uho^FW@C}Gwngq1*`@3stDLRmvxB+3((P?KzOChWS zVS34E;V2;<9nDRe)LV?zy`r5hE6hY+O@KKsuj3@1)9-Q`hS&UXG1XFPb(GC0o@^sK zUwRs(d4KjxZHtwl{JSU}rU0RMA;0aIcFcS{=WBW6&!I$Ze7uee(#W=vga&nVA!=47 zQrqchdNJg+YG5sWU7_AK2vY7F7&cPj2lS{Yv~_ z*{hN4Tn$T*{xBD@S@tG@aanU#4rF#;RONo;a8L=l>T#ef1R=@93HVLDR`TeT$QLmr zm!=S=f&^%7@F`}^uw3+`L!r9|y=uQnuJ9vtttuc8dG{>i1g?K4g4&NV@>ASJy>F84 zU$rtdcWk1C&K`-FmkJeeHh2?^3`U3)uMUv=LgD`iN^Ooyd>|!Rs04CHOdv&5?A6fg zLTGtiyj${Bc)0e>k0MbRq)B+KMMs)mncoAn@sx6R#x2KK6uK)Z{B?U{lg9Pb@j-Yy zL=IF!$S6+_uK~|y^Zf9FHa>=x+0BECKaRX+DE!=Fe657FXM>9_#YMVfi<=^F(}0s@lnxcG zfhWYC5A*XQHT{SHmF06G99DT-i{FV`WaK8g7>30m5?B(fvmUwJR#ln{YjQKyO1`U@ z_RP+;rVSUqKijF?Vij&1#2c=c2R(+2M5TUOUfQ%YJLT`}J9`tAtxM!w3f4&1idwT98hH?h^*i#198Q;Wa9~y`WKE>kc}Y2-ze{aEG&Eimdi zo|vA#$N#}PVP?quw2O~FP7iR~4ne)eO{O125!>qu9<_)ZbsS5iiyK1>DY+~Y>OF=^C z1d$tSQPVoAIHyDs3sE<*-Y>cFr#vPck51^+;*!hoXzQQ zDFS zuk)`e)EG#NMTHOkG=sPa7pMHri-xv%D@#b>Izn`mf5iJ_dR(%eKer~fmB8=VOG-EG ziUE|Bm69_99i@=SCre~8#d(``orDL)rVLd3X$AW{;$RNwVK=J6&1q3x)NFmt&@Ur6 zRYNMA%0ge2iQJz<2ViI*2CUe+y>;F5+c8;A8>~R)oGUAzp@Af4Z!$o}6epsfGx@Nh z+;;VDf``5!UYPW&DiVf^ddxDlY>?nSAmO#v6-c>3T(_BFEZN`0JwO9#jwQ*(@X>e;nqIVbvK-=tCa&fH%;RWtdx zZS7Bb(X(8jxSM~;Ng z(?Y~p5^kQz^s_lQtjAQ8zEs&W`%-;TduA<%WYIRu2^2P?0)V-_{-MHs1u=HhC{)xb zEvh!uF-@J@luerYR9se;c^t0W{J#9nUVe!v)V+^}x_l*Llb4bCUfhQ+Igk`u7kn(N z4J%Z#A;NA4>$k+>VmL0|Ka+P9L?F*nz6w6w%!5SN`6Q;04kt1#QC5~LeXbvFM)Jo- z_S^>A@A>BE@0T)2|IOOWpfV$8K~5}NdwQzyCg$j1o11jp~Yh* z%X0Xqu|*9uax=WD-|Tbc`05yep1CZ%HKdFzWiCEhX^STeEUJm23Z=Pp>MuRFvWiuY z7L3-R)=Vo$CHI??+H{b#u|erP$!F&%Lx`%kQPHipI$avOkethhm=hak;ToMVvxml5 zOAy;DryYOQI0{bKHw>nnCF92_-VS$i+azcZC8TAA?BYBi z=EelKp~apyi>Z3znm{+hMA0ed`uZ#iKlG?$SJ?e}dFRGwo3Z>b(OfUolb#Fwn`k>F zz(*H%=Z)n)wv~Az{}~tG;354^x6?3y{IM3p;^{F0b$JrDc4%ry1@|{7S$^w8dgc)Y z5!7-RP^isM0vMToJo$T|7YGaj^gA^sNi7?hvf>ePyF#pRDH?0cuN|61pRLi(5 zr{7&vv6&5EInW&v!U}#AKK^kWJ7Vb}9MPf%S*ZA`!?^+g06+KNEWoy0|0K?SuT!+O zaJuN`EFwUmvI{AZY7=5x%8LP~la^O&&HX(&Dl zImsmkGEvbMM=#X9Lz1w)4R>e#jQq>4%hU!}+>r#rsWlcIA0cw4Of#eT@eXa(@cVeF znrWzwd_F2-GTP)c8_^b79v{(DzU;9+fd)=LU3Pzm>Y|?#f1SVj+;` z27Grm$cL5KoFExjX?umA{rz*>oXougZOM|a;nno|D>!aF1-;n%Q8ILfuEsov*Qg0* zC``8`Wx+0)TordAu-O5 z%;d>w?BYv?sr!7IygYX?nZZ)s_u>XZlM_Ai`P`HTB1O{VOhuCgbnX>bTR{Yds1X@f zM01^7cT2lqHHWYDzFo=hLg*P;;O?NAjgX!OMrNd_EpB4c7afU!n?(f^=G(~S2|JE1QK=%Tg--glbq4=cLO?a|at8sqV zJIjYv;=pACDvNL9_o%5+b>`;jj_Q&c_#0u1rE8SlJMWc8Y_BbaA2L_A;*1X^0rXt1 z!gfSM-sJ}`JdRR9X`YvHs~;-1|54KLOM)nkAIiag)&)Kb?xe8T&;B7YH4#}mP(@6a z5wU4RxNO*8Mckh&IPOc)(11;&VwTl z8r38x@;&~R)GJ*=f}?F~+pzc7`gOilkGs+q5!-zUb}9mnG-wtIY2zTQMgiforeDn= zmFsNu2DMsc%9NJ&tB{xCLSFk)@rW%w+u(Y5E?m<*BqaE{HhWuXa^dZ%JJ?-55DpE~ zxEgUmjmOuw21Ub!S0gXjxl!PohbS+K`2d7+q*Oe;X$3I?Y8m+&nX11nvFk3aY*|9p6kZ0o_362a!4 z(0X{raAeBD^+)-69>jp9VWvkVh9iRApF1LHJ^_4NG+R(!?hN}c6=ZOJyPOClhF`#k z(uP9mu84_lSo|SE(LY7>=w|e!Q{f0Q11j4LtPjMk%(WnnQg+W{8>=$b4=q?u_K|wa z24Gny_-9wqO>%nyQdq-!Pyo7@LMg?w$sr*!54kpypvM? z8kVW-riF({sU?*VhMPvuOofE0^p&a%f~Pr%B`XU{GG6)>COJ7Q5l@p9H6D2dVV~+- za%B0}_{rCuz;dUh-p9k~kG5s8qx##T)>(uBMAU2rpeAn^YONGG)^_CB$mq^KyF@rL ztvdh{vDJpk>zxJ)@Dw5)c?Xa!}%jS<;A5cKug$r)!930QFXETJv)CD@1j z^N6^^c`&wb!2m#M8s2}qy?ytaL2sOy;Y)5uH)s3z3;vVZ%4+Z9V7AU(0sz?Y4w?U8 zNu%%rr&f6pEkQ&tgkR)TfBf~M=_|tG`1R3YnTUR222C* zf!#*{;HJlI@DC#*OqhM-&aCg}rTDviDIFjTq?qmj0GP?pvg=H?ijN@z05YZF%*~P{ zcY}ni1MsASaO;C8Zx(z}QnVvOQkYW6HesZm7EJXKQj9Uv&1B3L9DgLpcm_1>V=~;) z*@tvH(M8=cUMvxd2N`?Akzggq5pgW3R=+}7#T(EhBNF6AKT!=Tz+xxRQciru)udKV z<5l3&ir1mkRUo?&^@J9THIeI16CMX}N4&-K%5!Cgw`hv~L2Z-ns^KS&lKX}-XXDM4 zpMqanFz5M`STvEZ>}ame9%D5{OnQuS;Fu5rIO5F7J~}M*Lx2@2!+Pombvo1 zJxovBz4h!5Us&vL*#RUzSYIhhOL9pC0~G_c6@?l~Qd%CYRV02Sm?+I52I`C&<)8H3 zSkEJbR@_{%1`4b+OV}wm9GHP$rAPQ_vE!t<8M;!T6@OIdEMPClEx2mYoUqmw@F)<| zWK68uIJF=MX0lSUj7RSm?GNt*_eu7tZj6zOtRy@tp0sxvwI&^vhz>9hcn;ugbGapT z%k#B1YRYu5cvFm|8p_(Wnlxi;aCpAt3r6AYTR_hD?$w4O@gnD6zDQleNBwfeDAIic0Xd7L$o)lk*o1l zMTPDg*}pqgl9ctx`2KROzQ`@^Hw!CPajV|+?>l-S4d!^)d3SlKzL!TefNw>mg)|Fs#xb^btto9fEFoQKnPr)?6P4CAFH-(jE=dj}(P7kNk9N;x)Md1s9-f|x zURm{@>P5bmYF8Cgm9UDi%Gfk``rGtU9&dpt*E07}!BN3-!B&Tbk)IK3drf<%tzyf* zQI+9Ed#`J>k)UCvfv%Bht6W1-1KmN7&HS0d3-RFJciAT# zb_TPUvn#;8cAP1u5r-n}f<<{d0=q7+$lI?8amo=>>^WlW1tg;z<9+)|-uE7+I{^## zMFp(|L+3SXO&5F@4i~ZOC19QsreZ^&*WB>9^tR@~yf{ip7N^rzeTcfWdbnNN5eOwi0W6exX_yILkr|wGO@Hv5>GNEq9Nj7uz|{c{4BMB>hUOd zCT+$6Jz68=cxev7^3XD2I0Tg;IyPEAMm*X5s{!#mR}Qc5{JON)mduMxw#;UM9hbUb z6Ppq*v)FfbN0C+jZk}%OQFf3a_^&~cfugfAsUSOZBcH2t|JK|2ZR~E*F3MgYK`zI* zsk?vK%+OtQHGVBSHmgv!gvwVnfy^sOOy(N=M7kRsU%F&u{~@v=J8F)3zWJ{!(k9I0 zYBvhZssZU|X}!Guw7-FstOs3D47I7fDrhPj1wzI6`972`+@eB#T#@N%sUA|_sZ=xC z?f%&vf_$dxZKGR0e!C`gaFAyqOgOcs*25XHA-79a9qSk|E;A#v*xBlXeoe*q9v^w9^XI)$4JPDTv2K{=hG=pr9P2cNDlpPQ;1a2Rg2S-!*+k%-~N#W&)v z5#->7t}w3q9gxJjP5#PT&-__iH_~M@3t9l39B#4|8>44`&pFINcSVD&{zTi-`+0u< zAiKtqGtIcx6WK1^RsKt)v)8tIVrE)cT&F_kPUl63?_y!2&2st4b^d}Ru=j!I?C+X! z@8i4JME{yDl&(b8e186tAoYnm;P-AyveUoU=;F*82T&S>P$GSoc2iyX?mF-dT|SOgIwm6`nD<>GZy7Sv$bd@6^w45;AxB=lxIQBV$p0 ztf#Oizp=~2sp}%3uPqStctTRlKeKVy=YNg=x%9VKP;{g3Ho)!i_v8H*{zix=!~4us z55$+@qM=v4*ZfZ6ToNL^5&Ku%_+x$Lay`^>z_;{veqH_%K00#tBkeu~LmDn6FJ&qQ zE#|aXKul0@SKRacTNvv5L6o74pZ3o`?zQKtUo(g@GCpBpa&O

>52LK36D#}Re z_^qB5`S#lC`ak}fH441czUb-$lf_6<#=~RDXh8i4ha-s_vH#XDH(wJ+Y%QsJr}{(I znrAe7B!*rtkq#auLWz}EK3vgh_t)ctT3{zm! zyYcL8eKlK#3<4Rcf_o9uajEnqy3lcIOC&mz@c%DVoR$6`;eYys{$JsL)&4*8{IA;o zGtd7z^#5MyNi~A)eX4u@OD2hJ6sKKctOq?ky#c?FQl$xf&PuZb4o#$VCiQGz0UsA9 z=lHq_nW3DoF9v!nl%&<^lis9aM4oksWjHU*@Hi^cjR?&(SMtF68#E9FrJjgf(my$3 zr?=B?QEmw}X_p2j4wa5PnWt5{l{L^EsJiT{+Z|KM83jo#BbS_RCf0y|TX@CUhbQ7p z$W7ydSye(TV2fgl-xbz)YGdejxP4kl661z{c^Pj-J)$Od7+o3BTS~5%Mc$vBxRpq! zWrX#Fqi0|^snweZJKU#*aau~5&?>~F1v_77JJ0f;8OyL)q8hz+%tWSeB7xaphy2XlX7T4{r4hxU z*7d}hS*)b_kK#^#5n?1)Xr6xKI2RryoK3NVz%as9H+Y zQGhuF(io5QZ3id$C6O8M3sfvb%@=FV(M;yG8o8N0x%|rj4PD<#uanYC9`H4QNwJ^9 z1BdhV7dVjxO7Zm5!|7TsCI(#pqacqQ6E$kwdLa>!KZ0b7LP{@Oar<6d8$*PN&JdfS z+GqJJP9bIZ+7AIMFw_6_-IT~ZBhR!BlRg?oJ7-K6rfeAu59@@w1mzm1qXItr;c-SG z%q@^<*VXSd>WjGr>-vdCWhU;f;Z!e4oY?_I6xk4ZI$96EgmP!82lqF+hrHg+*M|G# zRG|bW*gBNMc_2|0yUB&Gizx~rGZO)ZV{jdquI24Lkrf~{XNG%)e2xdixl;wv0i?7cz{ot1U2@g+0l4;p~RxzJ&@52m4}bzLNrb#{f_+LSAf(<=BMsQ$}gPm zYfv%+Z3FWIngcX9AAKc5_qZ*(ZB(D}Zq4VSQ$#3o}CIbp4ja`gS*^>4gRTP4P*d!&R|kW!)j}65Ls-$&S0-g&(lU zc|~)rf{hpa-e33{EG>=`G@bWqBVjzihqms$_e@#U2J%O9uOrjTL?OM;0qbf3X=(V-c$ztT0b_AL@6LG(IvXfCZJ@f)*`);U}+^RVJ~$uBI1`{9;D*%?vGSd zP;zu~J>N~MDXHw`X4I)%VfvTBw{9bcuHc1H@KSAUK}<}H+obeUIPj6kg0e#7tHCVn zjQa|fM8g5Q&{CpxxIW_MWpijT*6?78*U4Dgoh4IYnqrd|Nd5fcUpMTL zS%$LRWi{v+PIz_uckQ=DP7<|@8mlQ^0p~1lF62*3kT};5!zxnbd$M)0->QfD%w#=9 zG{}M_ZWXVIfSUzs0b%qFnz&p{GDZKe7~^AuA|f0D`q$!%V&od%5X%t8 zsOH8xBVm&T8n9nQ9^JVUrbusQCb^lClH1hH8??JQLa^eua2$5n`c(29N_I&h1OE^2 z+l%O2;or#1ueJX$j+m~{hsE-$mvE0_NM)-VrUi`$YYR7EsLIyAH2<>0)BZraEi z;fg9F-+uP=$E4Ijb3le6*FnKM?VH8qEtJF#O7>OW zx=sQ|@MH9&F)Ue=Tz|EmA(A4HM9l$Z=D?#61ixe(H$=%#`(!T>N3|Ih977e~1{C0i zYJM<1e}8(++lVLWFF3!5iJ0lPU({qfy`fIUN6&X))TDr-FaJHwyAui1kIt#D6)Iba zmHhtK52Tw;oE=fAr*=~3SqA6+o6)f@ftgeO+eVZtaP(qC)QxKSCx>9+eF{IRctbBP1$iMect^fr05y%Z=GK>q){+eR&O!hfu|cFBy%Eo;5ld z#?pveXjFw;YuY`7hwBYWa6gj#k4A};Mmg##Z$gDyjWqlM`sRZ|x-tvH1Yn2rJQ8n% zk_RZPFE$+Vi``}{2_2TAM22N2coyIc5eW!I#`216K`H!~o8EKz^WkjP#^z2dy|jF@ zGdJU#A#$Xhtg)$BEY6){uly;49w-ZgXeauh3%~8p3OZkU1N|xS&h_U`!LM$_igFnEA`fM9Kh^9OOQ>(G)LhxYv9GF=ltn@UoKIqd*BBZiD}r7a@Z_X{ zpVg(tU!Td5L->N{w=pBx;(EAI$^w=Qx;u-i=7V1ouElm zP!4DQ14c@CWG|sE@{5b;Yy>0wk$Kjzw+o>9r4c!Rt>P?n6H`k~rpEHUQ1;I7#pc{* zCQMjK9GJ@mnTzx}7#Lpiq^6=dRkMr*?_1kvkM)@JtdE*RCNM$>keo1KjFHqd1kb@= z;G>{r+jY58lG%Rb%Xhj1!^1aWSWyIU;&@0x?iTmiftm@+iSp)p^#V^(v9Up^UfoXN z@U~puMsBRTPj;m`Gu;FUs+vTLQQN65)p1-Yf3ASIZ%lqnmh-$|N}6I2@?DWg)N($= znAIL8%GWYE#zK!;YlqU+B2gglFn_#nY4-+)q$b~~L}%e1Jy>SNpYV~Kb8&-O{SUQC z)zIwIjND`gNY<^R=DhlDli`;HqNTgH{S%4@?z`x#~?^O5z^OOKjTCj#Gw! zzjSRTDhw(*dKo|s+QEx_DCZWlI5YMtLS$bb0Vy#yuk|zj+wuLTAegB2)+&YAi-B zB*#6bX<2iB4eX}y;@Jlg=%~~3$Ge)u92ySn4iqaqfH0!;4ufwX5JQli9T6e1Toy7t z7Ojm>?d-X4e$|%fh)zJ1R}#;xdV}w>S*`11(3<|DTsQJlA@0nm-XX<$4p~b!B6=Hn z6e_Xa({^@0d&bix&(#Lc(GZUYFOA+n2Q`r8x2NJW?y%TFp|CecL5Y9j6s))x&WBOJ zO^>ZJ9|XT_0t9sHR*=~oCMVCP)q*SZQq$t-luRt6Oj7Wv0SXf@a($K0HFR(z;ojB+ z`Kalfb#I}s6N4FLHJz@*Bxw{Vpk}_TIHC55d49Msvsfj!u9hr|ivI42eKkZB%tk4( zWxQTQJNi6t9w&cvzcdF%s_`4RH8d(=Y?_`9CD zzL|b^ZJ_l{R;UTd53boCL)ARHG>^NWvl(u}jqG2|tkxl?c++R=NY`R0FTC(w&@(;R zSL}_#+}6_N6a4MTUL*Ou^-%qP#{JcxNhY2L&qR_SMCeAqN$?r;B2EVBPoGR4wn^S% z;97QbOtcjgeuY=3#+&jq>e?WaEgMrh$_28Ob^ec;0Z4F`2_xyniB8>#tD=MLOnhHx z32n!|)GvN9!Rz0RO6Le^--NHiVNjQm8mB5TgznsxNf?-tOClhxsHaAr6hHVMCCjiX ze(|)X{#0V=B?_lzs5>$7S{!*YC(~gvXKvk^-MLZeZoStqBkX;>6;UF0S?!qUu&`se znrJG^Iim~y(BbW4e)7B{eF$(^F zs58?y^R#8lU6+h5W_qVzYzxJgntiQ&h%wm;Qnff+DU3#iZB1ygj<`h(*H|L8Ft{%Og$UwExi^A z!ivkhn9j9^eL{hXUeUf!y3kb-ziI9Ek?^?*VPA7^$Ff;|75vl#23i~2swx)(4fgup zMSC}g=}s{Y6WXx1)JZBid`{c4JC!*DA3XWvq()ZScrVSE{l3Rhz#;M;4G4m}X1iBP zPWu7_D|NbwpR~6nKMST7r4T96NP6MMvT_dug%s!}?IwplE9caN zJ#RzUetzx3%)Zcb^UvL{lmG%pP@>RSSH|KxpV;8tKCJQB=vMI&1f!@+n#PObai+*^ zw<$dK!;QL}R*Kxj*Y}Y^PV)^drO!OP4sVDEq{4d9FCQz<#h~(#J~Bq3mmT_N8zy5B z=&Re~_~}Hza`mO`HnkY}t%3hBj>o2>@M1kll=U!AR48Gi2UT<$B&SnAYlOc(QMF^MxL?WLGzf?Bgtm^j^WOGhIXm8mHLUn}Is~LwlbuIJmh)|H|}| zYo@^qD%cT{dN(R)cRcVDTdMiwEmcjA_o{m;f}wX}c}K;gMVmDw8+z$Ld;K3mv5E#k>_lK)Do3GodLUt8mh#)WjYpb{P}4lWIxJkFX5*U7JYlny8}%+Z1m$hGbgYAv%FTuAW*zlF2@^*CGnV zgC>um;v~j#-%sI?{f3epZz4nQ)0p$3Q`op)j3n=eC>!)t37m;qI(xDOd_car1lY#V zu&!;xUmh`eTH=EuKwEhevXN) zZFBr}seT7$(W_7=-d}5it4noZn)wRh@XUDTDK$@&;pT&5`m6;L+JiKM7U}1@7NNCN zlTyL6da%u<_YB(kAvZiVMG@iiv5X=|)5wI5gZouG<`5mcelDbKfKZk{B|2-Rq5=El zw*{_juPdSezF$kBFa(Sa-R+Af-~qCs3KHWt+Z$?^IWkSh?lkYkSR4q#Jd|nLhWudj z7>)BE%+mzBjWtV<4O^DUWFA%=P!|ZRB~&SwQ*EOW&u|ibaY5--I#=?f$v)o5%3E~A z%AlV|L3`xOYv3Okx`}|&>ih{!LHzpH#r*x%(dB9`Ee+?=o{yuCkBxi#7s1FYcW#1J zcX#)y)w>Uz=xkxI;_-wuOr$t$un6pX-L}4qxI?WDwGUSeiZ)RD^K8%EBlL1$i(1sz zS9rBOp1-wh{FQ33ybnP=szTM`7f&Zyt8jhu4!u!|P4YLHa)_3i%R;*sixMuGZgn$HxYSSCYQhuLvT>KV8k1$3e+Jkt9A=59eq`izs zrUdzJxRYVIhs*6M&SAlSX(yUA!7#Hb+rdzW--eQ8-1G@VH+ZEG86|c&{$A!j@yrxM zKga>qe2>onC5TiiKFi%-rJ^F-$x(aEMsLTTMZJ0lw`I;{QCylXT+n2PBN$qOFC@{y z#C#Y_KdG&q(lQ%laOuq2P9bQ( z|5FJ_?z|GH65xPNfwVwCtD29ei1qu-;A*D7#n$=sk;OD?ajNpBpa|mDDqS}m997M@ zckVLQ^~m%1Dm_m5t|Qp=fTe$tz|XKsA;k_TCuEStYKy{D z0I~9vX?jm>eWJopq8;`=Bw%71@(<8}J_?+Z9C z{?^&0tpV5?ErQVm^B&C0WX^c!F^I6%Gxht)b9_Jq9+)>&mu9*He$N_=CIpZ8YXf=c>D@h~NM(7_qr~)iqSzP?Ocn|vykor{)CU~W zKZx!WEkFoaOCQ&*hUDzPMZ?YKoc6BF-Xck7=Xrf&t-J?Ozd6hfzL(%GD z0Cgc?yK-%j4`@0>3LNe~!}SuU@0zQAQ2k z@-ZS&kT(O;j37S#KV{;Me>7b~K-w>PUZ303rj9F9h9)^u=)DOw{P|I(=KIKYF`0|B z?*FDR##A_#b9_I)q4@EQe&%+yhL^vY z>3nfV&r3q;H!-s>Cas1f1kD}%AmZoE2QQ!UtGcY;b;aLadTQQf(JF=r<6^~C#tnkm zoEHfu1-9B{7{*V2k%_Def1a$o)^P?BxogaOtRPZZm>C?(jKU2H@DALjhCs7w1j+vu zd}JK{7z*;P1vg^&KSd3jg?W2d>WKOIxp)3&c^>ACQv_se`b-c6LZGIJ7)L*?o4g4mYcpnA&1ul)wor)0$p6#qhhmQH zKMR^H%_@S4=ur$2OT5D=1g?#GKXHJ80xQGe`ua+hENHK8iC%10EZFP>@zzq=!R2{X zTVKnc#!ZCghcELXwQOVee5!V&mhP~Gpb+6t+uilxy{PPQJh7YtF|&Mj(@I+NJPHf9 ze?5IYt}%s2vRIdlbbsio&?%*=6no`Bc2*Zkg`etRQCp(_4$4GM!CYlL8K#e*v#hc| z$6P@e!)tv6q z;YU5TT;W)ZXEX(I?2Eeb&w+md_JC?-LxL|WbB_+FkDiMgFu=c_tTFf;cYN6}XVnQ#RDcVD;mR2`b z>zb%@wsl>OD*O;jl~XO^B903zjaLG&e8l!j{+x7#Vu?JxA#skTVXrRLE;F_ zx33u4%JrugAg>S!gOrVd%L2%W|GxT+#T?wZY|jd6vR5p>OP9s>rq|u|p0yu;jTiI; z^4(C6;TVIk&SncCOrVFL$fKzz2bRxkrvY`aKj;t?N2A_l5;1@Q{SqE0KjjxQY&M=A zWj36gA<($=*YxR&bS_`3N20?pFQ?N_k%4nLJl=}}vvQIJ*3XCvH^cqVpiqZNdk<-v zYuKYFccv=PXq8AP;BVVKb#ev-WpP9t&RI0!3vZ9ip;T^ar-Yj?^zf3I&x1gP032i>=l zRNs}16YEx@ErCq%P2R(o$80~x!GbQiQFU57OsTqHn~t38JnBj$40Uli@$MD^`2 z5AmbiBx&xrS8|HAm==>JdiV>BwYZqUs5Doxy`B(E>z0-N+ObQmhIOfNn(KH`kA;Qv zZXo?Az5HvOaV%ccD~F~({u9UCGTzW4@T4;Bmcg=JEN;)5j!bxN?Nv>2ndvwP$PV+L zwHv&G!azY`{*KD(j>k8-Rgqt2=3id}0!oNI)rS3|ynG0oi(kLnciFcK(VP$r$T}3i z+ZD&wF)Q|(FpU1oXwly#3l1*CAJd`x_qgTz2pB^qCj%$*Lag`iw?$Qk3oFc@3hQjU z`EpI&Y?hGsmM0E?u&|*#Wz%4s(xLp2fTCLH|;GKx2Dnut3ob@KRH<%pcg@1)ix8>_Fe>L}j!B zVNAZ(dGS%MlzN@qbc0SeTAM$sS?GWBv<4j`G^2NRW3!vGYFtHE@BoAR)K~+vn0^ z_P*%lp;}cD=7b*C6}lh4 zLSU-(o{pl3Q9y{16t3Ru@P( zCKC1b?-_La(x~(WKaHa=4sWeSaUOpJ4uL*ht7V+#$eGKwoeXXb$LRfETs})^3~xN@ zzGmrG#r(Dlp@}1BcozjtKTN)q3W-W&RU~1`LJ-d8X}@EfGcFmeCyP)5Nwl=RCI7kZ zpMZA;+7AbxVse&rq*|Th?>=oUO}AM>0PStPZN1?SnrYwyW)Pj?5=~M*q-G?{vM(lp zi-o_H$D^T_|EQ37y6Q&6$vcbXRx%$71ZzyTirP+x@+>;D-fL|8WfzY&CDC?aEjFRZfbveQ^x?3&K#c(31lPJtbr`(2lLWTs&yDOK}NlrwC|hXjO@ zOSWCQf24YK<~e9hYGyqsKF@AOzR)H-CsXwHr~1U7U8Fo+pxFHQ6XI|G@Pt)GBQOJx zUGpSK%@R#t(MI9zih}mHB^b5d>P_c41B23@RoPC8bsvyi_7QkSZDDIM#YEF{7mO&} zc{Ien{w4kR^aPU4(QNY(UpOy?&kfAitC(TSa@LJBj<6RR zBUmhK0^8ihNJ6)Een?*YmTX1Bw;#PqSif$Qn-OvMp785P$Q^nz%)e2eE)^Q~~0Jp9oxqWN<}GxF6_-jYbL{(QMUHxvE% zhl}tgDOW4X{fdj;z6OD?IDOlzq9P-)cSv~}(3*OtH0>x+XpiX70hbRTJ=4jER>=En zK*fY!jZ~AIu>4DEk07(2_j(SIqVh{pTmNfIbS6V~K(oqImw%9lz^qx8nelsWc3!hH z`ZxQMxX$mv4?eesvV%oz+4%xT?LB`WU4r!0)#JCljIKT6K3fyWNP_fFJ<+J@YSLF4 zZiR;Nd2W9_+j@M0Ey^amPe*$=&CG9Pm-yC{>_q1d5|sOTT_PxdbXOvj(M~qL-~zJ+Qcrbu{71J^ zc06g+#9x}m$0&PEE>Klu^w`!kSYUj1z!u-=EXLG5*>(=z)mR+5FlK)4^^)17iJLZV z=M)zA{ft#p3!016`)aS5Vc1=#=lVL^^d6_a@@E~jKX4th{N;h zr;^MeDMFyNa+tC7jIo_|yVBiHT79Z4Xd=$0Z*50HmtDXbjq3#1pLEOgw9+%~NkZK6 z_+y>U%loa2X9s33%VMeLX4)(AlCpgZu6+z^)Vvk`YV4Y7=@Fz21k+0d@gdfgQ9M)E zX&gqc{#%Zix|sIhzTD>s|P`>=#W!i7w-rsDB?a}tvF1R4ula*f=6sqVyw&9z>0a4g6uUL{q6Gr*=sNKye zoi)vH+NW=Z{cI!mfIr>6!D+^qfW_hn`}H6QRC)jT0CXc)s7MU=g{f})h8*hRE+)*< za|Hd_aeq4&Hp{x#DvlshFB|_0rvGQba+Yz)a|IoaarH{YR*V-xq$%v1qk8ujeuxYC z^~civ(SV)q9n9+Rs2u|-d=ONPMPdeoMWMm|0@qM@j#v}dgbZq2(6>8pqlZMRpUBVu z)a)^ad-v}ETmrTQNil419Uh>I%#XLZ?4V++a%wbDo6ZcWP$^AM0Ac;>wM*hBtg9gx zlYZHo_@KaGbGdO(H^b4wvrAy5i#%}vGbjZ%b&)3L8S-EKQyQGZ`sz6`9Rz(4Uf>Tt zFu`cctkbtx!6bmmJ`!a|+Ja)el4qQ3^-D1*=e95$Vj1nWP>iK=xkg5{(RiQ&Cooul zKw#`1YOy!J1;vOZ#26jsv~lSMlR0&8-kk(_sFotLaXsm~FqG^0bMN4QlHn3W!qbWa zU~njH>*NAfRaJGgGC&15ZiKpyz80?es3hjm4JaN8D!L0;9#xrOBv#ohs$+M*?Nl)PdAq&ynIj7?}oyx?~3?q{b?ojYfeO?rZ_9}xjn^`34C+v8C z2ichsAy%J*3a5-pSlKKrf72b2pmo3Gf4j3D0HY(!*(hIFsyp(mS&BKwa%|3%vn0qcGpnm3w2q<+`Yz8k;(GbWwXv>?GnP6I#2kNqZs4 zKJ_E2_cq|RRaXwJYIlwXjxGEslvAAI1N424Kkbi5F5Y^{hrK(bUKq5re11JoXu`MK z$nhm-pN|vn^~62PJ)bpbHodaK07;FMvrUknDl>^81#Y%=bPIR(SgnixZf=+t238Mq zdmF^Wc-=9(7@Kq)F5qx`Ys8?VfMQ+pSAsT8jW2c^8+k#_ zqda!;Z!W$Zwv%jT-D(KB1TBI{13#A~)ZI zw{>3w`CSS^^5K1v_}Jp5qkm&=wygE+Th?|4_7(=ZpYE{jkJiz>+gt}sv(Hveu|>6* zC;j2AYid^hVfe-2ZDLRZI7N|u0LhvWPG^)mCTvz(n?YTeTe`kJ9|=-#kZUvAm`H|D zL}8k^`C>SdW#Z=bNLnG6&z8oTdLueVB?cWKfNXO_eB|@}Tg!hoq?|s)2jd=7?{ zHKOJY*-0XCS#Ipnoxn{YnlY-&Mt=8lsoQxpwA=N&WzKOx%TXj)`$6=S9%*E z&KEM1gC8OA0k01-EjkN@dH27?0{kxo#)fV#OmqiD3|4?tIoPeo{F!@&*3F?7hEY+} zvTkuK|K#ANQND8r3P@rgtz@wi-dJ+UF9?r)uesDM9a2bxYpB<4dXtHZjX;YPfFRcU zjtWIqZCArHz<7}J{M?)LuG6n&FmmWKl(uQ~_tQ!WMESP5F>}^llr;CEM#}d3H19Dl zkvVaa$D&ke(_+F*##Xzqy!hJ8I@Po}@uBd-|FutnNJ>7Yz`rLxTvZnP0|Kg4DJ2LgdPSTFcNf!#gg z_{MkYh~P6G3<|6O(K#Mq{xm(%t1d(HC-I2ubB(i_f9JOYoKtQC3R>Z@P3W<$>Df)@cRvIwvKHua#Uhy&=D%&^PHNY0y`-0AeAS zoIFwCCJd1VQ7$ZX()kv|3KOAjH998^IcA=FtmaWamjL{ z2J#=S)5qQ41X$K>C*5X?_!(4l4S%V~NB@xv>*?6iZ5;!V!h2|}lop~Wr3vFz$96e9 z@_d2l<0Zf_%~}<;e+i$@%{NT%*F%UlviZnM=RBGLkjY>b_xiN;FXPeazKTl>v(0mn zFlQ+yn&bYPO};kB$s;qA={OU*tY>#B%A||I-$M@+-tyf_RqFn-=DL!kIt}gx1v&mx zm0WKxFKN`P;}Q~@%uh=@mkV{<(o7FhX5Ku!Y2Gdi$IEatu=+vn!sX1rZkA-)+vRw6 zRu{o#6ZdkjlBK&V@vp@`6{8|*sRZA+-IlSHPSRmcqe-9b^v*_KRInqei$XlOmRKc?}KC% zX9tyOj6__qq5_6I!jj^|cjrs}|NbowLV3|w*ovXbEUfvl?)D8!@6QMg=Al8j4ZVHJ zI(Er!Lq?E)9IB!0bkRp3xV-KU)0e^Bc(gX^f>^2{F(E8wXn!Qr!G~8lpn5Z z_8stce!_o8Ayihq;gxixZ>?)XMOcn~^?2;;2b+TiLHBAsM$f~j2D%&_f||N_X`JialSLdn9m3Wd)i3D*omD8CxRb}$8NOKf?k(UR(3v| zhg$@mq6yTxuAhoTt>1XI%wiLadu9Z|S_DI?p5#ClhRsjS(BMOHD@OEX+^8LLebC1e`qDreclKF2V(t_qiOW>(G~MM zi5_38G%g*2WWia_gUIEoHV)!(D0P?c^ciN?v5LltM0zHyhGM4{<+N7SsGWzhO!c6+ zLO^O>U)6n`wmdvSGTMSI3-tkV{}8hf7xE`x3&#RZ&WPuta|w^#bWFLd6|rddM*EV5 zRs#kQpTb!RL6Eh4_5CK<2sy9{^T?w2Z({kEuv#|&itK5=2nOp4G)?5NjiH>Vss{-A zgZX;?cG{V+v`$YL-2jV8Kc9|AMc9UcU)i0|naf*w3f_T1+bo4+CUR9yse9n%L2wGx zm61oXe_<|R-QeV3gD3xdF)}bjj+y{7hLgRb$?MfS2mTlUo7sbA`C2Ex1u`6KCW%yc z*_^fMNaBV_V+>wTw-;?ps)|_bF#hqy9>W|tTPJPx^4eQ%Pmw?l*0P`*K4*t=%Ciph zQLA#0Gc!o*^PEDT-F~4SvCNk86s2+I6>(Ol*q;Zzeh_ssylNQx;OV*RE zD<~*9vA<8dkPe)WOu{mfpF2M?sJ#tBWNb6WwDTV&dtAr)?`!L3%KFKw^1~cSdpIAi|ubS^I}v-g*f=JO-_RUz|Vy zN#F-BIl|jLwh_fnMH}NB`BZWtA}c4t%Y_?&O5I~7U{$kDE4}%@LV?qzMp1}hhaX>n zt4R(?y^RcuPG02iA+Cj=f@}XF6^uiIk4|kFm#PK5-fr5^7$DdFEPP#<`wd@oZ|gEM zhN!DLf%qAwoXJpXB%mG)#NSeF|19?WrUz_c_D(dX!9-^&n|e!)DBjJd4N^_rTyX(=R>A)& zX$9B8o60N=(_q(!8CQcJ%J$Kqb&aVmmHoF~<;0IRr?k6M%put5fM2HW98_~)ZdSE_ zV(}UpyUqQTJGEhYEE5DkYv-UsQ2zIho>%d1=D*_Lo8H;~Z%00QoS*vf{o6E#%vnik zES>9uezLre{#SGS>EXl5y$cpKsGT)(3Usb5e)D(kxtz5kt8BNfQ`YjVxp?V)9N#Ae zr;i`%XP$^nOIt2~KrPka&59ep`-4@I&TN_Y{*kReu)!O<`R1G`U_b1|X1UzSR-e}K z?VG!#db6c@>J10AcW0a=iq8dl3h`W@(DJ#jbLGi{H&q?~H14ZgHT6``*XcRW>w)`P z-ik>-*xPQBnpgL4)6Jhh--l~Yecmdt=lI@r6PMl>nia6xx>cKR&->zYwu@eDd33|d zuk`puNNK12 z_uAZ&=enzD(V0(tA_bp6E97!beN`Sa$tJHX>(`;qm7GTdj~raQ*yGpCBmNCRvzE<% z<(-!#9{F6{`gm0c zlC-&a+*EwGoV;Ch>$~E*=_ai+a?NJj$6B3aJ{Oqx^`7D8d=D!bkER(+syZk4l-*wc zE%L!`A<-jhhhOE|Cv4HoTh65n6T^5~#Q#IHeT<5 k;M5AU=%Cz(|7;AGf6a`~S@v)la0euVr>mdKI;Vst0KdS;Z2$lO literal 0 HcmV?d00001 From 0bb20de3a46b9d358d82fa9129e5797c4aafc834 Mon Sep 17 00:00:00 2001 From: chrismarino Date: Tue, 15 Nov 2016 21:39:54 -0800 Subject: [PATCH 31/45] Fix broken link by removing invalid characters.... (#1605) * Fix broken link by removing invalid characters.... The link to this page is broken because of invalid characters introduced with PR #1549. * Update networking.md --- docs/admin/networking.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/admin/networking.md b/docs/admin/networking.md index 2acbf062d4..0cfd9d7989 100644 --- a/docs/admin/networking.md +++ b/docs/admin/networking.md @@ -83,7 +83,7 @@ talk to other VMs in your project. This is the same basic model. Until now this document has talked about containers. In reality, Kubernetes applies IP addresses at the `Pod` scope - containers within a `Pod` share their network namespaces - including their IP address. This means that containers -within a `Pod` can all reach each other’s ports on `localhost`. This does imply +within a `Pod` can all reach each other's ports on `localhost`. This does imply that containers within a `Pod` must coordinate port usage, but this is no different than processes in a VM. We call this the "IP-per-pod" model. This is implemented in Docker as a "pod container" which holds the network namespace @@ -163,7 +163,7 @@ Lars Kellogg-Stedman. [Weave Net](https://www.weave.works/products/weave-net/) is a resilient and simple to use network for Kubernetes and its hosted applications. Weave Net runs as a [CNI plug-in](https://www.weave.works/docs/net/latest/cni-plugin/) -or stand-alone. In either version, it doesn’t require any configuration or extra code +or stand-alone. In either version, it doesn't require any configuration or extra code to run, and in both cases, the network provides one IP address per pod - as is standard for Kubernetes. From 625774cc1b43aae3570572960fdff2baeb7f4e9a Mon Sep 17 00:00:00 2001 From: Eric Tune Date: Fri, 14 Oct 2016 10:38:46 -0700 Subject: [PATCH 32/45] Added init container documentation. Based loosely on the design proposal. --- docs/getting-started-guides/rkt/notes.md | 2 +- docs/user-guide/petset/bootstrapping/index.md | 2 +- .../petset/bootstrapping/petset_peers.yaml | 2 +- docs/user-guide/pods/init-container.md | 169 ++++++++++++++++++ docs/user-guide/production-pods.md | 2 + 5 files changed, 174 insertions(+), 3 deletions(-) create mode 100644 docs/user-guide/pods/init-container.md diff --git a/docs/getting-started-guides/rkt/notes.md b/docs/getting-started-guides/rkt/notes.md index 28a622d2ab..096beab3c5 100644 --- a/docs/getting-started-guides/rkt/notes.md +++ b/docs/getting-started-guides/rkt/notes.md @@ -59,7 +59,7 @@ Under rktnetes, `kubectl get logs` currently cannot get logs from applications t ## Init containers -The alpha [init container](https://github.com/kubernetes/kubernetes/blob/master/docs/proposals/container-init.md) feature is currently not supported. +The beta [init container](/docs/user-guide/pods/init-containers.md) feature is currently not supported. ## Container restart back-off diff --git a/docs/user-guide/petset/bootstrapping/index.md b/docs/user-guide/petset/bootstrapping/index.md index 03ba721edc..9dc4f7e899 100644 --- a/docs/user-guide/petset/bootstrapping/index.md +++ b/docs/user-guide/petset/bootstrapping/index.md @@ -88,7 +88,7 @@ vm-1 # printf "GET / HTTP/1.0\r\n\r\n" | netcat vm-0.ub 80 It's worth exploring what just happened. Init containers run sequentially *before* the application container. In this example we used the init container to copy shared libraries from the rootfs, while preserving user installed packages across container restart. ```yaml -pod.alpha.kubernetes.io/init-containers: '[ +pod.beta.kubernetes.io/init-containers: '[ { "name": "rootfs", "image": "ubuntu:15.10", diff --git a/docs/user-guide/petset/bootstrapping/petset_peers.yaml b/docs/user-guide/petset/bootstrapping/petset_peers.yaml index f8393b5c2c..4f992ead71 100644 --- a/docs/user-guide/petset/bootstrapping/petset_peers.yaml +++ b/docs/user-guide/petset/bootstrapping/petset_peers.yaml @@ -29,7 +29,7 @@ spec: app: nginx annotations: pod.alpha.kubernetes.io/initialized: "true" - pod.alpha.kubernetes.io/init-containers: '[ + pod.beta.kubernetes.io/init-containers: '[ { "name": "peerfinder", "image": "gcr.io/google_containers/peer-finder:0.1", diff --git a/docs/user-guide/pods/init-container.md b/docs/user-guide/pods/init-container.md new file mode 100644 index 0000000000..75b6efcac3 --- /dev/null +++ b/docs/user-guide/pods/init-container.md @@ -0,0 +1,169 @@ +--- +assignees: +- erictune + +--- + +* TOC +{:toc} + +In addition to having one or more main containers (or **app containers**), a +pod can also have one or more **init containers** which run before the app +containers. Init containers allow you to reduce and reorganize setup scripts +and "glue code". + +## Overview + +An init container is exactly like a regular container, except that it always +runs to completion and each init container must complete successfully before +the next one is started. If the init container fails, Kubernetes will restart +the pod until the init container succeeds. If a pod is marked as `RestartNever`, +the pod will fail if the init container fails. + +You specify a container as an init container by adding an annotation +The annotation key is `pod.beta.kubernetes.io/init-containers`. The annotation +value is a JSON array of [objects of type `v1.Container` +](http://kubernetes.io/docs/api-reference/v1/definitions/#_v1_container) + +Once the feature exits beta, the init containers will be specified on the Pod +Spec alongside the app `containers` array. +The status of the init containers is returned as another annotation - +`pod.beta.kubernetes.io/init-container-statuses` -- as an array of the +container statuses (similar to the `status.containerStatuses` field). + +Init containers support all of the same features as normal containers, +including resource limits, volumes, and security settings. The resource +requests and limits for an init container are [handled slightly differently]( +#resources). Init containers do not support readiness probes since they will +run to completion before the pod can be ready. +An init container has all of the fields of an app container. + +If you specify multiple init containers for a pod, those containers run one at +a time in sequential order. Each must succeed before the next can run. Once all +init containers have run to completion, Kubernetes initializes the pod and runs +the application containers as usual. + +## What are Init Containers Good For? + +Because init containers have separate images from application containers, they +have some advantages for start-up related code. These include: + +* they can contain utilities that are not desirable to include in the app container + image for security reasons, +* they can contain utilities or custom code for setup that is not present in an app + image. (No need to make an image `FROM` another image just to use a tool like + `sed`, `awk`, `python`, `dig`, etc during setup). +* the application image builder and the deployer roles can work independently without + the need to jointly build a single app image. + +Because init containers have different filesystem view (Linux namespaces) from +app containers, they can be given access to Secrets that the app containers are +not able to access. + +Since init containers run to completion before any app containers start, and +since app containers run in parallel, they provide an easier way to block or +delay the startup of application containers until some precondition is met. + +Because init containers run in sequence and there can be multiple init containers, +they can be composed easily. + +Here are some ideas for how to use init containers: +- Wait for a service to be created with a shell command like: + `for i in {1..100}; do sleep 1; if dig myservice; then exit 0; fi; exit 1` +- Register this pod with a remote server with a command like: + `curl -X POST http://$MANAGEMENT_SERVICE_HOST:$MANAGEMENT_SERVICE_PORT/register -d 'instance=$(POD_NAME)&ip=$(POD_IP)'` + using `POD_NAME` and `POD_IP` from the downward API. +- Wait for some time before starting the app container with a command like `sleep 60`. +- Clone a git repository into a volume +- Place values like a POD_IP into a configuration file, and run a template tool (e.g. jinja) + to generate a configuration file to be consumed by the main app contianer. +``` + +Complete usage examples can be found in the [PetSets +guide](docs/user-guide/petset/bootstrapping/index.md) and the [Production Pods +guide](/docs/user-guide/production-pods.md#handling-initialization). + + +## Detailed Behavior + +Each pod may have 0..N init containers defined along with the existing +1..M app containers. + +On startup of the pod, after the network and volumes are initialized, the init +containers are started in order. Each container must exit successfully before +the next is invoked. If a container fails to start (due to the runtime) or +exits with failure, it is retried according to the pod RestartPolicy, except +when the pod restart policy is RestartPolicyAlways, in which case just the init +containers use RestartPolicyOnFailure. + +A pod cannot be ready until all init containers have succeeded. The ports on an +init container are not aggregated under a service. A pod that is being +initialized is in the `Pending` phase but should has a condition `Initializing` +set to `true`. + +If the pod is [restarted](#pod-restart-reasons) all init containers must +execute again. + +Changes to the init container spec are limited to the container image field. +Altering a init container image field is equivalent to restarting the pod. + +Because init containers can be restarted, retried, or reexecuted, init container +code should be idempotent. In particular, code that writes to files on EmptyDirs +should be prepared for the possibility that an output file already exists. + +An init container has all of the fields of an app container. The following +fields are prohibited from being used on init containers by validation: + +* `readinessProbe` - init containers must exit for pod startup to continue, + are not included in rotation, and so cannot define readiness distinct from + completion. + +Init container authors may use `activeDeadlineSeconds` on the pod and +`livenessProbe` on the container to prevent init containers from failing +forever. The active deadline includes init containers. + +The name of each app and init container in a pod must be unique - it is a +validation error for any container to share a name. + +### Resources + +Given the ordering and execution for init containers, the following rules +for resource usage apply: + +* The highest of any particular resource request or limit defined on all init + containers is the **effective init request/limit** +* The pod's **effective request/limit** for a resource is the higher of: + * sum of all app containers request/limit for a resource + * effective init request/limit for a resource +* Scheduling is done based on effective requests/limits, which means + init containers can reserve resources for initialization that are not used + during the life of the pod. +* QoS tier of the pod's **effective QoS tier** is the QoS tier for init containers + and app containers alike. + +Quota and limits are applied based on the effective pod request and +limit. + +Pod level cGroups are based on the effective pod request and limit, the +same as the scheduler. + + +## Pod Restart Reasons + +A Pod may "restart", causing reexecution of init containers, for the following +reasons: + +* An init container image is changed by a user updating the Pod Spec. + * App container image changes only restart the app container. +* The pod infrastructure container is restarted + * This is uncommon and would have to be done by someone with root access to nodes. +* All containers in a pod are terminated, requiring a restart (RestartPolicyAlways) AND the record of init container completion has been lost due to garbage collection. + +## Support and compatibilty + +A cluster with Kubelet and Apiserver version 1.4.0 or greater supports init +containers with the beta annotations. Support varies for other combinations of +Kubelet and Apiserver version; see the [release notes +](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG.md) for details. + + diff --git a/docs/user-guide/production-pods.md b/docs/user-guide/production-pods.md index 440ac4619a..1c3ac18142 100644 --- a/docs/user-guide/production-pods.md +++ b/docs/user-guide/production-pods.md @@ -204,6 +204,8 @@ The status of the init containers is returned as another annotation - `pod.beta. Init containers support all of the same features as normal containers, including resource limits, volumes, and security settings. The resource requests and limits for an init container are handled slightly different than normal containers since init containers are run one at a time instead of all at once - any limits or quotas will be applied based on the largest init container resource quantity, rather than as the sum of quantities. Init containers do not support readiness probes since they will run to completion before the pod can be ready. +[Complete Init Container Documentation](/docs/user-guide/pods/init-containers.md) + ## Lifecycle hooks and termination notice From d41078c48226729c85cdd17280a5b3b09b396b83 Mon Sep 17 00:00:00 2001 From: Eric Tune Date: Thu, 7 Jul 2016 10:53:13 -0700 Subject: [PATCH 33/45] Delete references to v1beta3, which was deleted --- docs/api.md | 43 ------------------------------------------- 1 file changed, 43 deletions(-) diff --git a/docs/api.md b/docs/api.md index aa9aea1d7f..9ea627721e 100644 --- a/docs/api.md +++ b/docs/api.md @@ -95,46 +95,3 @@ DaemonSets, Deployments, HorizontalPodAutoscalers, Ingress, Jobs and ReplicaSets Other extensions resources can be enabled by setting runtime-config on apiserver. runtime-config accepts comma separated values. For ex: to disable deployments and jobs, set `--runtime-config=extensions/v1beta1/deployments=false,extensions/v1beta1/jobs=false` - -## v1beta1, v1beta2, and v1beta3 are deprecated; please move to v1 ASAP - -As of June 4, 2015, the Kubernetes v1 API has been enabled by default. The v1beta1 and v1beta2 APIs were deleted on June 1, 2015. v1beta3 is planned to be deleted on July 6, 2015. - -### v1 conversion tips (from v1beta3) - -We're working to convert all documentation and examples to v1. Use `kubectl create --validate` in order to validate your json or yaml against our Swagger spec. - -Changes to services are the most significant difference between v1beta3 and v1. - -* The `service.spec.portalIP` property is renamed to `service.spec.clusterIP`. -* The `service.spec.createExternalLoadBalancer` property is removed. Specify `service.spec.type: "LoadBalancer"` to create an external load balancer instead. -* The `service.spec.publicIPs` property is deprecated and now called `service.spec.deprecatedPublicIPs`. This property will be removed entirely when v1beta3 is removed. The vast majority of users of this field were using it to expose services on ports on the node. Those users should specify `service.spec.type: "NodePort"` instead. Read [External Services](/docs/user-guide/services/#external-services) for more info. If this is not sufficient for your use case, please file an issue or contact @thockin. - -Some other difference between v1beta3 and v1: - -* The `pod.spec.containers[*].privileged` and `pod.spec.containers[*].capabilities` properties are now nested under the `pod.spec.containers[*].securityContext` property. See [Security Contexts](/docs/user-guide/security-context). -* The `pod.spec.host` property is renamed to `pod.spec.nodeName`. -* The `endpoints.subsets[*].addresses.IP` property is renamed to `endpoints.subsets[*].addresses.ip`. -* The `pod.status.containerStatuses[*].state.termination` and `pod.status.containerStatuses[*].lastState.termination` properties are renamed to `pod.status.containerStatuses[*].state.terminated` and `pod.status.containerStatuses[*].lastState.terminated` respectively. -* The `pod.status.Condition` property is renamed to `pod.status.conditions`. -* The `status.details.id` property is renamed to `status.details.name`. - -### v1beta3 conversion tips (from v1beta1/2) - -Some important differences between v1beta1/2 and v1beta3: - -* The resource `id` is now called `name`. -* `name`, `labels`, `annotations`, and other metadata are now nested in a map called `metadata` -* `desiredState` is now called `spec`, and `currentState` is now called `status` -* `/minions` has been moved to `/nodes`, and the resource has kind `Node` -* The namespace is required (for all namespaced resources) and has moved from a URL parameter to the path: `/api/v1beta3/namespaces/{namespace}/{resource_collection}/{resource_name}`. If you were not using a namespace before, use `default` here. -* The names of all resource collections are now lower cased - instead of `replicationControllers`, use `replicationcontrollers`. -* To watch for changes to a resource, open an HTTP or Websocket connection to the collection query and provide the `?watch=true` query parameter along with the desired `resourceVersion` parameter to watch from. -* The `labels` query parameter has been renamed to `labelSelector`. -* The `fields` query parameter has been renamed to `fieldSelector`. -* The container `entrypoint` has been renamed to `command`, and `command` has been renamed to `args`. -* Container, volume, and node resources are expressed as nested maps (e.g., `resources{cpu:1}`) rather than as individual fields, and resource values support [scaling suffixes](/docs/user-guide/compute-resources/#specifying-resource-quantities) rather than fixed scales (e.g., milli-cores). -* Restart policy is represented simply as a string (e.g., `"Always"`) rather than as a nested map (`always{}`). -* Pull policies changed from `PullAlways`, `PullNever`, and `PullIfNotPresent` to `Always`, `Never`, and `IfNotPresent`. -* The volume `source` is inlined into `volume` rather than nested. -* Host volumes have been changed from `hostDir` to `hostPath` to better reflect that they can be files or directories. \ No newline at end of file From 8563dd273b3240ee7a0a1b2228a1812ba993fb31 Mon Sep 17 00:00:00 2001 From: Tom von Schwerdtner Date: Thu, 17 Nov 2016 14:20:29 -0500 Subject: [PATCH 34/45] Address formatting Currently the 'sample environment' code block all shows on one line at http://kubernetes.io/docs/getting-started-guides/vsphere/, hoping this will fix it. --- docs/getting-started-guides/vsphere.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/getting-started-guides/vsphere.md b/docs/getting-started-guides/vsphere.md index b3679c56e8..a1e59b0cd0 100644 --- a/docs/getting-started-guides/vsphere.md +++ b/docs/getting-started-guides/vsphere.md @@ -65,6 +65,7 @@ export GOVC_DATACENTER='ha-datacenter' # The datacenter to be used by vSphere cl ``` Sample environment + ```shell export GOVC_URL='10.161.236.217' export GOVC_USERNAME='administrator' @@ -79,6 +80,7 @@ export GOVC_DATACENTER='Datacenter' ``` Import this VMDK into your vSphere datastore: + ```shell govc import.vmdk kube.vmdk ./kube/ ``` From 053cdb3a1ee6527cf5f9baf8bc1e6863b105f8d5 Mon Sep 17 00:00:00 2001 From: Jeff Mendoza Date: Tue, 1 Nov 2016 13:29:42 -0700 Subject: [PATCH 35/45] Add stateful application tutorial. --- _data/tutorials.yml | 4 + docs/tutorials/index.md | 4 + .../stateful-application/gce-volume.yaml | 12 + .../mysql-deployment.yaml | 51 ++++ .../run-stateful-application.md | 220 ++++++++++++++++++ 5 files changed, 291 insertions(+) create mode 100644 docs/tutorials/stateful-application/gce-volume.yaml create mode 100644 docs/tutorials/stateful-application/mysql-deployment.yaml create mode 100644 docs/tutorials/stateful-application/run-stateful-application.md diff --git a/_data/tutorials.yml b/_data/tutorials.yml index ae9401c918..61555427d1 100644 --- a/_data/tutorials.yml +++ b/_data/tutorials.yml @@ -51,3 +51,7 @@ toc: path: /docs/tutorials/stateless-application/expose-external-ip-address-service/ - title: Exposing an External IP Address to Access an Application in a Cluster path: /docs/tutorials/stateless-application/expose-external-ip-address/ +- title: Stateful Applications + section: + - title: Running a Single-Instance Stateful Application + path: /docs/tutorials/stateful-application/run-stateful-application/ diff --git a/docs/tutorials/index.md b/docs/tutorials/index.md index 14530ca25e..60aab6a8fb 100644 --- a/docs/tutorials/index.md +++ b/docs/tutorials/index.md @@ -15,6 +15,10 @@ The Tutorials section of the Kubernetes documentation is a work in progress. * [Exposing an External IP Address to Access an Application in a Cluster](/docs/tutorials/stateless-application/expose-external-ip-address/) +#### Stateful Applications + +* [Running a Single-Instance Stateful Application](/docs/tutorials/stateful-application/run-stateful-application/) + ### What's next If you would like to write a tutorial, see diff --git a/docs/tutorials/stateful-application/gce-volume.yaml b/docs/tutorials/stateful-application/gce-volume.yaml new file mode 100644 index 0000000000..ddb9ecc3ce --- /dev/null +++ b/docs/tutorials/stateful-application/gce-volume.yaml @@ -0,0 +1,12 @@ +apiVersion: v1 +kind: PersistentVolume +metadata: + name: mysql-pv +spec: + capacity: + storage: 20Gi + accessModes: + - ReadWriteOnce + gcePersistentDisk: + pdName: mysql-disk + fsType: ext4 diff --git a/docs/tutorials/stateful-application/mysql-deployment.yaml b/docs/tutorials/stateful-application/mysql-deployment.yaml new file mode 100644 index 0000000000..3b2aa22f6c --- /dev/null +++ b/docs/tutorials/stateful-application/mysql-deployment.yaml @@ -0,0 +1,51 @@ +apiVersion: v1 +kind: Service +metadata: + name: mysql +spec: + ports: + - port: 3306 + selector: + app: mysql + clusterIP: None +--- +apiVersion: v1 +kind: PersistentVolumeClaim +metadata: + name: mysql-pv-claim +spec: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 20Gi +--- +apiVersion: extensions/v1beta1 +kind: Deployment +metadata: + name: mysql +spec: + strategy: + type: Recreate + template: + metadata: + labels: + app: mysql + spec: + containers: + - image: mysql:5.6 + name: mysql + env: + # Use secret in real usage + - name: MYSQL_ROOT_PASSWORD + value: password + ports: + - containerPort: 3306 + name: mysql + volumeMounts: + - name: mysql-persistent-storage + mountPath: /var/lib/mysql + volumes: + - name: mysql-persistent-storage + persistentVolumeClaim: + claimName: mysql-pv-claim diff --git a/docs/tutorials/stateful-application/run-stateful-application.md b/docs/tutorials/stateful-application/run-stateful-application.md new file mode 100644 index 0000000000..443d9cdea5 --- /dev/null +++ b/docs/tutorials/stateful-application/run-stateful-application.md @@ -0,0 +1,220 @@ +--- +--- + +{% capture overview %} + +This page shows you how to run a single-instance stateful application +in Kubernetes using a PersistentVolume and a Deployment. The +application is MySQL. + +{% endcapture %} + + +{% capture objectives %} + +* Create a PersistentVolume referencing a disk in your environment. +* Create a MySQL Deployment. +* Expose MySQL to other pods in the cluster at a known DNS name. + +{% endcapture %} + + +{% capture prerequisites %} + +* {% include task-tutorial-prereqs.md %} + +* For data persistence we will create a Persistent Volume that + references a disk in your + environment. See + [here](/docs/user-guide/persistent-volumes/#types-of-persistent-volumes) for + the types of environments supported. This Tutorial will demonstrate + `GCEPersistentDisk` but any type will work. `GCEPersistentDisk` + volumes only work on Google Compute Engine. + +{% endcapture %} + + +{% capture lessoncontent %} + +### Set up a disk in your environment + +You can use any type of persistent volume for your stateful app. See +[Types of Persistent Volumes](/docs/user-guide/persistent-volumes/#types-of-persistent-volumes) +for a list of supported environment disks. For Google Compute Engine, run: + +``` +gcloud compute disks create --size=20GB mysql-disk +``` + +Next create a PersistentVolume that points to the `mysql-disk` +disk just created. Here is a configuration file for a PersistentVolume +that points to the Compute Engine disk above: + +{% include code.html language="yaml" file="gce-volume.yaml" ghlink="/docs/tutorials/stateful-application/gce-volume.yaml" %} + +Notice that the `pdName: mysql-disk` line matches the name of the disk +in the Compute Engine environment. See the +[Persistent Volumes](/docs/user-guide/persistent-volumes/) +for details on writing a PersistentVolume configuration file for other +environments. + +Create the persistent volume: + +``` +kubectl create -f http://k8s.io/docs/tutorials/stateful-application/gce-volume.yaml +``` + + +### Deploy MySQL + +You can run a stateful application by creating a Kubernetes Deployment +and connecting it to an existing PersistentVolume using a +PersistentVolumeClaim. For example, this YAML file describes a +Deployment that runs MySQL and references the PersistentVolumeClaim. The file +defines a volume mount for /var/lib/mysql, and then creates a +PersistentVolumeClaim that looks for a 20G volume. This claim is +satisfied by any volume that meets the requirements, in this case, the +volume created above. + +Note: The password is defined in the config yaml, and this is insecure. See +[Kubernetes Secrets](/docs/user-guide/secrets/) +for a secure solution. + +{% include code.html language="yaml" file="mysql-deployment.yaml" ghlink="/docs/tutorials/stateful-application/mysql-deployment.yaml" %} + +1. Deploy the contents of the YAML file: + + kubectl create -f http://k8s.io/docs/tutorials/stateful-application/mysql-deployment.yaml + +1. Display information about the Deployment: + + kubectl describe deployment mysql + + Name: mysql + Namespace: default + CreationTimestamp: Tue, 01 Nov 2016 11:18:45 -0700 + Labels: app=mysql + Selector: app=mysql + Replicas: 1 updated | 1 total | 0 available | 1 unavailable + StrategyType: Recreate + MinReadySeconds: 0 + OldReplicaSets: + NewReplicaSet: mysql-63082529 (1/1 replicas created) + Events: + FirstSeen LastSeen Count From SubobjectPath Type Reason Message + --------- -------- ----- ---- ------------- -------- ------ ------- + 33s 33s 1 {deployment-controller } Normal ScalingReplicaSet Scaled up replica set mysql-63082529 to 1 + +1. List the pods created by the Deployment: + + kubectl get pods -l app=mysql + + NAME READY STATUS RESTARTS AGE + mysql-63082529-2z3ki 1/1 Running 0 3m + +1. Inspect the Persistent Volume: + + kubectl describe pv mysql-pv + + Name: mysql-pv + Labels: + Status: Bound + Claim: default/mysql-pv-claim + Reclaim Policy: Retain + Access Modes: RWO + Capacity: 20Gi + Message: + Source: + Type: GCEPersistentDisk (a Persistent Disk resource in Google Compute Engine) + PDName: mysql-disk + FSType: ext4 + Partition: 0 + ReadOnly: false + No events. + +1. Inspect the PersistentVolumeClaim: + + kubectl describe pvc mysql-pv-claim + + Name: mysql-pv-claim + Namespace: default + Status: Bound + Volume: mysql-pv + Labels: + Capacity: 20Gi + Access Modes: RWO + No events. + +### Accessing the MySQL instance + +The preceding YAML file creates a service that +allows other Pods in the cluster to access the database. The Service option +`clusterIP: None` lets the Service DNS name resolve directly to the +Pod's IP address. This is optimal when you have only one Pod +behind a Service and you don't intend to increase the number of Pods. + +Run a MySQL client to connect to the server: + +``` +kubectl run -it --rm --image=mysql:5.6 mysql-client -- mysql -h mysql -ppassword +``` + +This command creates a new Pod in the cluster running a mysql client +and connects it to the server through the Service. If it connects, you +know your stateful MySQL database is up and running. + +``` +Waiting for pod default/mysql-client-274442439-zyp6i to be running, status is Pending, pod ready: false +If you don't see a command prompt, try pressing enter. + +mysql> +``` + +### Updating + +The image or any other part of the Deployment can be updated as usual +with the `kubectl apply` command. Here are some precautions that are +specific to stateful apps: + +* Don't scale the app. This setup is for single-instance apps + only. The underlying PersistentVolume can only be mounted to one + Pod. For clustered stateful apps, see the + [StatefulSet documentation](/docs/user-guide/petset/). +* Use `strategy:` `type: Recreate` in the Deployment configuration + YAML file. This instructs Kubernetes to _not_ use rolling + updates. Rolling updates will not work, as you cannot have more than + one Pod running at a time. The `Recreate` strategy will stop the + first pod before creating a new one with the updated configuration. + +### Deleting a deployment + +Delete the deployed objects by name: + +``` +kubectl delete deployment,svc mysql +kubectl delete pvc mysql-pv-claim +kubectl delete pv mysql-pv +``` + +Also, if you are using Compute Engine disks: + +``` +gcloud compute disks delete mysql-disk +``` + +{% endcapture %} + + +{% capture whatsnext %} + +* Learn more about [Deployment objects](/docs/user-guide/deployments/). + +* Learn more about [Deploying applications](/docs/user-guide/deploying-applications/) + +* [kubectl run documentation](/docs/user-guide/kubectl/kubectl_run/) + +* [Volumes](/docs/user-guide/volumes/) and [Persistent Volumes](/docs/user-guide/persistent-volumes/) + +{% endcapture %} + +{% include templates/tutorial.md %} From 34a31f41ba232a9ac280d8fbad18a1ad19fb4dce Mon Sep 17 00:00:00 2001 From: Ahmet Alp Balkan Date: Thu, 17 Nov 2016 15:07:13 -0800 Subject: [PATCH 36/45] List Deployment next to ReplicationController RestartPolicy section gives an example of what kind of controller should be used for what type of workload. However it only lists rc for web-server type workloads. Adding Deployment next to rc as well. Signed-off-by: Ahmet Alp Balkan --- docs/user-guide/pod-states.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/user-guide/pod-states.md b/docs/user-guide/pod-states.md index b29270e5f8..8f745e9f56 100644 --- a/docs/user-guide/pod-states.md +++ b/docs/user-guide/pod-states.md @@ -66,8 +66,8 @@ The possible values for RestartPolicy are `Always`, `OnFailure`, or `Never`. If Three types of controllers are currently available: - Use a [`Job`](/docs/user-guide/jobs/) for pods which are expected to terminate (e.g. batch computations). -- Use a [`ReplicationController`](/docs/user-guide/replication-controller/) for pods which are not expected to - terminate (e.g. web servers). +- Use a [`ReplicationController`](/docs/user-guide/replication-controller/) or [`Deployment`](/docs/user-guide/deployments/) + for pods which are not expected to terminate (e.g. web servers). - Use a [`DaemonSet`](/docs/admin/daemons/): Use for pods which need to run 1 per machine because they provide a machine-specific system service. If you are unsure whether to use ReplicationController or Daemon, then see [Daemon Set versus From 9e25fddd86e02a261215160d50d1822029a80e37 Mon Sep 17 00:00:00 2001 From: Francois Deppierraz Date: Fri, 18 Nov 2016 00:10:14 +0100 Subject: [PATCH 37/45] typo kuberntes -> kubernetes --- docs/user-guide/node-selection/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/user-guide/node-selection/index.md b/docs/user-guide/node-selection/index.md index 49d30b51c9..725848b544 100644 --- a/docs/user-guide/node-selection/index.md +++ b/docs/user-guide/node-selection/index.md @@ -173,7 +173,7 @@ on node N if node N has a label with key `failure-domain.beta.kubernetes.io/zone such that there is at least one node in the cluster with key `failure-domain.beta.kubernetes.io/zone` and value V that is running a pod that has a label with key "security" and value "S1".) The pod anti-affinity rule says that the pod cannot schedule onto a node if that node is already running a pod with label -having key "security" and value "S2". (If the `topologyKey` were `failure-domain.beta.kuberntes.io/zone` then +having key "security" and value "S2". (If the `topologyKey` were `failure-domain.beta.kubernetes.io/zone` then it would mean that the pod cannot schedule onto a node if that node is in the same zone as a pod with label having key "security" and value "S2".) See the [design doc](https://github.com/kubernetes/kubernetes/blob/{{page.githubbranch}}/docs/design/podaffinity.md). for many more examples of pod affinity and anti-affinity, both the `requiredDuringSchedulingIgnoredDuringExecution` From 7f7fa5ac738b858e6e1271aa696c28f0494c3d64 Mon Sep 17 00:00:00 2001 From: Ahmet Alp Balkan Date: Thu, 17 Nov 2016 19:31:59 -0800 Subject: [PATCH 38/45] Fix broken links to some github paths --- docs/user-guide/accessing-the-cluster.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/user-guide/accessing-the-cluster.md b/docs/user-guide/accessing-the-cluster.md index 6f78ab5293..63134b4909 100644 --- a/docs/user-guide/accessing-the-cluster.md +++ b/docs/user-guide/accessing-the-cluster.md @@ -129,7 +129,7 @@ To use it, * Write an application atop of the client-go clients. Note that client-go defines its own API objects, so if needed, please import API definitions from client-go rather than from the main repository, e.g., `import "k8s.io/client-go/1.4/pkg/api/v1"` is correct. The Go client can use the same [kubeconfig file](/docs/user-guide/kubeconfig-file) -as the kubectl CLI does to locate and authenticate to the apiserver. See this [example](https://github.com/kubernetes/client-go/examples/out-of-cluster.go): +as the kubectl CLI does to locate and authenticate to the apiserver. See this [example](https://github.com/kubernetes/client-go/blob/master/examples/out-of-cluster/main.go): ```golang import ( @@ -183,7 +183,8 @@ From within a pod the recommended ways to connect to API are: in any container of the pod can access it. See this [example of using kubectl proxy in a pod](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/kubectl-container/). - use the Go client library, and create a client using the `client.NewInCluster()` factory. - This handles locating and authenticating to the apiserver. [example](https://github.com/kubernetes/client-go/examples/in-cluster.go) + This handles locating and authenticating to the apiserver. See this [example of using Go client + library in a pod](https://github.com/kubernetes/client-go/blob/master/examples/in-cluster/main.go). In each case, the credentials of the pod are used to communicate securely with the apiserver. From 9f0531ba75451a87f0350127d551906b16601a1f Mon Sep 17 00:00:00 2001 From: scotty Date: Fri, 18 Nov 2016 10:27:25 -0800 Subject: [PATCH 39/45] added "contribute to codebase" and "download k8s" buttons to the footer hide left nav area in docs if no nav is present update text link styles for better readability --- _includes/footer.html | 2 ++ _layouts/docwithnav.html | 27 +++++++++++++++++++++++++++ _sass/_base.sass | 19 +++++++++++++++++++ _sass/_reset.sass | 2 +- 4 files changed, 49 insertions(+), 1 deletion(-) diff --git a/_includes/footer.html b/_includes/footer.html index 15dacb72e0..e7fa36d26a 100644 --- a/_includes/footer.html +++ b/_includes/footer.html @@ -20,6 +20,8 @@ Events Calendar

© {{ 'now' | date: "%Y" }} Kubernetes
diff --git a/_layouts/docwithnav.html b/_layouts/docwithnav.html index 8ca0906065..c15077857a 100755 --- a/_layouts/docwithnav.html +++ b/_layouts/docwithnav.html @@ -80,6 +80,33 @@ })(window,document,'script','//www.google-analytics.com/analytics.js','ga'); ga('create', 'UA-36037335-10', 'auto'); ga('send', 'pageview'); + + // hide docs nav area if no nav is present + (function () { + window.addEventListener('DOMContentLoaded', init) + + // play nice with our neighbors + function init() { + window.removeEventListener('DOMContentLoaded', init) + hideNav() + } + + function hideNav(toc){ + if (!toc) toc = document.querySelector('#docsToc') + var container = toc.querySelector('.container') + + // container is built dynamically, so it may not be present on the first runloop + if (container) { + if (container.childElementCount === 0) { + document.getElementById('docsContent').style.width = '100%' + } + } else { + requestAnimationFrame(function () { + hideNav(toc) + }) + } + } + })(); diff --git a/_sass/_base.sass b/_sass/_base.sass index a8ac4b47c4..0feee846ff 100644 --- a/_sass/_base.sass +++ b/_sass/_base.sass @@ -389,6 +389,14 @@ footer display: block height: 0 overflow: hidden + + &.button + background-image: none + width: auto + height: auto + + &:hover + color: $blue a.twitter background-position: 0 0 @@ -874,8 +882,19 @@ dd img max-width: 100% + a + font-weight: 700 + text-decoration: underline + + a:visited + color: blueviolet + a.button border-radius: 2px + text-decoration: none + + &:visited + color: white a.issue margin-left: 20px diff --git a/_sass/_reset.sass b/_sass/_reset.sass index 9f4a43a68e..2a8bb1b6d2 100755 --- a/_sass/_reset.sass +++ b/_sass/_reset.sass @@ -15,7 +15,7 @@ ul, li ul margin: 0 padding: 0 - + a text-decoration: none From 88e32adc986da0cf4263aa2b670ac640febe0ed3 Mon Sep 17 00:00:00 2001 From: steveperry-53 Date: Thu, 17 Nov 2016 17:07:34 -0800 Subject: [PATCH 40/45] Write new task: Determining the Reason for Pod Failure. --- _data/tasks.yml | 8 ++ .../determine-reason-pod-failure.md | 110 ++++++++++++++++++ .../termination.yaml | 10 ++ 3 files changed, 128 insertions(+) create mode 100644 docs/tasks/debug-application-cluster/determine-reason-pod-failure.md create mode 100644 docs/tasks/debug-application-cluster/termination.yaml diff --git a/_data/tasks.yml b/_data/tasks.yml index 4c6cd1b709..9898fae32f 100644 --- a/_data/tasks.yml +++ b/_data/tasks.yml @@ -15,6 +15,14 @@ toc: section: - title: Using Port Forwarding to Access Applications in a Cluster path: /docs/tasks/access-application-cluster/port-forward-access-application-cluster/ + + +- title: Debugging Applications in a Cluster + section: + - title: Determining the Reason for Pod Failure + path: /docs/tasks/debug-application-cluster/determine-reason-pod-failure/ + + - title: Accessing the Kubernetes API section: - title: Using an HTTP Proxy to Access the Kubernetes API diff --git a/docs/tasks/debug-application-cluster/determine-reason-pod-failure.md b/docs/tasks/debug-application-cluster/determine-reason-pod-failure.md new file mode 100644 index 0000000000..f0f611e235 --- /dev/null +++ b/docs/tasks/debug-application-cluster/determine-reason-pod-failure.md @@ -0,0 +1,110 @@ +--- +--- + +{% capture overview %} + +This page shows how to write and read a Container +termination message. + +Termination messages provide a way for containers to write +information about fatal events to a location where it can +be easily retrieved and surfaced by tools like dashboards +and monitoring software. In most cases, information that you +put in a termination message should also be written to +the general +[Kubernetes logs](/docs/user-guide/logging/). + +{% endcapture %} + + +{% capture prerequisites %} + +{% include task-tutorial-prereqs.md %} + +{% endcapture %} + + +{% capture steps %} + +### Writing and reading a termination message + +In this exercise, you create a Pod that runs one container. +The configuration file specifies a command that runs when +the container starts. + +{% include code.html language="yaml" file="termination.yaml" ghlink="/docs/tasks/debug-pod-container/termination.yaml" %} + +1. Create a Pod based on the YAML configuration file: + + export REPO=https://raw.githubusercontent.com/kubernetes/kubernetes.github.io/master + kubectl create -f $REPO/docs/tasks/debug-pod-container/termination.yaml + + In the YAML file, in the `cmd` and `args` fields, you can see that the + container sleeps for 10 seconds and then writes "Sleep expired" to + the `/dev/termination-log` file. After the container writes + the "Sleep expired" message, it terminates. + +1. Display information about the Pod: + + kubectl get pod termination-demo + + Repeat the preceding command until the Pod is no longer running. + +1. Display detailed information about the Pod: + + kubectl get pod --output=yaml + + The output includes the "Sleep expired" message: + + apiVersion: v1 + kind: Pod + ... + lastState: + terminated: + containerID: ... + exitCode: 0 + finishedAt: ... + message: | + Sleep expired + ... + +1. Use a Go template to filter the output so that it includes +only the termination message: + +``` +{% raw %} kubectl get pod termination-demo -o go-template="{{range .status.containerStatuses}}{{.lastState.terminated.message}}{{end}}"{% endraw %} +``` + +### Setting the termination log file + +By default Kubernetes retrieves termination messages from +`/dev/termination-log`. To change this to a different file, +specify a `terminationMessagePath` field for your Container. + +For example, suppose your Container writes termination messages to +`/tmp/my-log`, and you want Kubernetes to retrieve those messages. +Set `terminationMessagePath` as shown here: + + apiVersion: v1 + kind: Pod + metadata: + name: msg-path-demo + spec: + containers: + - name: msg-path-demo-container + image: debian + terminationMessagePath: "/tmp/my-log" + +{% endcapture %} + +{% capture whatsnext %} + +* See the `terminationMessagePath` field in + [Container](/docs/api-reference/v1/definitions#_v1_container). +* Learn about [retrieving logs](/docs/user-guide/logging/). +* Learn about [Go templates](https://golang.org/pkg/text/template/). + +{% endcapture %} + + +{% include templates/task.md %} diff --git a/docs/tasks/debug-application-cluster/termination.yaml b/docs/tasks/debug-application-cluster/termination.yaml new file mode 100644 index 0000000000..3f63748f72 --- /dev/null +++ b/docs/tasks/debug-application-cluster/termination.yaml @@ -0,0 +1,10 @@ +apiVersion: v1 +kind: Pod +metadata: + name: termination-demo +spec: + containers: + - name: termination-demo-container + image: debian + command: ["/bin/sh"] + args: ["-c", "sleep 10 && echo Sleep expired > /dev/termination-log"] From 39d1962b287b0636748c695f5199ebbbf625d20b Mon Sep 17 00:00:00 2001 From: scotty Date: Fri, 18 Nov 2016 12:09:19 -0800 Subject: [PATCH 41/45] unbold a tags in #docsContent --- _sass/_base.sass | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_sass/_base.sass b/_sass/_base.sass index 0feee846ff..1eabc9ac14 100644 --- a/_sass/_base.sass +++ b/_sass/_base.sass @@ -883,7 +883,7 @@ dd max-width: 100% a - font-weight: 700 + //font-weight: 700 text-decoration: underline a:visited From 81b7f61c3b5d70a976574865014ba4eff3aaaffb Mon Sep 17 00:00:00 2001 From: scotty Date: Fri, 18 Nov 2016 12:56:07 -0800 Subject: [PATCH 42/45] disable TOC if there's only a link to the current page --- _layouts/docwithnav.html | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/_layouts/docwithnav.html b/_layouts/docwithnav.html index c15077857a..f3d885d246 100755 --- a/_layouts/docwithnav.html +++ b/_layouts/docwithnav.html @@ -81,7 +81,7 @@ ga('create', 'UA-36037335-10', 'auto'); ga('send', 'pageview'); - // hide docs nav area if no nav is present + // hide docs nav area if no nav is present, or if nav only contains a link to the current page (function () { window.addEventListener('DOMContentLoaded', init) @@ -97,7 +97,8 @@ // container is built dynamically, so it may not be present on the first runloop if (container) { - if (container.childElementCount === 0) { + if (container.childElementCount === 0 || toc.querySelectorAll('a.item').length === 1) { + toc.style.display = 'none' document.getElementById('docsContent').style.width = '100%' } } else { From 2a4501ed6306a71b320fc008ae629159c2eaa46d Mon Sep 17 00:00:00 2001 From: Jared Date: Fri, 18 Nov 2016 15:58:19 -0800 Subject: [PATCH 43/45] Removed confusing "Kubernetes" button from docs landing page. --- docs/index.md | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/docs/index.md b/docs/index.md index 0bdaa33b5c..c430dac710 100644 --- a/docs/index.md +++ b/docs/index.md @@ -5,9 +5,7 @@ assignees: --- -

The Kubernetes documentation can help you set up Kubernetes, learn about the system, or get your applications and workloads running on Kubernetes.

- -

Read the Kubernetes Overview

+

Kubernetes documentation can help you set up Kubernetes, learn about the system, or get your applications and workloads running on Kubernetes. To learn the basics of what Kubernetes is and how it works, read "What is Kubernetes".

Interactive Tutorial

@@ -40,4 +38,4 @@ assignees:

Tools

-

The tools page contains a list of native and third-party tools for Kubernetes.

\ No newline at end of file +

The tools page contains a list of native and third-party tools for Kubernetes.

From a8e129af8315bd612e0206dfc7ef044121500c6b Mon Sep 17 00:00:00 2001 From: Casey Davenport Date: Sat, 19 Nov 2016 17:52:21 -0500 Subject: [PATCH 44/45] Sort network plugins alphabetically. (#1583) --- docs/admin/addons.md | 10 ++++--- docs/admin/networking.md | 40 ++++++++++++-------------- docs/getting-started-guides/scratch.md | 8 +++--- 3 files changed, 29 insertions(+), 29 deletions(-) diff --git a/docs/admin/addons.md b/docs/admin/addons.md index f2bd18aaa5..192b0c86a8 100644 --- a/docs/admin/addons.md +++ b/docs/admin/addons.md @@ -7,18 +7,20 @@ Add-ons extend the functionality of Kubernetes. This page lists some of the available add-ons and links to their respective installation instructions. +Add-ons in each section are sorted alphabetically - the ordering does not imply any preferential status. + ## Networking and Network Policy -* [Weave Net](https://github.com/weaveworks/weave-kube) provides networking and network policy, will carry on working on both sides of a network partition, and does not require an external database. -* [Calico](http://docs.projectcalico.org/v1.5/getting-started/kubernetes/installation/hosted/) is a secure L3 networking and network policy provider. -* [Flannel](https://github.com/coreos/flannel/blob/master/Documentation/kube-flannel.yml) is a overlay network provider that can be used with Kubernetes. +* [Calico](http://docs.projectcalico.org/v1.6/getting-started/kubernetes/installation/hosted/) is a secure L3 networking and network policy provider. * [Canal](https://github.com/tigera/canal/tree/master/k8s-install/kubeadm) unites Flannel and Calico, providing networking and network policy. +* [Flannel](https://github.com/coreos/flannel/blob/master/Documentation/kube-flannel.yml) is a overlay network provider that can be used with Kubernetes. * [Romana](http://romana.io) is a Layer 3 networking solution for pod networks that also supports the [NetworkPolicy API](/docs/user-guide/networkpolicies/). Kubeadm add-on installation details available [here](https://github.com/romana/romana/tree/master/containerize). +* [Weave Net](https://github.com/weaveworks/weave-kube) provides networking and network policy, will carry on working on both sides of a network partition, and does not require an external database. ## Visualization & Control -* [Weave Scope](https://www.weave.works/documentation/scope-latest-installing/#k8s) is a tool for graphically visualizing your containers, pods, services etc. Use it in conjunction with a [Weave Cloud account](https://cloud.weave.works/) or host the UI yourself. * [Dashboard](https://github.com/kubernetes/dashboard#kubernetes-dashboard) is a dashboard web interface for Kubernetes. +* [Weave Scope](https://www.weave.works/documentation/scope-latest-installing/#k8s) is a tool for graphically visualizing your containers, pods, services etc. Use it in conjunction with a [Weave Cloud account](https://cloud.weave.works/) or host the UI yourself. ## Legacy Add-ons diff --git a/docs/admin/networking.md b/docs/admin/networking.md index 0cfd9d7989..406148ee9e 100644 --- a/docs/admin/networking.md +++ b/docs/admin/networking.md @@ -100,8 +100,19 @@ existence or non-existence of host ports. There are a number of ways that this network model can be implemented. This document is not an exhaustive study of the various methods, but hopefully serves as an introduction to various technologies and serves as a jumping-off point. -If some techniques become vastly preferable to others, we might detail them more -here. + +The following networking options are sorted alphabetically - the order does not +imply any preferential status. + +### Contiv + +[Contiv](https://github.com/contiv/netplugin) provides configurable networking (native l3 using BGP, overlay using vxlan, classic l2, or Cisco-SDN/ACI) for various use cases. [Contiv](http://contiv.io) is all open sourced. + +### Flannel + +[Flannel](https://github.com/coreos/flannel#flannel) is a very simple overlay +network that satisfies the Kubernetes requirements. Many +people have reported success with Flannel and Kubernetes. ### Google Compute Engine (GCE) @@ -158,29 +169,12 @@ Follow the "With Linux Bridge devices" section of [this very nice tutorial](http://blog.oddbit.com/2014/08/11/four-ways-to-connect-a-docker/) from Lars Kellogg-Stedman. -### Weave Net from Weaveworks - -[Weave Net](https://www.weave.works/products/weave-net/) is a -resilient and simple to use network for Kubernetes and its hosted applications. -Weave Net runs as a [CNI plug-in](https://www.weave.works/docs/net/latest/cni-plugin/) -or stand-alone. In either version, it doesn't require any configuration or extra code -to run, and in both cases, the network provides one IP address per pod - as is standard for Kubernetes. - - -### Flannel - -[Flannel](https://github.com/coreos/flannel#flannel) is a very simple overlay -network that satisfies the Kubernetes requirements. It installs in minutes and -should get you up and running if the above techniques are not working. Many -people have reported success with Flannel and Kubernetes. - ### OpenVSwitch [OpenVSwitch](/docs/admin/ovs-networking) is a somewhat more mature but also complicated way to build an overlay network. This is endorsed by several of the "Big Shops" for networking. - ### Project Calico [Project Calico](https://github.com/projectcalico/calico-containers/blob/master/docs/cni/kubernetes/README.md) is an open source container networking provider and network policy engine. @@ -193,9 +187,13 @@ Calico can also be run in policy enforcement mode in conjunction with other netw [Romana](http://romana.io) is an open source network and security automation solution that lets you deploy Kubernetes without an overlay network. Romana supports Kubernetes [Network Policy](/docs/user-guide/networkpolicies/) to provide isolation across network namespaces. -### Contiv +### Weave Net from Weaveworks -[Contiv](https://github.com/contiv/netplugin) provides configurable networking (native l3 using BGP, overlay using vxlan, classic l2, or Cisco-SDN/ACI) for various use cases. [Contiv](http://contiv.io) is all open sourced. +[Weave Net](https://www.weave.works/products/weave-net/) is a +resilient and simple to use network for Kubernetes and its hosted applications. +Weave Net runs as a [CNI plug-in](https://www.weave.works/docs/net/latest/cni-plugin/) +or stand-alone. In either version, it doesn't require any configuration or extra code +to run, and in both cases, the network provides one IP address per pod - as is standard for Kubernetes. ## Other reading diff --git a/docs/getting-started-guides/scratch.md b/docs/getting-started-guides/scratch.md index 84b0ffb5af..3fc23ec3dc 100644 --- a/docs/getting-started-guides/scratch.md +++ b/docs/getting-started-guides/scratch.md @@ -81,12 +81,12 @@ to implement one of the above options: - **Use a network plugin which is called by Kubernetes** - Kubernetes supports the [CNI](https://github.com/containernetworking/cni) network plugin interface. - - There are a number of solutions which provide plugins for Kubernetes: + - There are a number of solutions which provide plugins for Kubernetes (listed alphabetically): + - [Calico](http://docs.projectcalico.org/) - [Flannel](https://github.com/coreos/flannel) - - [Calico](https://github.com/projectcalico/calico-containers) - - [Weave](https://weave.works/) - - [Romana](http://romana.io/) - [Open vSwitch (OVS)](http://openvswitch.org/) + - [Romana](http://romana.io/) + - [Weave](http://weave.works/) - [More found here](/docs/admin/networking#how-to-achieve-this) - You can also write your own. - **Compile support directly into Kubernetes** From 5d6460b97953181bc001a11fc7dc5100dc90efcd Mon Sep 17 00:00:00 2001 From: Jared Date: Mon, 21 Nov 2016 13:23:58 -0800 Subject: [PATCH 45/45] Added link to DNS Admin Guide --- docs/user-guide/services/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/user-guide/services/index.md b/docs/user-guide/services/index.md index 94faabcd1c..e20d072d05 100644 --- a/docs/user-guide/services/index.md +++ b/docs/user-guide/services/index.md @@ -345,7 +345,7 @@ can do a DNS SRV query for `"_http._tcp.my-service.my-ns"` to discover the port number for `"http"`. The Kubernetes DNS server is the only way to access services of type -`ExternalName`. +`ExternalName`. More information is available in the [DNS Admin Guide](http://kubernetes.io/docs/admin/dns/). ## Headless services