From 1f5a82297491d714cda99b574366f695262ccbea Mon Sep 17 00:00:00 2001 From: Jordan Liggitt Date: Mon, 14 Nov 2016 21:38:15 -0500 Subject: [PATCH] kubelet authnz --- _data/reference.yml | 10 ++- docs/admin/index.md | 1 + .../kubelet-authentication-authorization.md | 81 +++++++++++++++++++ docs/admin/kubelet-tls-bootstrapping.md | 6 +- docs/admin/master-node-communication.md | 37 ++++++--- 5 files changed, 121 insertions(+), 14 deletions(-) create mode 100644 docs/admin/kubelet-authentication-authorization.md diff --git a/_data/reference.yml b/_data/reference.yml index 6f71d1677c..9cfe45d1a0 100644 --- a/_data/reference.yml +++ b/_data/reference.yml @@ -178,7 +178,15 @@ toc: - title: kube-scheduler path: /docs/admin/kube-scheduler/ - title: kubelet - path: /docs/admin/kubelet/ + section: + - title: Overview + path: /docs/admin/kubelet/ + - title: Master-Node communication + path: /docs/admin/master-node-communication/ + - title: TLS bootstrapping + path: /docs/admin/kubelet-tls-bootstrapping/ + - title: Kubelet authentication/authorization + path: /docs/admin/kubelet-authentication-authorization/ - title: Glossary section: diff --git a/docs/admin/index.md b/docs/admin/index.md index f6bde390f9..3624bb4202 100644 --- a/docs/admin/index.md +++ b/docs/admin/index.md @@ -88,3 +88,4 @@ project](/docs/admin/salt). * **Securing the kubelet** * [Master-Node communication](/docs/admin/master-node-communication/) * [TLS bootstrapping](/docs/admin/kubelet-tls-bootstrapping/) + * [Kubelet authentication/authorization](/docs/admin/kubelet-authentication-authorization/) diff --git a/docs/admin/kubelet-authentication-authorization.md b/docs/admin/kubelet-authentication-authorization.md new file mode 100644 index 0000000000..b0617b8854 --- /dev/null +++ b/docs/admin/kubelet-authentication-authorization.md @@ -0,0 +1,81 @@ +--- +assignees: +- liggitt + +--- + +* TOC +{:toc} + +## Overview + +A kubelet's HTTPS endpoint exposes APIs which give access to data of varying sensitivity, +and allow you to perform operations with varying levels of power on the node and within containers. + +This document describes how to authenticate and authorize access to the kubelet's HTTPS endpoint. + +## Kubelet authentication + +By default, requests to the kubelet's HTTPS endpoint that are not rejected by other configured +authentication methods are treated as anonymous requests, and given a username of `system:anonymous` +and a group of `system:unauthenticated`. + +To disable anonymous access and send `401 Unauthorized` responses to unauthenticated requests: +* start the kubelet with the `--anonymous-auth=false` flag + +To enable X509 client certificate authentication to the kubelet's HTTPS endpoint: +* start the kubelet with the `--client-ca-file` flag, providing a CA bundle to verify client certificates with +* start the apiserver with `--kubelet-client-certificate` and `--kubelet-client-key` flags +* see the [apiserver authentication documentation](/docs/admin/authentication/#x509-client-certs) for more details + +To enable API bearer tokens (including service account tokens) to be used to authenticate to the kubelet's HTTPS endpoint: +* ensure the `authentication.k8s.io/v1beta1` API group is enabled in the API server +* start the kubelet with the `--authentication-token-webhook`, `--kubeconfig`, and `--require-kubeconfig` flags +* the kubelet calls the `TokenReview` API on the configured API server to determine user information from bearer tokens + +## Kubelet authorization + +Any request that is successfully authenticated (including an anonymous request) is then authorized. The default authorization mode is `AlwaysAllow`, which allows all requests. + +There are many possible reasons to subdivide access to the kubelet API: +* anonymous auth is enabled, but anonymous users' ability to call the kubelet API should be limited +* bearer token auth is enabled, but arbitrary API users' (like service accounts) ability to call the kubelet API should be limited +* client certificate auth is enabled, but only some of the client certificates signed by the configured CA should be allowed to use the kubelet API + +To subdivide access to the kubelet API, delegate authorization to the API server: +* ensure the `authorization.k8s.io/v1beta1` API group is enabled in the API server +* start the kubelet with the `--authorization-mode=Webhook`, `--kubeconfig`, and `--require-kubeconfig` flags +* the kubelet calls the `SubjectAccessReview` API on the configured API server to determine whether each request is authorized + +The kubelet authorizes API requests using the same [request attributes](/docs/admin/authorization/#request-attributes) approach as the apiserver. + +The verb is determined from the incoming request's HTTP verb: + +HTTP verb | request verb +----------|--------------- +POST | create +GET, HEAD | get +PUT | update +PATCH | patch +DELETE | delete + +The resource and subresource is determined from the incoming request's path: + +Kubelet API | resource | subresource +-------------|----------|------------ +/stats/* | nodes | stats +/metrics/* | nodes | metrics +/logs/* | nodes | log +/spec/* | nodes | spec +*all others* | nodes | proxy + +The namespace and API group attributes are always an empty string, and +the resource name is always the name of the kubelet's `Node` API object. + +When running in this mode, ensure the user identified by the `--kubelet-client-certificate` and `--kubelet-client-key` +flags passed to the apiserver is authorized for the following attributes: +* verb=*, resource=nodes, subresource=proxy +* verb=*, resource=nodes, subresource=stats +* verb=*, resource=nodes, subresource=log +* verb=*, resource=nodes, subresource=spec +* verb=*, resource=nodes, subresource=metrics diff --git a/docs/admin/kubelet-tls-bootstrapping.md b/docs/admin/kubelet-tls-bootstrapping.md index 6d93c8e46b..3458bdb310 100644 --- a/docs/admin/kubelet-tls-bootstrapping.md +++ b/docs/admin/kubelet-tls-bootstrapping.md @@ -7,9 +7,9 @@ assignees: * TOC {:toc} -## Summary +## Overview -This document describes setting up TLS client certificate bootstrapping for kubelets. +This document describes how to set up TLS client certificate boostrapping for kubelets. Kubernetes 1.4 introduces an experimental API for requesting certificates from a cluster-level Certificate Authority (CA). The first supported use of this API is the provisioning of TLS client certificates for kubelets. The proposal can be found [here](https://github.com/kubernetes/kubernetes/pull/20439) @@ -92,5 +92,5 @@ approval controller, but for the alpha version of the API it can be done manuall An administrator can list CSRs with `kubectl get csr`, describe one in detail with `kubectl describe `. There are [currently no direct approve/deny commands](https://github.com/kubernetes/kubernetes/issues/30163) so an approver will need to update the Status field directly. A rough example of how to do this in bash which should only be used until the porcelain merges is available -at https://github.com/gtank/csrctl. +at [https://github.com/gtank/csrctl](https://github.com/gtank/csrctl). diff --git a/docs/admin/master-node-communication.md b/docs/admin/master-node-communication.md index d7424467de..9e8b9cfa9e 100644 --- a/docs/admin/master-node-communication.md +++ b/docs/admin/master-node-communication.md @@ -9,7 +9,7 @@ assignees: * TOC {:toc} -## Summary +## Overview This document catalogs the communication paths between the master (really the apiserver) and the Kubernetes cluster. The intent is to allow users to @@ -23,7 +23,11 @@ All communication paths from the cluster to the master terminate at the apiserver (none of the other master components are designed to expose remote services). In a typical deployment, the apiserver is configured to listen for remote connections on a secure HTTPS port (443) with one or more forms of -client [authentication](/docs/admin/authentication/) enabled. +client [authentication](/docs/admin/authentication/) enabled. One or more forms +of [authorization](/docs/admin/authorization/) should be enabled, especially +if [anonymous requests](/docs/admin/authentication/#anonymous-requests) or +[service account tokens](/docs/admin/authentication/#service-account-tokens) +are allowed. Nodes should be provisioned with the public root certificate for the cluster such that they can connect securely to the apiserver along with valid client @@ -58,16 +62,29 @@ cluster. The first is from the apiserver to the kubelet process which runs on each node in the cluster. The second is from the apiserver to any node, pod, or service through the apiserver's proxy functionality. +### apiserver -> kubelet + The connections from the apiserver to the kubelet are used for fetching logs for pods, attaching (through kubectl) to running pods, and using the kubelet's -port-forwarding functionality. These connections terminate at the kubelet's -HTTPS endpoint, which is typically using a self-signed certificate, and -ignore the certificate presented by the kubelet (although you can override this -behavior by specifying the `--kubelet-certificate-authority`, -`--kubelet-client-certificate`, and `--kubelet-client-key` flags when starting -the cluster apiserver). By default, these connections **are not currently safe** -to run over untrusted and/or public networks as they are subject to -man-in-the-middle attacks. +port-forwarding functionality. These connections terminate at the kubelet's +HTTPS endpoint. + +By default, the apiserver does not verify the kubelet's serving certificate, +which makes the connection subject to man-in-the-middle attacks, and +**unsafe** to run over untrusted and/or public networks. + +To verify this connection, use the `--kubelet-certificate-authority` flag to +provide the apiserver with a root certificates bundle to use to verify the +kubelet's serving certificate. + +If that is not possible, use [SSH tunneling](/docs/admin/master-node-communication/#ssh-tunnels) +between the apiserver and kubelet if required to avoid connecting over an +untrusted or public network. + +Finally, [Kubelet authentication and/or authorization](/docs/admin/kubelet-authentication-authorization/) +should be enabled to secure the kubelet API. + +### apiserver -> nodes, pods, and services The connections from the apiserver to a node, pod, or service default to plain HTTP connections and are therefore neither authenticated nor encrypted. They