Merge pull request #38563 from sftim/20221219_revise_service_concept_after_introduction_for_a_bit

Revise Service concept, part 13
pull/41161/head
Kubernetes Prow Robot 2023-05-15 18:05:14 -07:00 committed by GitHub
commit d219572a20
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 232 additions and 194 deletions

View File

@ -94,8 +94,7 @@ mechanisms to find the target it wants to connect to.
## Defining a Service
A Service in Kubernetes is an
{{< glossary_tooltip text="object" term_id="object" >}}
A Service is an {{< glossary_tooltip text="object" term_id="object" >}}
(the same way that a Pod or a ConfigMap is an object). You can create,
view or modify Service definitions using the Kubernetes API. Usually
you use a tool such as `kubectl` to make those API calls for you.
@ -401,7 +400,57 @@ also start and end with an alphanumeric character.
For example, the names `123-abc` and `web` are valid, but `123_abc` and `-web` are not.
{{< /note >}}
## Choosing your own IP address
## Service type {#publishing-services-service-types}
For some parts of your application (for example, frontends) you may want to expose a
Service onto an external IP address, one that's accessible from outside of your
cluster.
Kubernetes Service types allow you to specify what kind of Service you want.
The available `type` values and their behaviors are:
[`ClusterIP`](#type-clusterip)
: Exposes the Service on a cluster-internal IP. Choosing this value
makes the Service only reachable from within the cluster. This is the
default that is used if you don't explicitly specify a `type` for a Service.
You can expose the Service to the public internet using an [Ingress](/docs/concepts/services-networking/ingress/) or a
[Gateway](https://gateway-api.sigs.k8s.io/).
[`NodePort`](#type-nodeport)
: Exposes the Service on each Node's IP at a static port (the `NodePort`).
To make the node port available, Kubernetes sets up a cluster IP address,
the same as if you had requested a Service of `type: ClusterIP`.
[`LoadBalancer`](#loadbalancer)
: Exposes the Service externally using an external load balancer. Kubernetes
does not directly offer a load balancing component; you must provide one, or
you can integrate your Kubernetes cluster with a cloud provider.
[`ExternalName`](#externalname)
: Maps the Service to the contents of the `externalName` field (for example,
to the hostname `api.foo.bar.example`). The mapping configures your cluster's
DNS server to return a `CNAME` record with that external hostname value.
No proxying of any kind is set up.
The `type` field in the Service API is designed as nested functionality - each level
adds to the previous. This is not strictly required on all cloud providers, but
the Kubernetes API design for Service requires it anyway.
### `type: ClusterIP` {#type-clusterip}
This default Service type assigns an IP address from a pool of IP addresses that
your cluster has reserved for that purpose.
Several of the other types for Service build on the `ClusterIP` type as a
foundation.
If you define a Service that has the `.spec.clusterIP` set to `"None"` then
Kubernetes does not assign an IP address. See [headless Services](#headless-services)
for more information.
#### Choosing your own IP address
You can specify your own cluster IP address as part of a `Service` creation
request. To do this, set the `.spec.clusterIP` field. For example, if you
@ -410,148 +459,14 @@ that are configured for a specific IP address and difficult to re-configure.
The IP address that you choose must be a valid IPv4 or IPv6 address from within the
`service-cluster-ip-range` CIDR range that is configured for the API server.
If you try to create a Service with an invalid clusterIP address value, the API
If you try to create a Service with an invalid `clusterIP` address value, the API
server will return a 422 HTTP status code to indicate that there's a problem.
## Discovering services
Read [avoiding collisions](#avoiding-collisions)
to learn how Kubernetes helps reduce the risk and impact of two different Services
both trying to use the same IP address.
Kubernetes supports 2 primary modes of finding a Service - environment
variables and DNS.
### Environment variables
When a Pod is run on a Node, the kubelet adds a set of environment variables
for each active Service. It adds `{SVCNAME}_SERVICE_HOST` and `{SVCNAME}_SERVICE_PORT` variables,
where the Service name is upper-cased and dashes are converted to underscores.
It also supports variables (see [makeLinkVariables](https://github.com/kubernetes/kubernetes/blob/dd2d12f6dc0e654c15d5db57a5f9f6ba61192726/pkg/kubelet/envvars/envvars.go#L72))
that are compatible with Docker Engine's
"_[legacy container links](https://docs.docker.com/network/links/)_" feature.
For example, the Service `redis-primary` which exposes TCP port 6379 and has been
allocated cluster IP address 10.0.0.11, produces the following environment
variables:
```shell
REDIS_PRIMARY_SERVICE_HOST=10.0.0.11
REDIS_PRIMARY_SERVICE_PORT=6379
REDIS_PRIMARY_PORT=tcp://10.0.0.11:6379
REDIS_PRIMARY_PORT_6379_TCP=tcp://10.0.0.11:6379
REDIS_PRIMARY_PORT_6379_TCP_PROTO=tcp
REDIS_PRIMARY_PORT_6379_TCP_PORT=6379
REDIS_PRIMARY_PORT_6379_TCP_ADDR=10.0.0.11
```
{{< note >}}
When you have a Pod that needs to access a Service, and you are using
the environment variable method to publish the port and cluster IP to the client
Pods, you must create the Service *before* the client Pods come into existence.
Otherwise, those client Pods won't have their environment variables populated.
If you only use DNS to discover the cluster IP for a Service, you don't need to
worry about this ordering issue.
{{< /note >}}
### DNS
You can (and almost always should) set up a DNS service for your Kubernetes
cluster using an [add-on](/docs/concepts/cluster-administration/addons/).
A cluster-aware DNS server, such as CoreDNS, watches the Kubernetes API for new
Services and creates a set of DNS records for each one. If DNS has been enabled
throughout your cluster then all Pods should automatically be able to resolve
Services by their DNS name.
For example, if you have a Service called `my-service` in a Kubernetes
namespace `my-ns`, the control plane and the DNS Service acting together
create a DNS record for `my-service.my-ns`. Pods in the `my-ns` namespace
should be able to find the service by doing a name lookup for `my-service`
(`my-service.my-ns` would also work).
Pods in other namespaces must qualify the name as `my-service.my-ns`. These names
will resolve to the cluster IP assigned for the Service.
Kubernetes also supports DNS SRV (Service) records for named ports. If the
`my-service.my-ns` Service has a port named `http` with the protocol set to
`TCP`, you can do a DNS SRV query for `_http._tcp.my-service.my-ns` to discover
the port number for `http`, as well as the IP address.
The Kubernetes DNS server is the only way to access `ExternalName` Services.
You can find more information about `ExternalName` resolution in
[DNS for Services and Pods](/docs/concepts/services-networking/dns-pod-service/).
## Headless Services
Sometimes you don't need load-balancing and a single Service IP. In
this case, you can create what are termed "headless" Services, by explicitly
specifying `"None"` for the cluster IP (`.spec.clusterIP`).
You can use a headless Service to interface with other service discovery mechanisms,
without being tied to Kubernetes' implementation.
For headless `Services`, a cluster IP is not allocated, kube-proxy does not handle
these Services, and there is no load balancing or proxying done by the platform
for them. How DNS is automatically configured depends on whether the Service has
selectors defined:
### With selectors
For headless Services that define selectors, the Kubernetes control plane creates
EndpointSlice objects in the Kubernetes API, and modifies the DNS configuration to return
A or AAAA records (IPv4 or IPv6 addresses) that point directly to the Pods backing
the Service.
### Without selectors
For headless Services that do not define selectors, the control plane does
not create EndpointSlice objects. However, the DNS system looks for and configures
either:
* DNS CNAME records for [`type: ExternalName`](#externalname) Services.
* DNS A / AAAA records for all IP addresses of the Service's ready endpoints,
for all Service types other than `ExternalName`.
* For IPv4 endpoints, the DNS system creates A records.
* For IPv6 endpoints, the DNS system creates AAAA records.
## Publishing Services (ServiceTypes) {#publishing-services-service-types}
For some parts of your application (for example, frontends) you may want to expose a
Service onto an external IP address, that's outside of your cluster.
Kubernetes `ServiceTypes` allow you to specify what kind of Service you want.
`Type` values and their behaviors are:
* `ClusterIP`: Exposes the Service on a cluster-internal IP. Choosing this value
makes the Service only reachable from within the cluster. This is the
default that is used if you don't explicitly specify a `type` for a Service.
You can expose the service to the public with an [Ingress](/docs/concepts/services-networking/ingress/) or the
[Gateway API](https://gateway-api.sigs.k8s.io/).
* [`NodePort`](#type-nodeport): Exposes the Service on each Node's IP at a static port
(the `NodePort`).
To make the node port available, Kubernetes sets up a cluster IP address,
the same as if you had requested a Service of `type: ClusterIP`.
* [`LoadBalancer`](#loadbalancer): Exposes the Service externally using a cloud
provider's load balancer.
* [`ExternalName`](#externalname): Maps the Service to the contents of the
`externalName` field (e.g. `foo.bar.example.com`), by returning a `CNAME` record
with its value. No proxying of any kind is set up.
{{< note >}}
You need either `kube-dns` version 1.7 or CoreDNS version 0.0.8 or higher
to use the `ExternalName` type.
{{< /note >}}
The `type` field was designed as nested functionality - each level adds to the
previous. This is not strictly required on all cloud providers (for example: Google
Compute Engine does not need to allocate a node port to make `type: LoadBalancer` work,
but another cloud provider integration might do). Although strict nesting is not required,
but the Kubernetes API design for Service requires it anyway.
You can also use [Ingress](/docs/concepts/services-networking/ingress/) to expose your Service.
Ingress is not a Service type, but it acts as the entry point for your cluster.
It lets you consolidate your routing rules into a single resource as it can expose multiple
services under the same IP address.
### Type NodePort {#type-nodeport}
### `type: NodePort` {#type-nodeport}
If you set the `type` field to `NodePort`, the Kubernetes control plane
allocates a port from a range specified by `--service-node-port-range` flag (default: 30000-32767).
@ -579,7 +494,7 @@ You also have to use a valid port number, one that's inside the range configured
for NodePort use.
Here is an example manifest for a Service of `type: NodePort` that specifies
a NodePort value (30007, in this example).
a NodePort value (30007, in this example):
```yaml
apiVersion: v1
@ -640,7 +555,7 @@ If the `--nodeport-addresses` flag for kube-proxy or the equivalent field
in the kube-proxy configuration file is set, `<NodeIP>` would be a filtered node IP address (or possibly IP addresses).
{{< /note >}}
### Type LoadBalancer {#loadbalancer}
### `type: LoadBalancer` {#loadbalancer}
On cloud providers which support external load balancers, setting the `type`
field to `LoadBalancer` provisions a load balancer for your Service.
@ -669,13 +584,13 @@ status:
- ip: 192.0.2.127
```
Traffic from the external load balancer is directed at the backend Pods.
The cloud provider decides how it is load balanced.
Traffic from the external load balancer is directed at the backend Pods. The cloud
provider decides how it is load balanced.
To implement a Service of `type: LoadBalancer`, Kubernetes typically starts off
by making the changes that are equivalent to you requesting a Service of
`type: NodePort`. The cloud-controller-manager component then configures the external load balancer to
forward traffic to that assigned node port.
`type: NodePort`. The cloud-controller-manager component then configures the external
load balancer to forward traffic to that assigned node port.
You can configure a load balanced Service to
[omit](#load-balancer-nodeport-allocation) assigning a node port, provided that the
@ -683,23 +598,22 @@ cloud provider implementation supports this.
Some cloud providers allow you to specify the `loadBalancerIP`. In those cases, the load-balancer is created
with the user-specified `loadBalancerIP`. If the `loadBalancerIP` field is not specified,
the loadBalancer is set up with an ephemeral IP address. If you specify a `loadBalancerIP`
the load balancer is set up with an ephemeral IP address. If you specify a `loadBalancerIP`
but your cloud provider does not support the feature, the `loadbalancerIP` field that you
set is ignored.
{{< note >}}
{{< note >}}
The`.spec.loadBalancerIP` field for a Service was deprecated in Kubernetes v1.24.
This field was under-specified and its meaning varies across implementations. It also cannot support dual-stack networking. This field may be removed in a future API version.
If you're integrating with a provider that supports specifying the load balancer IP address(es)
for a Service via a (provider specific) annotation, you should switch to doing that.
If you are writing code for a load balancer integration with Kubernetes, avoid using this field.
You can integrate with [Gateway](https://gateway-api.sigs.k8s.io/) rather than Service, or you
can define your own (provider specific) annotations on the Service that specify the equivalent detail.
{{< /note >}}
#### Load balancers with mixed protocol types
@ -714,17 +628,15 @@ The feature gate `MixedProtocolLBService` (enabled by default for the kube-apise
different protocols for LoadBalancer type of Services, when there is more than one port defined.
{{< note >}}
The set of protocols that can be used for LoadBalancer type of Services is still defined by the cloud provider. If a
cloud provider does not support mixed protocols they will provide only a single protocol.
The set of protocols that can be used for load balanced Services is defined by your
cloud provider; they may impose restrictions beyond what the Kubernetes API enforces.
{{< /note >}}
#### Disabling load balancer NodePort allocation {#load-balancer-nodeport-allocation}
{{< feature-state for_k8s_version="v1.24" state="stable" >}}
You can optionally disable node port allocation for a Service of `type=LoadBalancer`, by setting
You can optionally disable node port allocation for a Service of `type: LoadBalancer`, by setting
the field `spec.allocateLoadBalancerNodePorts` to `false`. This should only be used for load balancer implementations
that route traffic directly to pods as opposed to using node ports. By default, `spec.allocateLoadBalancerNodePorts`
is `true` and type LoadBalancer Services will continue to allocate node ports. If `spec.allocateLoadBalancerNodePorts`
@ -735,11 +647,15 @@ You must explicitly remove the `nodePorts` entry in every Service port to de-all
{{< feature-state for_k8s_version="v1.24" state="stable" >}}
`spec.loadBalancerClass` enables you to use a load balancer implementation other than the cloud provider default.
By default, `spec.loadBalancerClass` is `nil` and a `LoadBalancer` type of Service uses
the cloud provider's default load balancer implementation if the cluster is configured with
a cloud provider using the `--cloud-provider` component flag.
If `spec.loadBalancerClass` is specified, it is assumed that a load balancer
For a Service with `type` set to `LoadBalancer`, the `.spec.loadBalancerClass` field
enables you to use a load balancer implementation other than the cloud provider default.
By default, `.spec.loadBalancerClass` is not set and a `LoadBalancer`
type of Service uses the cloud provider's default load balancer implementation if the
cluster is configured with a cloud provider using the `--cloud-provider` component
flag.
If you specify `.spec.loadBalancerClass`, it is assumed that a load balancer
implementation that matches the specified class is watching for Services.
Any default load balancer implementation (for example, the one provided by
the cloud provider) will ignore Services that have this field set.
@ -1134,7 +1050,9 @@ in those modified security groups.
Further documentation on annotations for Elastic IPs and other common use-cases may be found
in the [AWS Load Balancer Controller documentation](https://kubernetes-sigs.github.io/aws-load-balancer-controller/latest/guide/service/annotations/).
### Type ExternalName {#externalname}
### `type: ExternalName` {#externalname}
Services of type ExternalName map a Service to a DNS name, not to a typical selector such as
`my-service` or `cassandra`. You specify these Services with the `spec.externalName` parameter.
@ -1154,10 +1072,11 @@ spec:
```
{{< note >}}
ExternalName accepts an IPv4 address string, but as a DNS name comprised of digits, not as an IP address.
ExternalNames that resemble IPv4 addresses are not resolved by CoreDNS or ingress-nginx because ExternalName
is intended to specify a canonical DNS name. To hardcode an IP address, consider using
[headless Services](#headless-services).
A Service of `type: ExternalName` accepts an IPv4 address string, but treats that string as a DNS name comprised of digits,
not as an IP address (the internet does not however allow such names in DNS). Services with external names that resemble IPv4
addresses are not resolved by DNS servers.
If you want to map a Service directly to a specific IP address, consider using [headless Services](#headless-services).
{{< /note >}}
When looking up the host `my-service.prod.svc.cluster.local`, the cluster DNS Service
@ -1168,7 +1087,7 @@ forwarding. Should you later decide to move your database into your cluster, you
can start its Pods, add appropriate selectors or endpoints, and change the
Service's `type`.
{{< warning >}}
{{< caution >}}
You may have trouble using ExternalName for some common protocols, including HTTP and HTTPS.
If you use ExternalName then the hostname used by clients inside your cluster is different from
the name that the ExternalName references.
@ -1176,22 +1095,151 @@ the name that the ExternalName references.
For protocols that use hostnames this difference may lead to errors or unexpected responses.
HTTP requests will have a `Host:` header that the origin server does not recognize;
TLS servers will not be able to provide a certificate matching the hostname that the client connected to.
{{< /warning >}}
{{< /caution >}}
## Headless Services
Sometimes you don't need load-balancing and a single Service IP. In
this case, you can create what are termed _headless Services_, by explicitly
specifying `"None"` for the cluster IP address (`.spec.clusterIP`).
You can use a headless Service to interface with other service discovery mechanisms,
without being tied to Kubernetes' implementation.
For headless Services, a cluster IP is not allocated, kube-proxy does not handle
these Services, and there is no load balancing or proxying done by the platform
for them. How DNS is automatically configured depends on whether the Service has
selectors defined:
### With selectors
For headless Services that define selectors, the endpoints controller creates
EndpointSlices in the Kubernetes API, and modifies the DNS configuration to return
A or AAAA records (IPv4 or IPv6 addresses) that point directly to the Pods backing the Service.
### Without selectors
For headless Services that do not define selectors, the control plane does
not create EndpointSlice objects. However, the DNS system looks for and configures
either:
* DNS CNAME records for [`type: ExternalName`](#externalname) Services.
* DNS A / AAAA records for all IP addresses of the Service's ready endpoints,
for all Service types other than `ExternalName`.
* For IPv4 endpoints, the DNS system creates A records.
* For IPv6 endpoints, the DNS system creates AAAA records.
## Discovering services
For clients running inside your cluster, Kubernetes supports two primary modes of
finding a Service: environment variables and DNS.
### Environment variables
When a Pod is run on a Node, the kubelet adds a set of environment variables
for each active Service. It adds `{SVCNAME}_SERVICE_HOST` and `{SVCNAME}_SERVICE_PORT` variables,
where the Service name is upper-cased and dashes are converted to underscores.
It also supports variables (see [makeLinkVariables](https://github.com/kubernetes/kubernetes/blob/dd2d12f6dc0e654c15d5db57a5f9f6ba61192726/pkg/kubelet/envvars/envvars.go#L72))
that are compatible with Docker Engine's
"_[legacy container links](https://docs.docker.com/network/links/)_" feature.
For example, the Service `redis-primary` which exposes TCP port 6379 and has been
allocated cluster IP address 10.0.0.11, produces the following environment
variables:
```shell
REDIS_PRIMARY_SERVICE_HOST=10.0.0.11
REDIS_PRIMARY_SERVICE_PORT=6379
REDIS_PRIMARY_PORT=tcp://10.0.0.11:6379
REDIS_PRIMARY_PORT_6379_TCP=tcp://10.0.0.11:6379
REDIS_PRIMARY_PORT_6379_TCP_PROTO=tcp
REDIS_PRIMARY_PORT_6379_TCP_PORT=6379
REDIS_PRIMARY_PORT_6379_TCP_ADDR=10.0.0.11
```
{{< note >}}
This section is indebted to the [Kubernetes Tips - Part
1](https://akomljen.com/kubernetes-tips-part-1/) blog post from [Alen Komljen](https://akomljen.com/).
When you have a Pod that needs to access a Service, and you are using
the environment variable method to publish the port and cluster IP to the client
Pods, you must create the Service *before* the client Pods come into existence.
Otherwise, those client Pods won't have their environment variables populated.
If you only use DNS to discover the cluster IP for a Service, you don't need to
worry about this ordering issue.
{{< /note >}}
### External IPs
Kubernetes also supports and provides variables that are compatible with Docker
Engine's "_[legacy container links](https://docs.docker.com/network/links/)_" feature.
You can read [`makeLinkVariables`](https://github.com/kubernetes/kubernetes/blob/dd2d12f6dc0e654c15d5db57a5f9f6ba61192726/pkg/kubelet/envvars/envvars.go#L72)
to see how this is implemented in Kubernetes.
If there are external IPs that route to one or more cluster nodes, Kubernetes Services can be exposed on those
`externalIPs`. Traffic that ingresses into the cluster with the external IP (as destination IP), on the Service port,
will be routed to one of the Service endpoints. `externalIPs` are not managed by Kubernetes and are the responsibility
of the cluster administrator.
### DNS
In the Service spec, `externalIPs` can be specified along with any of the `ServiceTypes`.
In the example below, "`my-service`" can be accessed by clients on "`198.51.100.32:80`" (`externalIP:port`)
You can (and almost always should) set up a DNS service for your Kubernetes
cluster using an [add-on](/docs/concepts/cluster-administration/addons/).
A cluster-aware DNS server, such as CoreDNS, watches the Kubernetes API for new
Services and creates a set of DNS records for each one. If DNS has been enabled
throughout your cluster then all Pods should automatically be able to resolve
Services by their DNS name.
For example, if you have a Service called `my-service` in a Kubernetes
namespace `my-ns`, the control plane and the DNS Service acting together
create a DNS record for `my-service.my-ns`. Pods in the `my-ns` namespace
should be able to find the service by doing a name lookup for `my-service`
(`my-service.my-ns` would also work).
Pods in other namespaces must qualify the name as `my-service.my-ns`. These names
will resolve to the cluster IP assigned for the Service.
Kubernetes also supports DNS SRV (Service) records for named ports. If the
`my-service.my-ns` Service has a port named `http` with the protocol set to
`TCP`, you can do a DNS SRV query for `_http._tcp.my-service.my-ns` to discover
the port number for `http`, as well as the IP address.
The Kubernetes DNS server is the only way to access `ExternalName` Services.
You can find more information about `ExternalName` resolution in
[DNS for Services and Pods](/docs/concepts/services-networking/dns-pod-service/).
<!-- preserve existing hyperlinks -->
<a id="shortcomings" />
<a id="the-gory-details-of-virtual-ips" />
<a id="proxy-modes" />
<a id="proxy-mode-userspace" />
<a id="proxy-mode-iptables" />
<a id="proxy-mode-ipvs" />
<a id="ips-and-vips" />
## Virtual IP addressing mechanism
Read [Virtual IPs and Service Proxies](/docs/reference/networking/virtual-ips/) explains the
mechanism Kubernetes provides to expose a Service with a virtual IP address.
### Traffic policies
You can set the `.spec.internalTrafficPolicy` and `.spec.externalTrafficPolicy` fields
to control how Kubernetes routes traffic to healthy (“ready”) backends.
See [Traffic Policies](/docs/reference/networking/virtual-ips/#traffic-policies) for more details.
### Session stickiness
If you want to make sure that connections from a particular client are passed to
the same Pod each time, you can configure session affinity based on the client's
IP address. Read [session affinity](/docs/reference/networking/virtual-ips/#session-affinity)
to learn more.
## External IPs
If there are external IPs that route to one or more cluster nodes, Kubernetes Services
can be exposed on those `externalIPs`. When network traffic arrives into the cluster, with
the external IP (as destination IP) and the port matching that Service, rules and routes
that Kubernetes has configured ensure that the traffic is routed to one of the endpoints
for that Service.
When you define a Service, you can specify `externalIPs` for any
[service type](#publishing-services-service-types).
In the example below, the Service named `"my-service"` can be accessed by clients using TCP,
on `"198.51.100.32:80"` (calculated from `.spec.externalIP` and `.spec.port`).
```yaml
apiVersion: v1
@ -1210,26 +1258,16 @@ spec:
- 198.51.100.32
```
## Session stickiness
If you want to make sure that connections from a particular client are passed to
the same Pod each time, you can configure session affinity based on the client's
IP address. Read [session affinity](/docs/reference/networking/virtual-ips/#session-affinity)
to learn more.
{{< note >}}
Kubernetes does not manage allocation of `externalIPs`; these are the responsibility
of the cluster administrator.
{{< /note >}}
## API Object
Service is a top-level resource in the Kubernetes REST API. You can find more details
about the [Service API object](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#service-v1-core).
<!-- preserve existing hyperlinks -->
<a id="shortcomings" /><a id="#the-gory-details-of-virtual-ips" />
## Virtual IP addressing mechanism
Read [Virtual IPs and Service Proxies](/docs/reference/networking/virtual-ips/) to learn about the
mechanism Kubernetes provides to expose a Service with a virtual IP address.
## {{% heading "whatsnext" %}}
Learn more about Services and how they fit into Kubernetes: