kubernetes
/
website
mirror of https://github.com/kubernetes/website.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

ServiceCIDR documentation 

Add documentation about the new ServiceCIDR feature
and how to handle cluster updates of the default
ServiceCIDR
pull/49989/head
Antonio Ojea 2025-03-03 23:34:04 +00:00
parent 75b1f4a484
commit c582aa194e
2 changed files with 230 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

105
content/en/docs/tasks/network/extend-service-ip-ranges.md
View File

@ -19,9 +19,13 @@ This document shares how to extend the existing Service IP range assigned to a c
{{< version-check >}}
{{< note >}}
While you can use this feature with an earlier version, the feature is only GA and officially supported since v1.33.
{{< /note >}}
<!-- steps -->
## API
## Extend Service IP Ranges
Kubernetes clusters with kube-apiservers that have enabled the `MultiCIDRServiceAllocator`
[feature gate](/docs/reference/command-line-tools-reference/feature-gates/) and have the
@ -196,3 +200,102 @@ kubectl get servicecidr newcidr1
```
Error from server (NotFound): servicecidrs.networking.k8s.io "newcidr1" not found
```
## Kubernetes Service CIDR Policies
Cluster administrators can implement policies to control the creation and
modification of ServiceCIDR resources within the cluster. This allows for
centralized management of the IP address ranges used for Services and helps
prevent unintended or conflicting configurations. Kubernetes provides mechanisms
like Validating Admission Policies to enforce these rules.
### Preventing Unauthorized ServiceCIDR Creation/Update using Validating Admission Policy
There can be situations that the cluster administrators want to restrict the
ranges that can be allowed or to completely deny any changes to the cluster
Service IP ranges.
{{< note >}}
The default "kubernetes" ServiceCIDR is created by the kube-apiserver
to provide consistency in the cluster and is required for the cluster to work,
so it always must be allowed.
{{</ note >}}
#### Restrict Service CIDR ranges to some specific ranges
The following is an example of a `ValidatingAdmissionPolicy` that only allows
ServiceCIDRs to be created if they are subranges of the given `allowed` ranges.
(So the example policy would allow a ServiceCIDR with `cidrs: ['10.96.1.0/24']`
or `cidrs: ['2001:db8:0:0:ffff::/80', '10.96.0.0/20']` but would not allow a
ServiceCIDR with `cidrs: ['172.20.0.0/16']`.) You can copy this policy and change
the value of `allowed` to something appropriate for you cluster.
...
Consult the [CEL documentation](https://kubernetes.io/docs/reference/using-api/cel/)
to learn more about CEL if you want to write your own validation `expression`.
```yaml
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingAdmissionPolicy
metadata:
name: "servicecidrs.default"
spec:
failurePolicy: Fail
matchConstraints:
resourceRules:
- apiGroups: ["networking.k8s.io"]
apiVersions: ["v1","v1beta1"]
operations: ["CREATE", "UPDATE"]
resources: ["servicecidrs"]
matchConditions:
- name: 'exclude-default-servicecidr'
expression: "object.metadata.name != 'kubernetes'"
variables:
- name: allowed
expression: "['10.96.0.0/16','2001:db8::/64']"
validations:
- expression: "object.spec.cidrs.all(currentCIDR, variables.allowed.exists(allowedCIDR, cidr(allowedCIDR).containsCIDR(currentCIDR)))"
# For all CIDRs (i) listed in the spec.cidrs of the submitted ServiceCIDR
# object, check if there exists at least one CIDR (j) in the predefined
# allowed list such that the allowed CIDR (j) fully contains the submitted
# CIDR (i)."
---
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingAdmissionPolicyBinding
metadata:
name: "servicecidrs-binding"
spec:
policyName: "servicecidrs.default"
validationActions: [Deny,Audit]
```
#### Restrict any usage of the ServiceCIDR API
The following example demonstrates how to use a `ValidatingAdmissionPolicy` and
its binding to restrict the creation of any new Service CIDR ranges, excluding the default "kubernetes" ServiceCIDR:
```yaml
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingAdmissionPolicy
metadata:
name: "servicecidrs.deny"
spec:
failurePolicy: Fail
matchConstraints:
resourceRules:
- apiGroups: ["networking.k8s.io"]
apiVersions: ["v1","v1beta1"]
operations: ["CREATE", "UPDATE"]
resources: ["servicecidrs"]
validations:
- expression: "object.metadata.name == 'kubernetes'"
---
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingAdmissionPolicyBinding
metadata:
name: "servicecidrs-deny-binding"
spec:
policyName: "servicecidrs.deny"
validationActions: [Deny,Audit]
```

126
content/en/docs/tasks/network/reconfigure-default-service-ip-ranges.md Normal file
View File

@ -0,0 +1,126 @@
---
reviewers:
- thockin
- dwinship
min-kubernetes-server-version: v1.33
title: Kubernetes Default Service CIDR Reconfiguration
content_type: task
---
<!-- overview -->
{{< feature-state feature_gate_name="MultiCIDRServiceAllocator" >}}
This document shares how to reconfigure the default Service IP range(s) assigned
to a cluster.
## {{% heading "prerequisites" %}}
{{< include "task-tutorial-prereqs.md" >}}
{{< version-check >}}
<!-- steps -->
## Kubernetes Default Service CIDR Reconfiguration
This document explains how to manage the Service IP address range within a
Kubernetes cluster, which also influences the cluster's supported IP families
for Services.
The IP families available for Service ClusterIPs are determined by the
`--service-cluster-ip-range` flag to kube-apiserver. For a better understanding of Service IP address allocation, refer to the
[Services IP address allocation tracking](https://kubernetes.io/docs/reference/networking/virtual-ips/#ip-address-objects) documentation.
Since Kubernetes 1.33, the Service IP families configured for the cluster are
reflected by the `ServiceCIDR` object named `kubernetes`. The `kubernetes` `ServiceCIDR`
object is created by the first kube-apiserver instance that starts, based on its
configured `--service-cluster-ip-range` flag. To ensure consistent cluster behavior, all kube-apiserver instances must be configured with the same `--service-cluster-ip-range` values, which must match the default kubernetes ServiceCIDR object.
### Kubernetes Service CIDR Reconfiguration Categories
We can categorize Service CIDR reconfiguration into the following scenarios:
* **Extending the existing Service CIDRs:** This can be done dynamically by
adding new ServiceCIDR objects without the need of reconfiguration of the
kube-apiserver. Please refer to the dedicated documentation on
[Extending Service IP
Ranges](https://kubernetes.io/docs/tasks/network/extend-service-ip-ranges/).
* **Single-to-dual-stack conversion preserving the primary service CIDR:** This
involves introducing a secondary IP family (IPv6 to an IPv4-only cluster, or
IPv4 to an IPv6-only cluster) while keeping the original IP family as
primary. This requires an update to the kube-apiserver configuration and a
corresponding modification of various cluster components that need to handle
this additional IP family. These components include, but are not limited to,
kube-proxy, the CNI or network plugin, service mesh implementations, and DNS
services.
* **Dual-to-single conversion preserving the primary service CIDR:** This
involves removing the secondary IP family from a dual-stack cluster,
reverting to a single IP family while retaining the original primary IP
family. In addition to the reconfiguration of the components to match the
new IP family, you might need to address Services that were explicitly
configured to use the removed IP family.
* **Anything that results in changing the primary service CIDR:** Completely
replacing the default ServiceCIDR is a complex operation. If the new
ServiceCIDR does not overlap with the existing one, [it will require
renumbering all existing Services and changing the `kubernetes.default`
service](#Illustrative Reconfiguration Steps). The case where the primary IP
family also changes is even more complicated, and may require to change
multiple cluster components (kubelet, network plugins, etc.) to match the new
primary IP family.
### Manual Operations for Replacing the Default Service CIDR
Reconfiguring the default Service CIDR necessitates manual steps performed by
the cluster operator, administrator, or the software managing the cluster
lifecycle. These typically include:
1. **Updating** the kube-apiserver configuration: Modify the
`--service-cluster-ip-range` flag with the new IP range(s).
2. **Reconfiguring** the network components: This is a critical step and the
specific procedure depends on the different networking components in use. It
might involve updating configuration files, restarting agent pods, or
updating the components to manage the new Service CIDR(s) and the desired IP
family configuration for Pods. Typical components can be the implementation
of Kubernetes Services, such as kube-proxy, and the configured networking
plugin, and potentially other networking components like service mesh
controllers and DNS servers, to ensure they can correctly handle traffic and
perform service discovery with the new IP family configuration.
3. **Managing existing Services:** Services with IPs from the old CIDR need to
be addressed if they are not within the new configured ranges. Options
include recreation (leading to downtime and new IP assignments) or
potentially more complex reconfiguration strategies.
4. **Recreating internal Kubernetes services:** The `kubernetes.default`
service must be deleted and recreated to obtain an IP address from the new
Service CIDR if the primary IP family is changed or replaced by a different
network.
### Illustrative Reconfiguration Steps
The following steps describe a controlled reconfiguration focusing on the
completely replacement of the default Service CIDR and the recreation of the
`kubernetes.default` Service:
1. Start the kube-apiserver with the initial `--service-cluster-ip-range`.
2. Create initial Services that obtain IPs from this range.
3. Introduce a new Service CIDR as a temporary target for reconfiguration.
4. Mark the `kubernetes` default Service CIDR for deletion (it will remain
pending due to existing IPs and finalizers). This prevents new allocations
from the old range.
5. Recreate existing Services. They should now be allocated IPs from the new,
temporary Service CIDR.
6. Restart the kube-apiserver with the new Service CIDR(s) configured and shut
down the old instance.
7. Delete the `kubernetes.default` service. The new kube-apiserver will
recreate it within the new Service CIDR.
## {{% heading "whatsnext" %}}
* **Kubernetes Networking Concepts:**
[https://kubernetes.io/docs/concepts/cluster-administration/networking/](https://kubernetes.io/docs/concepts/cluster-administration/networking/)
* **Kubernetes Dual-Stack Services:**
[https://kubernetes.io/docs/concepts/services-networking/dual-stack/](https://kubernetes.io/docs/concepts/services-networking/dual-stack/)
* **Extending Kubernetes Service IP Ranges:**
[https://kubernetes.io/docs/tasks/network/extend-service-ip-ranges/](https://kubernetes.io/docs/tasks/network/extend-service-ip-ranges/)