Add config APIs metrics API

The 'metrics.k8s.io' APIs are exposed by the 'metrics-server' component to facilitate
auto-scaling. The other two APIs are for fetching metrics from custom or
external sources respectively.
pull/41212/head
Qiming Teng 2023-05-18 21:35:36 +08:00
parent 70b445f2b8
commit bc4758e3b2
6 changed files with 539 additions and 11 deletions

View File

@ -102,6 +102,15 @@ operator to use or manage a cluster.
* [v1beta3](/docs/reference/config-api/kubeadm-config.v1beta3/)
* [v1beta4](/docs/reference/config-api/kubeadm-config.v1beta4/)
## External APIs
These are the APIs defined by the Kubernetes project, but are not implemented
by the core project:
* [Metrics API (v1beta1)](/docs/reference/external-api/metrics.v1beta1/)
* [Custom Metrics API (v1beta2)](/docs/reference/external-api/custom-metrics.v1beta2)
* [External Metrics API (v1beta1)](/docs/reference/external-api/external-metrics.v1beta1)
## Design Docs
An archive of the design docs for Kubernetes functionality. Good starting points are

View File

@ -0,0 +1,4 @@
---
title: External APIs
weight: 135
---

View File

@ -0,0 +1,177 @@
---
title: Kubernetes Custom Metrics (v1beta2)
content_type: tool-reference
package: custom.metrics.k8s.io/v1beta2
auto_generated: true
---
<p>Package v1beta2 is the v1beta2 version of the custom_metrics API.</p>
## Resource Types
- [MetricListOptions](#custom-metrics-k8s-io-v1beta2-MetricListOptions)
- [MetricValue](#custom-metrics-k8s-io-v1beta2-MetricValue)
- [MetricValueList](#custom-metrics-k8s-io-v1beta2-MetricValueList)
## `MetricListOptions` {#custom-metrics-k8s-io-v1beta2-MetricListOptions}
<p>MetricListOptions is used to select metrics by their label selectors</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>apiVersion</code><br/>string</td><td><code>custom.metrics.k8s.io/v1beta2</code></td></tr>
<tr><td><code>kind</code><br/>string</td><td><code>MetricListOptions</code></td></tr>
<tr><td><code>labelSelector</code><br/>
<code>string</code>
</td>
<td>
<p>A selector to restrict the list of returned objects by their labels.
Defaults to everything.</p>
</td>
</tr>
<tr><td><code>metricLabelSelector</code><br/>
<code>string</code>
</td>
<td>
<p>A selector to restrict the list of returned metrics by their labels</p>
</td>
</tr>
</tbody>
</table>
## `MetricValue` {#custom-metrics-k8s-io-v1beta2-MetricValue}
**Appears in:**
- [MetricValueList](#custom-metrics-k8s-io-v1beta2-MetricValueList)
<p>MetricValue is the metric value for some object</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>apiVersion</code><br/>string</td><td><code>custom.metrics.k8s.io/v1beta2</code></td></tr>
<tr><td><code>kind</code><br/>string</td><td><code>MetricValue</code></td></tr>
<tr><td><code>describedObject</code> <B>[Required]</B><br/>
<a href="https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.28/#objectreference-v1-core"><code>core/v1.ObjectReference</code></a>
</td>
<td>
<p>a reference to the described object</p>
</td>
</tr>
<tr><td><code>metric</code> <B>[Required]</B><br/>
<a href="#custom-metrics-k8s-io-v1beta2-MetricIdentifier"><code>MetricIdentifier</code></a>
</td>
<td>
<span class="text-muted">No description provided.</span></td>
</tr>
<tr><td><code>timestamp</code> <B>[Required]</B><br/>
<a href="https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.28/#time-v1-meta"><code>meta/v1.Time</code></a>
</td>
<td>
<p>indicates the time at which the metrics were produced</p>
</td>
</tr>
<tr><td><code>windowSeconds</code> <B>[Required]</B><br/>
<code>int64</code>
</td>
<td>
<p>indicates the window ([Timestamp-Window, Timestamp]) from
which these metrics were calculated, when returning rate
metrics calculated from cumulative metrics (or zero for
non-calculated instantaneous metrics).</p>
</td>
</tr>
<tr><td><code>value</code> <B>[Required]</B><br/>
<a href="https://pkg.go.dev/k8s.io/apimachinery/pkg/api/resource#Quantity"><code>k8s.io/apimachinery/pkg/api/resource.Quantity</code></a>
</td>
<td>
<p>the value of the metric for this</p>
</td>
</tr>
</tbody>
</table>
## `MetricValueList` {#custom-metrics-k8s-io-v1beta2-MetricValueList}
<p>MetricValueList is a list of values for a given metric for some set of objects</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>apiVersion</code><br/>string</td><td><code>custom.metrics.k8s.io/v1beta2</code></td></tr>
<tr><td><code>kind</code><br/>string</td><td><code>MetricValueList</code></td></tr>
<tr><td><code>metadata</code> <B>[Required]</B><br/>
<a href="https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.28/#listmeta-v1-meta"><code>meta/v1.ListMeta</code></a>
</td>
<td>
<span class="text-muted">No description provided.</span></td>
</tr>
<tr><td><code>items</code> <B>[Required]</B><br/>
<a href="#custom-metrics-k8s-io-v1beta2-MetricValue"><code>[]MetricValue</code></a>
</td>
<td>
<p>the value of the metric across the described objects</p>
</td>
</tr>
</tbody>
</table>
## `MetricIdentifier` {#custom-metrics-k8s-io-v1beta2-MetricIdentifier}
**Appears in:**
- [MetricValue](#custom-metrics-k8s-io-v1beta2-MetricValue)
<p>MetricIdentifier identifies a metric by name and, optionally, selector</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>name</code> <B>[Required]</B><br/>
<code>string</code>
</td>
<td>
<p>name is the name of the given metric</p>
</td>
</tr>
<tr><td><code>selector</code><br/>
<a href="https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.28/#labelselector-v1-meta"><code>meta/v1.LabelSelector</code></a>
</td>
<td>
<p>selector represents the label selector that could be used to select
this metric, and will generally just be the selector passed in to
the query used to fetch this metric.
When left blank, only the metric's Name will be used to gather metrics.</p>
</td>
</tr>
</tbody>
</table>

View File

@ -0,0 +1,110 @@
---
title: Kubernetes External Metrics (v1beta1)
content_type: tool-reference
package: external.metrics.k8s.io/v1beta1
auto_generated: true
---
<p>Package v1beta1 is the v1beta1 version of the external metrics API.</p>
## Resource Types
- [ExternalMetricValue](#external-metrics-k8s-io-v1beta1-ExternalMetricValue)
- [ExternalMetricValueList](#external-metrics-k8s-io-v1beta1-ExternalMetricValueList)
## `ExternalMetricValue` {#external-metrics-k8s-io-v1beta1-ExternalMetricValue}
**Appears in:**
- [ExternalMetricValueList](#external-metrics-k8s-io-v1beta1-ExternalMetricValueList)
<p>ExternalMetricValue is a metric value for external metric
A single metric value is identified by metric name and a set of string labels.
For one metric there can be multiple values with different sets of labels.</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>apiVersion</code><br/>string</td><td><code>external.metrics.k8s.io/v1beta1</code></td></tr>
<tr><td><code>kind</code><br/>string</td><td><code>ExternalMetricValue</code></td></tr>
<tr><td><code>metricName</code> <B>[Required]</B><br/>
<code>string</code>
</td>
<td>
<p>the name of the metric</p>
</td>
</tr>
<tr><td><code>metricLabels</code> <B>[Required]</B><br/>
<code>map[string]string</code>
</td>
<td>
<p>a set of labels that identify a single time series for the metric</p>
</td>
</tr>
<tr><td><code>timestamp</code> <B>[Required]</B><br/>
<a href="https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.28/#time-v1-meta"><code>meta/v1.Time</code></a>
</td>
<td>
<p>indicates the time at which the metrics were produced</p>
</td>
</tr>
<tr><td><code>window</code> <B>[Required]</B><br/>
<code>int64</code>
</td>
<td>
<p>indicates the window ([Timestamp-Window, Timestamp]) from
which these metrics were calculated, when returning rate
metrics calculated from cumulative metrics (or zero for
non-calculated instantaneous metrics).</p>
</td>
</tr>
<tr><td><code>value</code> <B>[Required]</B><br/>
<a href="https://pkg.go.dev/k8s.io/apimachinery/pkg/api/resource#Quantity"><code>k8s.io/apimachinery/pkg/api/resource.Quantity</code></a>
</td>
<td>
<p>the value of the metric</p>
</td>
</tr>
</tbody>
</table>
## `ExternalMetricValueList` {#external-metrics-k8s-io-v1beta1-ExternalMetricValueList}
<p>ExternalMetricValueList is a list of values for a given metric for some set labels</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>apiVersion</code><br/>string</td><td><code>external.metrics.k8s.io/v1beta1</code></td></tr>
<tr><td><code>kind</code><br/>string</td><td><code>ExternalMetricValueList</code></td></tr>
<tr><td><code>metadata</code> <B>[Required]</B><br/>
<a href="https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.28/#listmeta-v1-meta"><code>meta/v1.ListMeta</code></a>
</td>
<td>
<span class="text-muted">No description provided.</span></td>
</tr>
<tr><td><code>items</code> <B>[Required]</B><br/>
<a href="#external-metrics-k8s-io-v1beta1-ExternalMetricValue"><code>[]ExternalMetricValue</code></a>
</td>
<td>
<p>value of the metric matching a given set of labels</p>
</td>
</tr>
</tbody>
</table>

View File

@ -0,0 +1,220 @@
---
title: Kubernetes Metrics (v1beta1)
content_type: tool-reference
package: metrics.k8s.io/v1beta1
auto_generated: true
---
<p>Package v1beta1 is the v1beta1 version of the metrics API.</p>
## Resource Types
- [NodeMetrics](#metrics-k8s-io-v1beta1-NodeMetrics)
- [NodeMetricsList](#metrics-k8s-io-v1beta1-NodeMetricsList)
- [PodMetrics](#metrics-k8s-io-v1beta1-PodMetrics)
- [PodMetricsList](#metrics-k8s-io-v1beta1-PodMetricsList)
## `NodeMetrics` {#metrics-k8s-io-v1beta1-NodeMetrics}
**Appears in:**
- [NodeMetricsList](#metrics-k8s-io-v1beta1-NodeMetricsList)
<p>NodeMetrics sets resource usage metrics of a node.</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>apiVersion</code><br/>string</td><td><code>metrics.k8s.io/v1beta1</code></td></tr>
<tr><td><code>kind</code><br/>string</td><td><code>NodeMetrics</code></td></tr>
<tr><td><code>metadata</code><br/>
<a href="https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.28/#objectmeta-v1-meta"><code>meta/v1.ObjectMeta</code></a>
</td>
<td>
<p>Standard object's metadata.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata</p>
Refer to the Kubernetes API documentation for the fields of the <code>metadata</code> field.</td>
</tr>
<tr><td><code>timestamp</code> <B>[Required]</B><br/>
<a href="https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.28/#time-v1-meta"><code>meta/v1.Time</code></a>
</td>
<td>
<p>The following fields define time interval from which metrics were
collected from the interval [Timestamp-Window, Timestamp].</p>
</td>
</tr>
<tr><td><code>window</code> <B>[Required]</B><br/>
<a href="https://pkg.go.dev/k8s.io/apimachinery/pkg/apis/meta/v1#Duration"><code>meta/v1.Duration</code></a>
</td>
<td>
<span class="text-muted">No description provided.</span></td>
</tr>
<tr><td><code>usage</code> <B>[Required]</B><br/>
<a href="https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.28/#resourcelist-v1-core"><code>core/v1.ResourceList</code></a>
</td>
<td>
<p>The memory usage is the memory working set.</p>
</td>
</tr>
</tbody>
</table>
## `NodeMetricsList` {#metrics-k8s-io-v1beta1-NodeMetricsList}
<p>NodeMetricsList is a list of NodeMetrics.</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>apiVersion</code><br/>string</td><td><code>metrics.k8s.io/v1beta1</code></td></tr>
<tr><td><code>kind</code><br/>string</td><td><code>NodeMetricsList</code></td></tr>
<tr><td><code>metadata</code> <B>[Required]</B><br/>
<a href="https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.28/#listmeta-v1-meta"><code>meta/v1.ListMeta</code></a>
</td>
<td>
<p>Standard list metadata.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds</p>
</td>
</tr>
<tr><td><code>items</code> <B>[Required]</B><br/>
<a href="#metrics-k8s-io-v1beta1-NodeMetrics"><code>[]NodeMetrics</code></a>
</td>
<td>
<p>List of node metrics.</p>
</td>
</tr>
</tbody>
</table>
## `PodMetrics` {#metrics-k8s-io-v1beta1-PodMetrics}
**Appears in:**
- [PodMetricsList](#metrics-k8s-io-v1beta1-PodMetricsList)
<p>PodMetrics sets resource usage metrics of a pod.</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>apiVersion</code><br/>string</td><td><code>metrics.k8s.io/v1beta1</code></td></tr>
<tr><td><code>kind</code><br/>string</td><td><code>PodMetrics</code></td></tr>
<tr><td><code>metadata</code><br/>
<a href="https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.28/#objectmeta-v1-meta"><code>meta/v1.ObjectMeta</code></a>
</td>
<td>
<p>Standard object's metadata.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata</p>
Refer to the Kubernetes API documentation for the fields of the <code>metadata</code> field.</td>
</tr>
<tr><td><code>timestamp</code> <B>[Required]</B><br/>
<a href="https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.28/#time-v1-meta"><code>meta/v1.Time</code></a>
</td>
<td>
<p>The following fields define time interval from which metrics were
collected from the interval [Timestamp-Window, Timestamp].</p>
</td>
</tr>
<tr><td><code>window</code> <B>[Required]</B><br/>
<a href="https://pkg.go.dev/k8s.io/apimachinery/pkg/apis/meta/v1#Duration"><code>meta/v1.Duration</code></a>
</td>
<td>
<span class="text-muted">No description provided.</span></td>
</tr>
<tr><td><code>containers</code> <B>[Required]</B><br/>
<a href="#metrics-k8s-io-v1beta1-ContainerMetrics"><code>[]ContainerMetrics</code></a>
</td>
<td>
<p>Metrics for all containers are collected within the same time window.</p>
</td>
</tr>
</tbody>
</table>
## `PodMetricsList` {#metrics-k8s-io-v1beta1-PodMetricsList}
<p>PodMetricsList is a list of PodMetrics.</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>apiVersion</code><br/>string</td><td><code>metrics.k8s.io/v1beta1</code></td></tr>
<tr><td><code>kind</code><br/>string</td><td><code>PodMetricsList</code></td></tr>
<tr><td><code>metadata</code> <B>[Required]</B><br/>
<a href="https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.28/#listmeta-v1-meta"><code>meta/v1.ListMeta</code></a>
</td>
<td>
<p>Standard list metadata.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds</p>
</td>
</tr>
<tr><td><code>items</code> <B>[Required]</B><br/>
<a href="#metrics-k8s-io-v1beta1-PodMetrics"><code>[]PodMetrics</code></a>
</td>
<td>
<p>List of pod metrics.</p>
</td>
</tr>
</tbody>
</table>
## `ContainerMetrics` {#metrics-k8s-io-v1beta1-ContainerMetrics}
**Appears in:**
- [PodMetrics](#metrics-k8s-io-v1beta1-PodMetrics)
<p>ContainerMetrics sets resource usage metrics of a container.</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>name</code> <B>[Required]</B><br/>
<code>string</code>
</td>
<td>
<p>Container name corresponding to the one from pod.spec.containers.</p>
</td>
</tr>
<tr><td><code>usage</code> <B>[Required]</B><br/>
<a href="https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.28/#resourcelist-v1-core"><code>core/v1.ResourceList</code></a>
</td>
<td>
<p>The memory usage is the memory working set.</p>
</td>
</tr>
</tbody>
</table>

View File

@ -81,7 +81,8 @@ Kubernetes implements horizontal pod autoscaling as a control loop that runs int
Once during each period, the controller manager queries the resource utilization against the
metrics specified in each HorizontalPodAutoscaler definition. The controller manager
finds the target resource defined by the `scaleTargetRef`,
then selects the pods based on the target resource's `.spec.selector` labels, and obtains the metrics from either the resource metrics API (for per-pod resource metrics),
then selects the pods based on the target resource's `.spec.selector` labels,
and obtains the metrics from either the resource metrics API (for per-pod resource metrics),
or the custom metrics API (for all other metrics).
- For per-pod resource metrics (like CPU), the controller fetches the metrics
@ -164,10 +165,12 @@ cannot exactly determine the first time a pod becomes ready when
determining whether to set aside certain CPU metrics. Instead, it
considers a Pod "not yet ready" if it's unready and transitioned to
ready within a short, configurable window of time since it started.
This value is configured with the `--horizontal-pod-autoscaler-initial-readiness-delay` flag, and its default is 30
seconds. Once a pod has become ready, it considers any transition to
This value is configured with the `--horizontal-pod-autoscaler-initial-readiness-delay`
flag, and its default is 30 seconds.
Once a pod has become ready, it considers any transition to
ready to be the first if it occurred within a longer, configurable time
since it started. This value is configured with the `--horizontal-pod-autoscaler-cpu-initialization-period` flag, and its
since it started. This value is configured with the
`--horizontal-pod-autoscaler-cpu-initialization-period` flag, and its
default is 5 minutes.
The `currentMetricValue / desiredMetricValue` base scale ratio is then
@ -205,7 +208,8 @@ the current value.
Finally, right before HPA scales the target, the scale recommendation is recorded. The
controller considers all recommendations within a configurable window choosing the
highest recommendation from within that window. This value can be configured using the `--horizontal-pod-autoscaler-downscale-stabilization` flag, which defaults to 5 minutes.
highest recommendation from within that window. This value can be configured using the
`--horizontal-pod-autoscaler-downscale-stabilization` flag, which defaults to 5 minutes.
This means that scaledowns will occur gradually, smoothing out the impact of rapidly
fluctuating metric values.
@ -341,27 +345,31 @@ overall maximum that you configured).
## Support for metrics APIs
By default, the HorizontalPodAutoscaler controller retrieves metrics from a series of APIs. In order for it to access these
APIs, cluster administrators must ensure that:
By default, the HorizontalPodAutoscaler controller retrieves metrics from a series of APIs.
In order for it to access these APIs, cluster administrators must ensure that:
- The [API aggregation layer](/docs/tasks/extend-kubernetes/configure-aggregation-layer/) is enabled.
- The corresponding APIs are registered:
- For resource metrics, this is the `metrics.k8s.io` API, generally provided by [metrics-server](https://github.com/kubernetes-sigs/metrics-server).
- For resource metrics, this is the `metrics.k8s.io` [API](/docs/reference/external-api/metrics.v1beta1/),
generally provided by [metrics-server](https://github.com/kubernetes-sigs/metrics-server).
It can be launched as a cluster add-on.
- For custom metrics, this is the `custom.metrics.k8s.io` API. It's provided by "adapter" API servers provided by metrics solution vendors.
- For custom metrics, this is the `custom.metrics.k8s.io` [API](/docs/reference/external-api/custom-metrics.v1beta2/).
It's provided by "adapter" API servers provided by metrics solution vendors.
Check with your metrics pipeline to see if there is a Kubernetes metrics adapter available.
- For external metrics, this is the `external.metrics.k8s.io` API. It may be provided by the custom metrics adapters provided above.
- For external metrics, this is the `external.metrics.k8s.io` [API](/docs/reference/external-api/external-metrics.v1beta1/).
It may be provided by the custom metrics adapters provided above.
For more information on these different metrics paths and how they differ please see the relevant design proposals for
[the HPA V2](https://git.k8s.io/design-proposals-archive/autoscaling/hpa-v2.md),
[custom.metrics.k8s.io](https://git.k8s.io/design-proposals-archive/instrumentation/custom-metrics-api.md)
and [external.metrics.k8s.io](https://git.k8s.io/design-proposals-archive/instrumentation/external-metrics-api.md).
For examples of how to use them see [the walkthrough for using custom metrics](/docs/tasks/run-application/horizontal-pod-autoscale-walkthrough/#autoscaling-on-multiple-metrics-and-custom-metrics)
For examples of how to use them see
[the walkthrough for using custom metrics](/docs/tasks/run-application/horizontal-pod-autoscale-walkthrough/#autoscaling-on-multiple-metrics-and-custom-metrics)
and [the walkthrough for using external metrics](/docs/tasks/run-application/horizontal-pod-autoscale-walkthrough/#autoscaling-on-metrics-not-related-to-kubernetes-objects).
## Configurable scaling behavior