Kuber

API Change
- The list-type of the alpha resourceClaims field introduced to Pods in 1.26.0 was modified from "set" to "map", resolving an incompatibility with use of this schema in CustomResourceDefinitions and with server-side apply. ([kubernetes/kubernetes114617](https://github.com/kubernetes/kubernetes/pull/114617), [JoelSpeed](https://github.com/JoelSpeed)) [SIG API Machinery]
- 'A new `preEnqueue` extension point was added to scheduler's component config
`v1beta2/v1beta3/v1`.'
([kubernetes/kubernetes113275](https://github.com/kubernetes/kubernetes/pull/113275), [Huang-Wei](https://github.com/Huang-Wei))
- 'Added a `ResourceClaim` API (in the `resource.k8s.io/v1alpha1` API group and
behind the `DynamicResourceAllocation` feature gate).
The new API is now more flexible than the existing Device Plugins feature of Kubernetes because it
allows Pods to request (claim) special kinds of resources, which can be available at node level, cluster
level, or following any other model you implement.' ([kubernetes/kubernetes111023](https://github.com/kubernetes/kubernetes/pull/111023), [pohly](https://github.com/pohly))
- 'Container `preStop` and `postStart` lifecycle handlers using `httpGet` now
honor the specified `scheme` and `headers` fields. This enables setting custom
headers and changing the scheme to `HTTPS`, consistent with container
startup/readiness/liveness probe capabilities. Lifecycle handlers configured
with `scheme: HTTPS` that encounter errors indicating the endpoint is actually
using HTTP fall back to making the request over HTTP for compatibility with
previous releases. When this happens, a `LifecycleHTTPFallback` event is recorded
in the namespace of the pod and a `kubelet_lifecycle_handler_http_fallbacks_total`
metric in the kubelet is incremented. Cluster administrators can opt out of the
expanded lifecycle handler capabilities by setting
`--feature-gates=ConsistentHTTPGetHandlers=false` in `kubelet`.'
([kubernetes/kubernetes86139](https://github.com/kubernetes/kubernetes/pull/86139), [jasimmons](https://github.com/jasimmons))
- 'Graduated `JobTrackingWithFinalizers` to stable.
Jobs created before the feature was enabled are still tracked without finalizers.
Jobs tracked with finalizers have the annotation batch.kubernetes.io/job-tracking.
If the annotation is present and the user attempts to remove it, the control plane adds it back.
The annotation `batch.kubernetes.io/job-tracking` is now deprecated.
The control plane will ignore it and stop adding it for new Jobs in v1.27.' ([kubernetes/kubernetes113510](https://github.com/kubernetes/kubernetes/pull/113510), [alculquicondor](https://github.com/alculquicondor))
- 'Kubelet added the following Pod failure conditions:
- `DisruptionTarget` (graceful node shutdown, node pressure eviction)' ([kubernetes/kubernetes112360](https://github.com/kubernetes/kubernetes/pull/112360), [mimowo](https://github.com/mimowo))
- 'Priority and Fairness has introduced a new feature called _borrowing_ that allows an API priority level
to borrow a number of seats from other priority level(s). As a cluster operator, you can enable borrowing
for a certain priority level configuration object via the two newly introduced fields `lendablePercent`, and
`borrowingLimitPercent` located under the `.spec.limited` field of the designated priority level.
This change added the following metrics:
- `apiserver_flowcontrol_nominal_limit_seats`: Nominal number of execution seats configured for each priority level
- `apiserver_flowcontrol_lower_limit_seats`: Configured lower bound on number of execution seats available to each priority level
- `apiserver_flowcontrol_upper_limit_seats`: Configured upper bound on number of execution seats available to each priority level
- `apiserver_flowcontrol_demand_seats`: Observations, at the end of every nanosecond, of (the number of seats each priority level could use) / (nominal number of seats for that level)
- `apiserver_flowcontrol_demand_seats_high_watermark`: High watermark, over last adjustment period, of demand_seats
- `apiserver_flowcontrol_demand_seats_average`: Time-weighted average, over last adjustment period, of demand_seats
- `apiserver_flowcontrol_demand_seats_stdev`: Time-weighted standard deviation, over last adjustment period, of demand_seats
- `apiserver_flowcontrol_demand_seats_smoothed`: Smoothed seat demands
- `apiserver_flowcontrol_target_seats`: Seat allocation targets
- `apiserver_flowcontrol_seat_fair_frac`: Fair fraction of server's concurrency to allocate to each priority level that can use it
- `apiserver_flowcontrol_current_limit_seats`: current derived number of execution seats available to each priority level
The possibility of borrowing means that the old metric `apiserver_flowcontrol_request_concurrency_limit` can no longer mean both the configured concurrency limit and the enforced concurrency limit. Henceforth it means the configured concurrency limit.' ([kubernetes/kubernetes113485](https://github.com/kubernetes/kubernetes/pull/113485), [MikeSpreitzer](https://github.com/MikeSpreitzer))
- '`NodeInclusionPolicy` in `podTopologySpread` plugin is now enabled by default.'
([kubernetes/kubernetes113500](https://github.com/kubernetes/kubernetes/pull/113500), [kerthcet](https://github.com/kerthcet))
- '`PodDisruptionBudget` now adds an alpha `spec.unhealthyPodEvictionPolicy` field.
When the `PDBUnhealthyPodEvictionPolicy` feature-gate is enabled in `kube-apiserver`,
setting this field to `"AlwaysAllow"` allows pods to be evicted if they do not
have a ready condition, regardless of whether the PodDisruptionBudget is currently
healthy.'
([kubernetes/kubernetes113375](https://github.com/kubernetes/kubernetes/pull/113375), [atiratree](https://github.com/atiratree))
- '`metav1.LabelSelectors` specified in API objects are now validated to ensure
they do not contain invalid label values that will error at time of use. Existing
invalid objects can be updated, but new objects are required to contain valid
label selectors.'
([kubernetes/kubernetes113699](https://github.com/kubernetes/kubernetes/pull/113699), [liggitt](https://github.com/liggitt))
- Add `percentageOfNodesToScore` as a scheduler profile level parameter to API version `v1`. When a profile `percentageOfNodesToScore` is set, it will override global `percentageOfNodesToScore`. ([kubernetes/kubernetes112521](https://github.com/kubernetes/kubernetes/pull/112521), [yuanchen8911](https://github.com/yuanchen8911))
- Add auth API to get self subject attributes (new selfsubjectreviews API is added).
The corresponding command for kubctl is provided - `kubectl auth whoami`. ([kubernetes/kubernetes111333](https://github.com/kubernetes/kubernetes/pull/111333), [nabokihms](https://github.com/nabokihms)) [SIG API Machinery, Auth, CLI and Testing]
- Added `kubernetes_feature_enabled` metric series to track whether each active feature gate is enabled. ([kubernetes/kubernetes112690](https://github.com/kubernetes/kubernetes/pull/112690), [logicalhan](https://github.com/logicalhan))
- Added a `--topology-manager-policy-options` flag to the kubelet to support fine tuning the topology manager policies. The first policy option, `prefer-closest-numa-nodes`, allows these policies to favor sets of NUMA nodes with shorter distance between nodes when making admission decisions. ([kubernetes/kubernetes112914](https://github.com/kubernetes/kubernetes/pull/112914), [PiotrProkop](https://github.com/PiotrProkop))
- Added a feature that allows a `StatefulSet` to start numbering replicas from an arbitrary non-negative ordinal, using the `.spec.ordinals.start` field. ([kubernetes/kubernetes112744](https://github.com/kubernetes/kubernetes/pull/112744), [pwschuurman](https://github.com/pwschuurman))
- Added a kube-proxy flag (`--iptables-localhost-nodeports`, default true) to allow disabling NodePort services on loopback addresses. Note: this only applies to iptables mode and ipv4. ([kubernetes/kubernetes108250](https://github.com/kubernetes/kubernetes/pull/108250), [cyclinder](https://github.com/cyclinder))
- Added a new namespace alpha field to `DataSourceRef` field in `PersistentVolumeClaim` API. ([kubernetes/kubernetes113186](https://github.com/kubernetes/kubernetes/pull/113186), [ttakahashi21](https://github.com/ttakahashi21))
- Aggregated discovery will be alpha and can be toggled with the `AggregatedDiscoveryEndpoint` feature flag. ([kubernetes/kubernetes113171](https://github.com/kubernetes/kubernetes/pull/113171), [Jefftree](https://github.com/Jefftree))
- Clarified the CFS quota as 100ms in the code comments and set the minimum `cpuCFSQuotaPeriod` to 1ms to match Linux kernel expectations. ([kubernetes/kubernetes112123](https://github.com/kubernetes/kubernetes/pull/112123), [paskal](https://github.com/paskal))
- Component-base: make the validation logic about LeaderElectionConfiguration consistent between component-base and client-go ([kubernetes/kubernetes111758](https://github.com/kubernetes/kubernetes/pull/111758), [SataQiu](https://github.com/SataQiu)) [SIG API Machinery and Scheduling]
- Deprecated the `apiserver_request_slo_duration_seconds` metric for v1.27 in favor of `apiserver_request_sli_duration_seconds` for naming consistency purposes with other SLI-specific metrics and to avoid any confusion between SLOs and SLIs. ([kubernetes/kubernetes112679](https://github.com/kubernetes/kubernetes/pull/112679), [dgrisonnet](https://github.com/dgrisonnet))
- Enable the "Retriable and non-retriable pod failures for jobs" feature into beta. ([kubernetes/kubernetes113360](https://github.com/kubernetes/kubernetes/pull/113360), [mimowo](https://github.com/mimowo))
- Enabled `kube-controller-manager` to support '--concurrent-horizontal-pod-autoscaler-syncs' flag to set the number of horizontal pod autoscaler controller workers. ([kubernetes/kubernetes108501](https://github.com/kubernetes/kubernetes/pull/108501), [zroubalik](https://github.com/zroubalik))
- Fixed spurious `field is immutable` errors validating updates to Event API objects via the `events.k8s.io/v1` API. ([kubernetes/kubernetes112183](https://github.com/kubernetes/kubernetes/pull/112183), [liggitt](https://github.com/liggitt))
- Graduated `ServiceInternalTrafficPolicy` feature to GA. ([kubernetes/kubernetes113496](https://github.com/kubernetes/kubernetes/pull/113496), [avoltz](https://github.com/avoltz))
- In 'kube-proxy`: The "userspace" proxy mode (deprecated for over a year) is no
longer supported on either Linux or Windows. Users should use "iptables" or "ipvs"
on Linux, or "kernelspace" on Windows.
([kubernetes/kubernetes112133](https://github.com/kubernetes/kubernetes/pull/112133), [knabben](https://github.com/knabben))
- Introduce `v1beta3` for Priority and Fairness with the following changes to the API spec:
- rename 'assuredConcurrencyShares' (located under `spec.limited') to 'nominalConcurrencyShares'.
- apply strategic merge patch annotations to 'Conditions' of flowschemas and `prioritylevelconfigurations`. ([kubernetes/kubernetes112306](https://github.com/kubernetes/kubernetes/pull/112306), [tkashem](https://github.com/tkashem))
- Introduced `v1alpha1` API for validating admission policies, enabling extensible admission control via CEL expressions (KEP 3488: CEL for Admission Control). To use, enable the `ValidatingAdmissionPolicy` feature gate and the `admissionregistration.k8s.io/v1alpha1` API via `--runtime-config`. ([kubernetes/kubernetes113314](https://github.com/kubernetes/kubernetes/pull/113314), [cici37](https://github.com/cici37))
- KMS: added validation for duplicate kms config name when auto reload is enabled. If you enabled automatic reload of encryption configuration with API server flag `--encryption-provider-config-automatic-reload`, ensure all the KMS provider names (v1 and v2) in the encryption configuration are unique. ([kubernetes/kubernetes113697](https://github.com/kubernetes/kubernetes/pull/113697), [aramase](https://github.com/aramase))
- Kubelet external Credential Provider feature is moved to GA. Credential Provider Plugin and Credential Provider Config APIs updated from `v1beta1` to `v1` with no API changes. ([kubernetes/kubernetes111616](https://github.com/kubernetes/kubernetes/pull/111616), [ndixita](https://github.com/ndixita))
- Legacy klog flags are no longer available. Only `-v` and `-vmodule` are still supported. ([kubernetes/kubernetes112120](https://github.com/kubernetes/kubernetes/pull/112120), [pohly](https://github.com/pohly)) [SIG Architecture, CLI, Instrumentation, Node and Testing]
- Moved `MixedProtocolLBService` from beta to GA. ([kubernetes/kubernetes112895](https://github.com/kubernetes/kubernetes/pull/112895), [janosi](https://github.com/janosi))
- New Pod API field `.spec.schedulingGates` is introduced to enable users to control when to mark a Pod as scheduling ready. ([kubernetes/kubernetes113274](https://github.com/kubernetes/kubernetes/pull/113274), [Huang-Wei](https://github.com/Huang-Wei))
- Protobuf serialization of metav1.MicroTime timestamps (used in `Lease` and `Event` API objects) has been corrected to truncate to microsecond precision, to match the documented behavior and JSON/YAML serialization. Any existing persisted data is truncated to microsecond when read from etcd. ([kubernetes/kubernetes111936](https://github.com/kubernetes/kubernetes/pull/111936), [haoruan](https://github.com/haoruan))
- Removed feature gates `ServiceLoadBalancerClass` and `ServiceLBNodePortControl`. These feature gates were enabled (and locked) since `v1.24`. ([kubernetes/kubernetes112577](https://github.com/kubernetes/kubernetes/pull/112577), [andrewsykim](https://github.com/andrewsykim))
- Reverted regression that prevented `client-go` latency metrics to be reported with a template URL to avoid label cardinality. ([kubernetes/kubernetes111752](https://github.com/kubernetes/kubernetes/pull/111752), [aanm](https://github.com/aanm))
- The `EndpointSliceTerminatingCondition` feature gate was graduated to GA. The gate is now locked and will be removed in v1.28. ([kubernetes/kubernetes113351](https://github.com/kubernetes/kubernetes/pull/113351), [andrewsykim](https://github.com/andrewsykim))
- `DynamicKubeletConfig` feature gate has been removed from the API server.
Dynamic kubelet reconfiguration now can't be used even when older nodes are still
attempting to rely on it. This is aligned with the Kubernetes version skew policy.
([kubernetes/kubernetes112643](https://github.com/kubernetes/kubernetes/pull/112643), [SergeyKanzhelev](https://github.com/SergeyKanzhelev))
- `kubectl wait` command with `jsonpath` flag will wait for target path until timeout.
([kubernetes/kubernetes109525](https://github.com/kubernetes/kubernetes/pull/109525), [jonyhy96](https://github.com/jonyhy96))
- Add a `ResourceClaim` API (in the resource.k8s.io/v1alpha1 API group and
behind the `DynamicResourceAllocation` feature gate).
The new API is more flexible than the existing Device Plugins feature of Kubernetes because it
allows Pods to request (claim) special kinds of resources, which can be available at node level, cluster
level, or following any other model you implement. ([kubernetes/kubernetes111023](https://github.com/kubernetes/kubernetes/pull/111023), [pohly](https://github.com/pohly)) [SIG API Machinery, Apps, Architecture, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Node, Release, Scheduling, Storage and Testing]
- PodDisruptionBudget adds an alpha `spec.unhealthyPodEvictionPolicy` field. When the `PDBUnhealthyPodEvictionPolicy` feature-gate is enabled in `kube-apiserver`, setting this field to `"AlwaysAllow"` allows pods to be evicted if they do not have a ready condition, regardless of whether the PodDisruptionBudget is currently healthy. ([kubernetes/kubernetes113375](https://github.com/kubernetes/kubernetes/pull/113375), [atiratree](https://github.com/atiratree)) [SIG API Machinery, Apps, Auth and Testing]
- A new `preEnqueue` extension point is added to scheduler's component config v1beta2/v1beta3/v1. ([kubernetes/kubernetes113275](https://github.com/kubernetes/kubernetes/pull/113275), [Huang-Wei](https://github.com/Huang-Wei)) [SIG API Machinery, Apps, Instrumentation, Scheduling and Testing]
- Add a new namespace alpha field to dataSourceRef field in PersistentVolumeClaim API. ([kubernetes/kubernetes113186](https://github.com/kubernetes/kubernetes/pull/113186), [ttakahashi21](https://github.com/ttakahashi21)) [SIG API Machinery, Apps, Storage and Testing]
- Add a kube-proxy flag (--iptables-localhost-nodeports, default true) to allow disabling NodePort services on loopback addresses. Note: this only applies to iptables mode and ipv4. ([kubernetes/kubernetes108250](https://github.com/kubernetes/kubernetes/pull/108250), [cyclinder](https://github.com/cyclinder)) [SIG API Machinery, Cloud Provider, Network, Node, Scalability, Storage and Testing]
- Added a --topology-manager-policy-options flag to the kubelet to support fine tuning the topology manager policies. The first policy option, `prefer-closest-numa-nodes`, allows these policies to favor sets of NUMA nodes with shorter distance between nodes when making admission decisions. ([kubernetes/kubernetes112914](https://github.com/kubernetes/kubernetes/pull/112914), [PiotrProkop](https://github.com/PiotrProkop)) [SIG API Machinery and Node]
- Added a feature that allows a StatefulSet to start numbering replicas from an arbitrary non-negative ordinal, using the `.spec.ordinals.start` field. ([kubernetes/kubernetes112744](https://github.com/kubernetes/kubernetes/pull/112744), [pwschuurman](https://github.com/pwschuurman)) [SIG API Machinery and Apps]
- Deprecate the apiserver_request_slo_duration_seconds metric for v1.27 in favor of apiserver_request_sli_duration_seconds for naming consistency purposes with other SLI-specific metrics and to avoid any confusion between SLOs and SLIs. ([kubernetes/kubernetes112679](https://github.com/kubernetes/kubernetes/pull/112679), [dgrisonnet](https://github.com/dgrisonnet)) [SIG API Machinery and Instrumentation]
- Enable the "Retriable and non-retriable pod failures for jobs" feature into beta ([kubernetes/kubernetes113360](https://github.com/kubernetes/kubernetes/pull/113360), [mimowo](https://github.com/mimowo)) [SIG Apps, Auth, Node, Scheduling and Testing]
- Graduate JobTrackingWithFinalizers to stable.
Jobs created before the feature was enabled are still tracked without finalizers.
Users can choose to migrate jobs to tracking with finalizers by adding the annotation batch.kubernetes.io/job-tracking.
If the annotation was already present and the user attempts to remove it, the control plane adds the annotation back. ([kubernetes/kubernetes113510](https://github.com/kubernetes/kubernetes/pull/113510), [alculquicondor](https://github.com/alculquicondor)) [SIG API Machinery, Apps and Testing]
- Graduate ServiceInternalTrafficPolicy feature to GA ([kubernetes/kubernetes113496](https://github.com/kubernetes/kubernetes/pull/113496), [avoltz](https://github.com/avoltz)) [SIG Apps and Network]
- If you enabled automatic reload of encryption configuration with API server flag --encryption-provider-config-automatic-reload, ensure all the KMS provider names (v1 and v2) in the encryption configuration are unique. ([kubernetes/kubernetes113697](https://github.com/kubernetes/kubernetes/pull/113697), [aramase](https://github.com/aramase)) [SIG API Machinery and Auth]
- Introduce v1alpha1 API for validating admission policies, enabling extensible admission control via CEL expressions (KEP 3488: CEL for Admission Control). To use, enable the `ValidatingAdmissionPolicy` feature gate and the `admissionregistration.k8s.io/v1alpha1` API via `--runtime-config`. ([kubernetes/kubernetes113314](https://github.com/kubernetes/kubernetes/pull/113314), [cici37](https://github.com/cici37)) [SIG API Machinery, Auth, Cloud Provider and Testing]
- Kubelet adds the following pod failure conditions:
- DisruptionTarget (graceful node shutdown, node pressure eviction) ([kubernetes/kubernetes112360](https://github.com/kubernetes/kubernetes/pull/112360), [mimowo](https://github.com/mimowo)) [SIG Apps, Node and Testing]
- Metav1.LabelSelectors specified in API objects are now validated to ensure they do not contain invalid label values that will error at time of use. Existing invalid objects can be updated, but new objects are required to contain valid label selectors. ([kubernetes/kubernetes113699](https://github.com/kubernetes/kubernetes/pull/113699), [liggitt](https://github.com/liggitt)) [SIG API Machinery, Apps, Auth, Network and Storage]
- Moving MixedProtocolLBService from beta to GA ([kubernetes/kubernetes112895](https://github.com/kubernetes/kubernetes/pull/112895), [janosi](https://github.com/janosi)) [SIG Apps, Network and Testing]
- New Pod API field `.spec.schedulingGates` is introduced to enable users to control when to mark a Pod as scheduling ready. ([kubernetes/kubernetes113274](https://github.com/kubernetes/kubernetes/pull/113274), [Huang-Wei](https://github.com/Huang-Wei)) [SIG Apps, Scheduling and Testing]
- NodeInclusionPolicy in podTopologySpread plugin is enabled by default. ([kubernetes/kubernetes113500](https://github.com/kubernetes/kubernetes/pull/113500), [kerthcet](https://github.com/kerthcet)) [SIG API Machinery, Apps, Scheduling and Testing]
- Priority and Fairness has introduced a new feature called _borrowing_ that allows an API priority level
to borrow a number of seats from other priority level(s). As a cluster operator, you can enable borrowing
for a certain priority level configuration object via the two newly introduced fields `lendablePercent`, and
`borrowingLimitPercent` located under the `.spec.limited` field of the designated priority level.
This PR adds the following metrics.
- `apiserver_flowcontrol_nominal_limit_seats`: Nominal number of execution seats configured for each priority level
- `apiserver_flowcontrol_lower_limit_seats`: Configured lower bound on number of execution seats available to each priority level
- `apiserver_flowcontrol_upper_limit_seats`: Configured upper bound on number of execution seats available to each priority level
- `apiserver_flowcontrol_demand_seats`: Observations, at the end of every nanosecond, of (the number of seats each priority level could use) / (nominal number of seats for that level)
- `apiserver_flowcontrol_demand_seats_high_watermark`: High watermark, over last adjustment period, of demand_seats
- `apiserver_flowcontrol_demand_seats_average`: Time-weighted average, over last adjustment period, of demand_seats
- `apiserver_flowcontrol_demand_seats_stdev`: Time-weighted standard deviation, over last adjustment period, of demand_seats
- `apiserver_flowcontrol_demand_seats_smoothed`: Smoothed seat demands
- `apiserver_flowcontrol_target_seats`: Seat allocation targets
- `apiserver_flowcontrol_seat_fair_frac`: Fair fraction of server's concurrency to allocate to each priority level that can use it
- `apiserver_flowcontrol_current_limit_seats`: current derived number of execution seats available to each priority level

The possibility of borrowing means that the old metric apiserver_flowcontrol_request_concurrency_limit can no longer mean both the configured concurrency limit and the enforced concurrency limit. Henceforth it means the configured concurrency limit. ([kubernetes/kubernetes113485](https://github.com/kubernetes/kubernetes/pull/113485), [MikeSpreitzer](https://github.com/MikeSpreitzer)) [SIG API Machinery and Testing]
- The EndpointSliceTerminatingCondition feature gate has graduated to GA. The gate is now locked and will be removed in v1.28. ([kubernetes/kubernetes113351](https://github.com/kubernetes/kubernetes/pull/113351), [andrewsykim](https://github.com/andrewsykim)) [SIG API Machinery, Apps, Network and Testing]
- Yes, aggregated discovery will be alpha and can be toggled with the AggregatedDiscoveryEndpoint feature flag ([kubernetes/kubernetes113171](https://github.com/kubernetes/kubernetes/pull/113171), [Jefftree](https://github.com/Jefftree)) [SIG API Machinery, Apps, Architecture, Auth, Autoscaling, CLI, Cloud Provider, Cluster Lifecycle, Network, Node, Release, Scalability, Scheduling, Storage and Testing]
- **Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.**:

<!--
This section can be blank if this pull request does not require a release note.

When adding links which point to resources within git repositories, like
KEPs or supporting documentation, please reference a specific commit and avoid
linking directly to the master branch. This ensures that links reference a
specific point in time, rather than a document that may change over time.

See here for guidance on getting permanent links to files: https://help.github.com/en/articles/getting-permanent-links-to-files

Please use the following format for linking documentation:
- [KEP]: <link>
- [Usage]: <link>
- [Other doc]: <link>
--> ([kubernetes/kubernetes86139](https://github.com/kubernetes/kubernetes/pull/86139), [jasimmons](https://github.com/jasimmons)) [SIG API Machinery, Apps, Architecture, Auth, Autoscaling, CLI, Contributor Experience, Instrumentation, Network, Node, Release, Scheduling, Storage and Testing]
- Add percentageOfNodesToScore as a scheduler profile level parameter to API version v1. If a profile percentageOfNodesToScore is set, it will override global percentageOfNodesToScore. ([kubernetes/kubernetes112521](https://github.com/kubernetes/kubernetes/pull/112521), [yuanchen8911](https://github.com/yuanchen8911)) [SIG API Machinery, Scheduling and Testing]
- Kube-controller-manager supports '--concurrent-horizontal-pod-autoscaler-syncs' flag to set the number of horizontal pod autoscaler controller workers. ([kubernetes/kubernetes108501](https://github.com/kubernetes/kubernetes/pull/108501), [zroubalik](https://github.com/zroubalik)) [SIG API Machinery, Apps and Autoscaling]
- Kube-proxy: The "userspace" proxy mode (deprecated for over a year) is no longer supported on either Linux or Windows. Users should use "iptables" or "ipvs" on Linux, or "kernelspace" on Windows. ([kubernetes/kubernetes112133](https://github.com/kubernetes/kubernetes/pull/112133), [knabben](https://github.com/knabben)) [SIG API Machinery, Network, Scalability, Testing and Windows]
- Kubectl wait command with jsonpath flag will wait for target path appear until timeout. ([kubernetes/kubernetes109525](https://github.com/kubernetes/kubernetes/pull/109525), [jonyhy96](https://github.com/jonyhy96)) [SIG CLI and Testing]
- Kubelet external Credential Provider feature is moved to GA. Credential Provider Plugin and Credential Provider Config APIs updated from v1beta1 to v1 with no API changes. ([kubernetes/kubernetes111616](https://github.com/kubernetes/kubernetes/pull/111616), [ndixita](https://github.com/ndixita)) [SIG API Machinery, Node, Scheduling and Testing]
- The `DynamicKubeletConfig` feature gate has been removed from the API server. Dynamic kubelet reconfiguration now cannot be used even when older nodes are still attempting to rely on it. This is aligned with the Kubernetes version skew policy. ([kubernetes/kubernetes112643](https://github.com/kubernetes/kubernetes/pull/112643), [SergeyKanzhelev](https://github.com/SergeyKanzhelev)) [SIG API Machinery, Apps, Auth, Node and Testing]
- Add `kubernetes_feature_enabled` metric series to track whether each active feature gate is enabled. ([kubernetes/kubernetes112690](https://github.com/kubernetes/kubernetes/pull/112690), [logicalhan](https://github.com/logicalhan)) [SIG API Machinery, Architecture, Cluster Lifecycle, Instrumentation, Network, Node and Scheduling]
- Introduce v1beta3 for Priority and Fairness with the following changes to the API spec:
- rename 'assuredConcurrencyShares' (located under spec.limited') to 'nominalConcurrencyShares'
- apply strategic merge patch annotations to 'Conditions' of flowschemas and prioritylevelconfigurations ([kubernetes/kubernetes112306](https://github.com/kubernetes/kubernetes/pull/112306), [tkashem](https://github.com/tkashem)) [SIG API Machinery and Testing]
- Legacy klog flags are no longer available. Only `-v` and `-vmodule` are still supported. ([kubernetes/kubernetes112120](https://github.com/kubernetes/kubernetes/pull/112120), [pohly](https://github.com/pohly)) [SIG Architecture, CLI, Instrumentation, Node and Testing]
- The feature gates ServiceLoadBalancerClass and ServiceLBNodePortControl have been removed. These feature gates were enabled (and locked) since v1.24. ([kubernetes/kubernetes112577](https://github.com/kubernetes/kubernetes/pull/112577), [andrewsykim](https://github.com/andrewsykim)) [SIG Apps]
- Add auth API to get self subject attributes (new selfsubjectreviews API is added).
The corresponding command for kubctl is provided - `kubectl auth whoami`. ([kubernetes/kubernetes111333](https://github.com/kubernetes/kubernetes/pull/111333), [nabokihms](https://github.com/nabokihms)) [SIG API Machinery, Auth, CLI and Testing]
- Clarified the CFS quota as 100ms in the code comments and set the minimum cpuCFSQuotaPeriod to 1ms to match Linux kernel expectations. ([kubernetes/kubernetes112123](https://github.com/kubernetes/kubernetes/pull/112123), [paskal](https://github.com/paskal)) [SIG API Machinery and Node]
- Component-base: make the validation logic about LeaderElectionConfiguration consistent between component-base and client-go ([kubernetes/kubernetes111758](https://github.com/kubernetes/kubernetes/pull/111758), [SataQiu](https://github.com/SataQiu)) [SIG API Machinery and Scheduling]
- Fixes spurious `field is immutable` errors validating updates to Event API objects via the `events.k8s.io/v1` API ([kubernetes/kubernetes112183](https://github.com/kubernetes/kubernetes/pull/112183), [liggitt](https://github.com/liggitt)) [SIG Apps]
- Protobuf serialization of metav1.MicroTime timestamps (used in `Lease` and `Event` API objects) has been corrected to truncate to microsecond precision, to match the documented behavior and JSON/YAML serialization. Any existing persisted data is truncated to microsecond when read from etcd. ([kubernetes/kubernetes111936](https://github.com/kubernetes/kubernetes/pull/111936), [haoruan](https://github.com/haoruan)) [SIG API Machinery]
- Revert regression that prevented client-go latency metrics to be reported with a template URL to avoid label cardinality. ([kubernetes/kubernetes111752](https://github.com/kubernetes/kubernetes/pull/111752), [aanm](https://github.com/aanm)) [SIG API Machinery]
- [kubelet] Change default `cpuCFSQuotaPeriod` value with enabled `cpuCFSQuotaPeriod` flag from 100ms to 100µs to match the Linux CFS and k8s defaults. `cpuCFSQuotaPeriod` of 100ms now requires `customCPUCFSQuotaPeriod` flag to be set to work. ([kubernetes/kubernetes111520](https://github.com/kubernetes/kubernetes/pull/111520), [paskal](https://github.com/paskal)) [SIG API Machinery and Node]

Feature
- Adds support for loading CA certificates from a file using the `idp-certificate-authority` key for the oidc plugin. (1916, vgupta3)

API Change
- Revert regression that prevented client-go latency metrics to be reported with a template URL to avoid label cardinality. ([kubernetes/kubernetes112055](https://github.com/kubernetes/kubernetes/pull/112055), [aanm](https://github.com/aanm)) [SIG API Machinery]
- Add `NodeInclusionPolicy` to `TopologySpreadConstraints` in PodSpec. ([kubernetes/kubernetes108492](https://github.com/kubernetes/kubernetes/pull/108492), [kerthcet](https://github.com/kerthcet))
- Added KMS v2alpha1 support. ([kubernetes/kubernetes111126](https://github.com/kubernetes/kubernetes/pull/111126), [aramase](https://github.com/aramase))
- Added a deprecated warning for node beta label usage in PV/SC/RC and CSI Storage Capacity. ([kubernetes/kubernetes108554](https://github.com/kubernetes/kubernetes/pull/108554), [pacoxu](https://github.com/pacoxu))
- Added a new feature gate `CheckpointRestore` to enable support to checkpoint containers. If enabled it is possible to checkpoint a container using the newly kubelet API (/checkpoint/{podNamespace}/{podName}/{containerName}). ([kubernetes/kubernetes104907](https://github.com/kubernetes/kubernetes/pull/104907), [adrianreber](https://github.com/adrianreber)) [SIG Node and Testing]
- Added alpha support for user namespaces in pods phase 1 (KEP 127, feature gate: UserNamespacesStatelessPodsSupport) ([kubernetes/kubernetes111090](https://github.com/kubernetes/kubernetes/pull/111090), [rata](https://github.com/rata))
- As of v1.25, the PodSecurity `restricted` level no longer requires pods that set .spec.os.name="windows" to also set Linux-specific securityContext fields. If a 1.25+ cluster has unsupported [out-of-skew](https://kubernetes.io/releases/version-skew-policy/#kubelet) nodes prior to v1.23 and wants to ensure namespaces enforcing the `restricted` policy continue to require Linux-specific securityContext fields on all pods, ensure a version of the `restricted` prior to v1.25 is selected by labeling the namespace (for example, `pod-security.kubernetes.io/enforce-version: v1.24`) ([kubernetes/kubernetes105919](https://github.com/kubernetes/kubernetes/pull/105919), [ravisantoshgudimetla](https://github.com/ravisantoshgudimetla))
- Changed ownership semantics of PersistentVolume's spec.claimRef from `atomic` to `granular`. ([kubernetes/kubernetes110495](https://github.com/kubernetes/kubernetes/pull/110495), [alexzielenski](https://github.com/alexzielenski))
- Extended ContainerStatus CRI API to allow runtime response with container resource requests and limits that are in effect.
- UpdateContainerResources CRI API now supports both Linux and Windows. ([kubernetes/kubernetes111645](https://github.com/kubernetes/kubernetes/pull/111645), [vinaykul](https://github.com/vinaykul))
- For v1.25, Kubernetes will be using Golang 1.19, In this PR the version is updated to 1.19rc2 as GA is not yet available. ([kubernetes/kubernetes111254](https://github.com/kubernetes/kubernetes/pull/111254), [dims](https://github.com/dims))
- Introduced NodeIPAM support for multiple ClusterCIDRs ([kubernetes/kubernetes2593](https://github.com/kubernetes/enhancements/issues/2593)) as an alpha feature.
Set feature gate `MultiCIDRRangeAllocator=true`, determines whether the `MultiCIDRRangeAllocator` controller can be used, while the kube-controller-manager flag below will pick the active controller.
Enabled the `MultiCIDRRangeAllocator` by setting `--cidr-allocator-type=MultiCIDRRangeAllocator` flag in kube-controller-manager. ([kubernetes/kubernetes109090](https://github.com/kubernetes/kubernetes/pull/109090), [sarveshr7](https://github.com/sarveshr7))
- Introduced PodHasNetwork condition for pods. ([kubernetes/kubernetes111358](https://github.com/kubernetes/kubernetes/pull/111358), [ddebroy](https://github.com/ddebroy))
- Introduced support for handling pod failures with respect to the configured pod failure policy rules. ([kubernetes/kubernetes111113](https://github.com/kubernetes/kubernetes/pull/111113), [mimowo](https://github.com/mimowo))
- Introduction of the `DisruptionTarget` pod condition type. Its `reason` field indicates the reason for pod termination:
- PreemptionByKubeScheduler (Pod preempted by kube-scheduler)
- DeletionByTaintManager (Pod deleted by taint manager due to NoExecute taint)
- EvictionByEvictionAPI (Pod evicted by Eviction API)
- DeletionByPodGC (an orphaned Pod deleted by PodGC) ([kubernetes/kubernetes110959](https://github.com/kubernetes/kubernetes/pull/110959), [mimowo](https://github.com/mimowo))
- Kube-Scheduler ComponentConfig is graduated to GA, `kubescheduler.config.k8s.io/v1` is available now.
Plugin `SelectorSpread` is removed in v1. ([kubernetes/kubernetes110534](https://github.com/kubernetes/kubernetes/pull/110534), [kerthcet](https://github.com/kerthcet))
- Local Storage Capacity Isolation feature is GA in 1.25 release. For systems (rootless) that cannot check root file system, please use kubelet config --local-storage-capacity-isolation=false to disable this feature. Once disabled, pod cannot set local ephemeral storage request/limit, and emptyDir sizeLimit niether. ([kubernetes/kubernetes111513](https://github.com/kubernetes/kubernetes/pull/111513), [jingxu97](https://github.com/jingxu97))
- Make PodSpec.Ports' description clearer on how this information is only informational and how it can be incorrect. ([kubernetes/kubernetes110564](https://github.com/kubernetes/kubernetes/pull/110564), [j4m3s-s](https://github.com/j4m3s-s)) [SIG API Machinery, Network and Node]
- On compatible systems, a mounter's Unmount implementation is changed to not return an error when the specified target can be detected as not a mount point. On Linux, the behavior of detecting a mount point depends on `umount` command is validated when the mounter is created. Additionally, mount point checks will be skipped in CleanupMountPoint/CleanupMountWithForce if the mounter's Unmount having the changed behavior of not returning error when target is not a mount point. ([kubernetes/kubernetes109676](https://github.com/kubernetes/kubernetes/pull/109676), [cartermckinnon](https://github.com/cartermckinnon)) [SIG Storage]
- PersistentVolumeClaim objects are no longer left with storage class set to `nil` forever, but will be updated retroactively once any StorageClass is set or created as default. ([kubernetes/kubernetes111467](https://github.com/kubernetes/kubernetes/pull/111467), [RomanBednar](https://github.com/RomanBednar))
- Promote StatefulSet minReadySeconds to GA. This means `--feature-gates=StatefulSetMinReadySeconds=true` are not needed on kube-apiserver and kube-controller-manager binaries and they'll be removed soon following policy at https://kubernetes.io/docs/reference/using-api/deprecation-policy/#deprecation ([kubernetes/kubernetes110896](https://github.com/kubernetes/kubernetes/pull/110896), [ravisantoshgudimetla](https://github.com/ravisantoshgudimetla)) [SIG API Machinery, Apps and Testing]
- Promoted CronJob's TimeZone support to beta. ([kubernetes/kubernetes111435](https://github.com/kubernetes/kubernetes/pull/111435), [soltysh](https://github.com/soltysh))
- Promoted DaemonSet MaxSurge to GA. This means `--feature-gates=DaemonSetUpdateSurge=true` are not needed on kube-apiserver and kube-controller-manager binaries and they'll be removed soon following policy at https://kubernetes.io/docs/reference/using-api/deprecation-policy/#deprecation . ([kubernetes/kubernetes111194](https://github.com/kubernetes/kubernetes/pull/111194), [ravisantoshgudimetla](https://github.com/ravisantoshgudimetla))
- Scheduler: included supported ScoringStrategyType list in error message for NodeResourcesFit plugin ([kubernetes/kubernetes111206](https://github.com/kubernetes/kubernetes/pull/111206), [SataQiu](https://github.com/SataQiu))
- The Go API for logging configuration in `k8s.io/component-base` was moved to `k8s.io/component-base/logs/api/v1`. The configuration file format and command line flags are the same as before. ([kubernetes/kubernetes105797](https://github.com/kubernetes/kubernetes/pull/105797), [pohly](https://github.com/pohly))
- The Pod `spec.podOS` field is promoted to GA. The `IdentifyPodOS` feature gate unconditionally enabled, and will no longer be accepted as a `--feature-gates` parameter in 1.27. ([kubernetes/kubernetes111229](https://github.com/kubernetes/kubernetes/pull/111229), [ravisantoshgudimetla](https://github.com/ravisantoshgudimetla))
- The PodTopologySpread is respected after rolling upgrades. ([kubernetes/kubernetes111441](https://github.com/kubernetes/kubernetes/pull/111441), [denkensk](https://github.com/denkensk))
- The `CSIInlineVolume` feature has moved from beta to GA. ([kubernetes/kubernetes111258](https://github.com/kubernetes/kubernetes/pull/111258), [dobsonj](https://github.com/dobsonj))
- The `PodSecurity` admission plugin has graduated to GA and is enabled by default. The admission configuration version has been promoted to `pod-security.admission.config.k8s.io/v1`. ([kubernetes/kubernetes110459](https://github.com/kubernetes/kubernetes/pull/110459), [wangyysde](https://github.com/wangyysde))
- The `endPort` field in Network Policy is now promoted to GA
Network Policy providers that support `endPort` field now can use it to specify a range of ports to apply a Network Policy.
Previously, each Network Policy could only target a single port.
Please be aware that `endPort` field MUST BE SUPPORTED by the Network Policy provider. In case your provider does not support `endPort` and this field is specified in a Network Policy, the Network Policy will be created covering only the port field (single port). ([kubernetes/kubernetes110868](https://github.com/kubernetes/kubernetes/pull/110868), [rikatz](https://github.com/rikatz))
- The `metadata.clusterName` field is completely removed. This should not have any user-visible impact. ([kubernetes/kubernetes109602](https://github.com/kubernetes/kubernetes/pull/109602), [lavalamp](https://github.com/lavalamp))
- The `minDomains` field in Pod Topology Spread is graduated to beta ([kubernetes/kubernetes110388](https://github.com/kubernetes/kubernetes/pull/110388), [sanposhiho](https://github.com/sanposhiho)) [SIG API Machinery and Apps]
- The command line flag `enable-taint-manager` for kube-controller-manager is deprecated and will be removed in 1.26. The feature that it supports, taint based eviction, is enabled by default and will continue to be implicitly enabled when the flag is removed. ([kubernetes/kubernetes111411](https://github.com/kubernetes/kubernetes/pull/111411), [alculquicondor](https://github.com/alculquicondor))
- This release added support for `NodeExpandSecret` for CSI driver client which enables the CSI drivers to make use of this secret while performing node expansion operation based on the user request. Previously there was no secret provided as part of the `nodeexpansion` call, thus CSI drivers did not make use of the same while expanding the volume at the node side. ([kubernetes/kubernetes105963](https://github.com/kubernetes/kubernetes/pull/105963), [zhucan](https://github.com/zhucan))
- [Ephemeral Containers](https://kubernetes.io/docs/concepts/workloads/pods/ephemeral-containers/) are now generally available (GA). The `EphemeralContainers` feature gate is always enabled and should be removed from `--feature-gates` flag on the kube-apiserver and the kubelet command lines. The `EphemeralContainers` feature gate is [deprecated and scheduled for removal](https://kubernetes.io/docs/reference/using-api/deprecation-policy/#deprecation) in a future release. ([kubernetes/kubernetes111402](https://github.com/kubernetes/kubernetes/pull/111402), [verb](https://github.com/verb))
- Introduces support for handling pod failures with respect to the configured pod failure policy rules ([kubernetes/kubernetes111113](https://github.com/kubernetes/kubernetes/pull/111113), [mimowo](https://github.com/mimowo)) [SIG API Machinery, Apps, Auth, Scheduling and Testing]
- NodeIPAM support for multiple ClusterCIDRs (https://github.com/kubernetes/enhancements/issues/2593) introduced as an alpha feature.
Setting feature gate MultiCIDRRangeAllocator=true, determines whether the MultiCIDRRangeAllocator controller can be used, while the kube-controller-manager flag below will pick the active controller.
Enable the MultiCIDRRangeAllocator by setting --cidr-allocator-type=MultiCIDRRangeAllocator flag in kube-controller-manager. ([kubernetes/kubernetes109090](https://github.com/kubernetes/kubernetes/pull/109090), [sarveshr7](https://github.com/sarveshr7)) [SIG API Machinery, Apps, Auth, CLI, Cloud Provider, Instrumentation, Network and Testing]
- The CSIInlineVolume feature has moved from beta to GA. ([kubernetes/kubernetes111258](https://github.com/kubernetes/kubernetes/pull/111258), [dobsonj](https://github.com/dobsonj)) [SIG API Machinery, Apps, Auth, Instrumentation, Storage and Testing]
- Added alpha support for user namespaces in pods phase 1 (KEP 127, feature gate: UserNamespacesSupport) ([kubernetes/kubernetes111090](https://github.com/kubernetes/kubernetes/pull/111090), [rata](https://github.com/rata)) [SIG Apps, Auth, Network, Node, Storage and Testing]
- Adds KMS v2alpha1 support ([kubernetes/kubernetes111126](https://github.com/kubernetes/kubernetes/pull/111126), [aramase](https://github.com/aramase)) [SIG API Machinery, Auth, Instrumentation and Testing]
- As of v1.25, the PodSecurity `restricted` level no longer requires pods that set .spec.os.name="windows" to also set Linux-specific securityContext fields. If a 1.25+ cluster has unsupported [out-of-skew](https://kubernetes.io/releases/version-skew-policy/#kubelet) nodes prior to v1.23 and wants to ensure namespaces enforcing the `restricted` policy continue to require Linux-specific securityContext fields on all pods, ensure a version of the `restricted` prior to v1.25 is selected by labeling the namespace (for example, `pod-security.kubernetes.io/enforce-version: v1.24`) ([kubernetes/kubernetes105919](https://github.com/kubernetes/kubernetes/pull/105919), [ravisantoshgudimetla](https://github.com/ravisantoshgudimetla)) [SIG API Machinery, Apps, Auth, Testing and Windows]
- Changes ownership semantics of PersistentVolume's spec.claimRef from `atomic` to `granular`. ([kubernetes/kubernetes110495](https://github.com/kubernetes/kubernetes/pull/110495), [alexzielenski](https://github.com/alexzielenski)) [SIG API Machinery, Architecture, Auth, CLI, Cloud Provider, Instrumentation and Testing]
- Extends ContainerStatus CRI API to allow runtime response with container resource requests and limits that are in effect.
- UpdateContainerResources CRI API now supports both Linux and Windows.
For details, see KEPs below. ([kubernetes/kubernetes111645](https://github.com/kubernetes/kubernetes/pull/111645), [vinaykul](https://github.com/vinaykul)) [SIG Node]
- For v1.25, Kubernetes will be using golang 1.19, In this PR we update to 1.19rc2 as GA is not yet available. ([kubernetes/kubernetes111254](https://github.com/kubernetes/kubernetes/pull/111254), [dims](https://github.com/dims)) [SIG Apps, Architecture, Auth, Autoscaling, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Network, Node, Release, Scheduling, Storage and Testing]
- Introduce PodHasNetwork condition for pods ([kubernetes/kubernetes111358](https://github.com/kubernetes/kubernetes/pull/111358), [ddebroy](https://github.com/ddebroy)) [SIG Apps, Node and Testing]
- Introduction of the `DisruptionTarget` pod condition type. Its `reason` field indicates the reason for pod termination:
- PreemptionByKubeScheduler (Pod preempted by kube-scheduler)
- DeletionByTaintManager (Pod deleted by taint manager due to NoExecute taint)
- EvictionByEvictionAPI (Pod evicted by Eviction API)
- DeletionByPodGC (an orphaned Pod deleted by PodGC) ([kubernetes/kubernetes110959](https://github.com/kubernetes/kubernetes/pull/110959), [mimowo](https://github.com/mimowo)) [SIG Apps, Auth, Node, Scheduling and Testing]
- Kube-Scheduler ComponentConfig is graduated to GA, `kubescheduler.config.k8s.io/v1` is available now.
Plugin `SelectorSpread` is removed in v1. ([kubernetes/kubernetes110534](https://github.com/kubernetes/kubernetes/pull/110534), [kerthcet](https://github.com/kerthcet)) [SIG API Machinery, Scheduling and Testing]
- Local Storage Capacity Isolation feature is GA in 1.25 release. For systems (rootless) that cannot check root file system, please use kubelet config --local-storage-capacity-isolation=false to disable this feature. Once disabled, pod cannot set local ephemeral storage request/limit, and emptyDir sizeLimit niether. ([kubernetes/kubernetes111513](https://github.com/kubernetes/kubernetes/pull/111513), [jingxu97](https://github.com/jingxu97)) [SIG API Machinery, Node, Scalability and Scheduling]
- PersistentVolumeClaim objects are no longer left with storage class set to `nil` forever, but will be updated retroactively once any StorageClass is set or created as default. ([kubernetes/kubernetes111467](https://github.com/kubernetes/kubernetes/pull/111467), [RomanBednar](https://github.com/RomanBednar)) [SIG Apps, Storage and Testing]
- Promote CronJob's TimeZone support to beta ([kubernetes/kubernetes111435](https://github.com/kubernetes/kubernetes/pull/111435), [soltysh](https://github.com/soltysh)) [SIG API Machinery, Apps and Testing]
- Promote DaemonSet MaxSurge to GA. This means `--feature-gates=DaemonSetUpdateSurge=true` are not needed on kube-apiserver and kube-controller-manager binaries and they'll be removed soon following policy at https://kubernetes.io/docs/reference/using-api/deprecation-policy/#deprecation ([kubernetes/kubernetes111194](https://github.com/kubernetes/kubernetes/pull/111194), [ravisantoshgudimetla](https://github.com/ravisantoshgudimetla)) [SIG Apps]
- Respect PodTopologySpread after rolling upgrades ([kubernetes/kubernetes111441](https://github.com/kubernetes/kubernetes/pull/111441), [denkensk](https://github.com/denkensk)) [SIG API Machinery, Apps, Scheduling and Testing]
- Scheduler: include supported ScoringStrategyType list in error message for NodeResourcesFit plugin ([kubernetes/kubernetes111206](https://github.com/kubernetes/kubernetes/pull/111206), [SataQiu](https://github.com/SataQiu)) [SIG Scheduling]
- The Pod `spec.podOS` field is promoted to GA. The `IdentifyPodOS` feature gate unconditionally enabled, and will no longer be accepted as a `--feature-gates` parameter in 1.27. ([kubernetes/kubernetes111229](https://github.com/kubernetes/kubernetes/pull/111229), [ravisantoshgudimetla](https://github.com/ravisantoshgudimetla)) [SIG API Machinery, Apps and Windows]
- The command line flag `enable-taint-manager` for kube-controller-manager is deprecated and will be removed in 1.26.
The feature that it supports, taint based eviction, is enabled by default and will continue to be implicitly enabled when the flag is removed. ([kubernetes/kubernetes111411](https://github.com/kubernetes/kubernetes/pull/111411), [alculquicondor](https://github.com/alculquicondor)) [SIG API Machinery]
- [Ephemeral Containers](https://kubernetes.io/docs/concepts/workloads/pods/ephemeral-containers/) are now generally available. The `EphemeralContainers` feature gate is always enabled and should be removed from `--feature-gates` flag on the kube-apiserver and the kubelet command lines. The `EphemeralContainers` feature gate is [deprecated and scheduled for removal](https://kubernetes.io/docs/reference/using-api/deprecation-policy/#deprecation) in a future release. ([kubernetes/kubernetes111402](https://github.com/kubernetes/kubernetes/pull/111402), [verb](https://github.com/verb)) [SIG API Machinery, Apps, Node, Storage and Testing]
- Added a new feature gate `CheckpointRestore` to enable support to checkpoint containers. If enabled it is possible to checkpoint a container using the newly kubelet API (/checkpoint/{podNamespace}/{podName}/{containerName}). ([kubernetes/kubernetes104907](https://github.com/kubernetes/kubernetes/pull/104907), [adrianreber](https://github.com/adrianreber)) [SIG Node and Testing]
- EndPort field in Network Policy is now promoted to GA
Network Policy providers that support endPort field now can use it to specify a range of ports to apply a Network Policy.
Previously, each Network Policy could only target a single port.
Please be aware that endPort field MUST BE SUPPORTED by the Network Policy provider. In case your provider does not support endPort and this field is specified in a Network Policy, the Network Policy will be created covering only the port field (single port). ([kubernetes/kubernetes110868](https://github.com/kubernetes/kubernetes/pull/110868), [rikatz](https://github.com/rikatz)) [SIG API Machinery, Network and Testing]
- Make PodSpec.Ports' description clearer on how this information is only informational and how it can be incorrect. ([kubernetes/kubernetes110564](https://github.com/kubernetes/kubernetes/pull/110564), [j4m3s-s](https://github.com/j4m3s-s)) [SIG API Machinery, Network and Node]
- On compatible systems, a mounter's Unmount implementation is changed to not return an error when the specified target can be detected as not a mount point. On Linux, the behavior of detecting a mount point depends on `umount` command is validated when the mounter is created. Additionally, mount point checks will be skipped in CleanupMountPoint/CleanupMountWithForce if the mounter's Unmount having the changed behavior of not returning error when target is not a mount point. ([kubernetes/kubernetes109676](https://github.com/kubernetes/kubernetes/pull/109676), [cartermckinnon](https://github.com/cartermckinnon)) [SIG Storage]
- Promote StatefulSet minReadySeconds to GA. This means `--feature-gates=StatefulSetMinReadySeconds=true` are not needed on kube-apiserver and kube-controller-manager binaries and they'll be removed soon following policy at https://kubernetes.io/docs/reference/using-api/deprecation-policy/#deprecation ([kubernetes/kubernetes110896](https://github.com/kubernetes/kubernetes/pull/110896), [ravisantoshgudimetla](https://github.com/ravisantoshgudimetla)) [SIG API Machinery, Apps and Testing]
- The Pod `spec.podOS` field is promoted to GA. The `IdentifyPodOS` feature gate unconditionally enabled, and will no longer be accepted as a `--feature-gates` parameter in 1.27. ([kubernetes/kubernetes111229](https://github.com/kubernetes/kubernetes/pull/111229), [ravisantoshgudimetla](https://github.com/ravisantoshgudimetla)) [SIG API Machinery, Apps and Windows]
- The `minDomains` field in Pod Topology Spread is graduated to beta ([kubernetes/kubernetes110388](https://github.com/kubernetes/kubernetes/pull/110388), [sanposhiho](https://github.com/sanposhiho)) [SIG API Machinery and Apps]
- The Go API for logging configuration in k8s.io/component-base was moved to k8s.io/component-base/logs/api/v1. The configuration file format and command line flags are the same as before. ([kubernetes/kubernetes105797](https://github.com/kubernetes/kubernetes/pull/105797), [pohly](https://github.com/pohly)) [SIG API Machinery, Architecture, Cluster Lifecycle, Instrumentation, Node, Scheduling and Testing]
- The PodSecurity admission plugin has graduated to GA and is enabled by default. The admission configuration version has been promoted to `pod-security.admission.config.k8s.io/v1`. ([kubernetes/kubernetes110459](https://github.com/kubernetes/kubernetes/pull/110459), [wangyysde](https://github.com/wangyysde)) [SIG API Machinery, Architecture, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Node, Storage and Testing]
- Introduce NodeInclusionPolicies to specify nodeAffinity/nodeTaint strategy when calculating pod topology spread skew. ([kubernetes/kubernetes108492](https://github.com/kubernetes/kubernetes/pull/108492), [kerthcet](https://github.com/kerthcet)) [SIG API Machinery, Apps, Scheduling and Testing]
- The `metadata.clusterName` field is completely removed. This should not have any user-visible impact. ([kubernetes/kubernetes109602](https://github.com/kubernetes/kubernetes/pull/109602), [lavalamp](https://github.com/lavalamp)) [SIG API Machinery, Apps, Auth and Testing]
- This release add support for NodeExpandSecret for CSI driver client which enables the CSI drivers to make use of this secret while performing node expansion operation based on the user request. Previously there was no secret provided as part of the nodeexpansion call, thus CSI drivers were not make use of the same while expanding the volume at node side. ([kubernetes/kubernetes105963](https://github.com/kubernetes/kubernetes/pull/105963), [zhucan](https://github.com/zhucan)) [SIG API Machinery, Apps and Storage]

Uncategorized
- The dynamic client now support the `_request_timeout` parameter to configure connection and request timeouts. (1732, philipp-sontag-by)

API Change
- Add 2 new options for kube-proxy running in winkernel mode. `--forward-healthcheck-vip`, if specified as true, health check traffic whose destination is service VIP will be forwarded to kube-proxy's healthcheck service. `--root-hnsendpoint-name` specifies the name of the hns endpoint for the root network namespace. This option enables the pass-through load balancers like Google's GCLB to correctly health check the backend services. Without this change, the health check packets is dropped, and Windows node will be considered to be unhealthy by those load balancers. ([kubernetes/kubernetes99287](https://github.com/kubernetes/kubernetes/pull/99287), [anfernee](https://github.com/anfernee))
- Added CEL runtime cost calculation into CustomerResource validation. CustomerResource validation will fail if runtime cost exceeds the budget. ([kubernetes/kubernetes108482](https://github.com/kubernetes/kubernetes/pull/108482), [cici37](https://github.com/cici37))
- Added a new metric `webhook_fail_open_count` to monitor webhooks that fail to open. ([kubernetes/kubernetes107171](https://github.com/kubernetes/kubernetes/pull/107171), [ltagliamonte-dd](https://github.com/ltagliamonte-dd))
- Adds a new Status subresource in Network Policy objects ([kubernetes/kubernetes107963](https://github.com/kubernetes/kubernetes/pull/107963), [rikatz](https://github.com/rikatz))
- Adds support for `InterfaceNamePrefix` and `BridgeInterface` as arguments to `--detect-local-mode` option and also introduces a new optional `--pod-interface-name-prefix` and `--pod-bridge-interface` flags to kube-proxy. ([kubernetes/kubernetes95400](https://github.com/kubernetes/kubernetes/pull/95400), [tssurya](https://github.com/tssurya))
- CEL CRD validation expressions may now reference existing object state using the identifier `oldSelf`. ([kubernetes/kubernetes108073](https://github.com/kubernetes/kubernetes/pull/108073), [benluddy](https://github.com/benluddy))
- CRD deep copies should no longer contain shallow copies of `JSONSchemaProps.XValidations`. ([kubernetes/kubernetes107956](https://github.com/kubernetes/kubernetes/pull/107956), [benluddy](https://github.com/benluddy))
- CRD writes will generate validation errors if a CEL validation rule references the identifier `oldSelf` on a part of the schema that does not support it. ([kubernetes/kubernetes108013](https://github.com/kubernetes/kubernetes/pull/108013), [benluddy](https://github.com/benluddy))
- CSIStorageCapacity.storage.k8s.io: The v1beta1 version of this API is deprecated in favor of v1, and will be removed in v1.27. If a CSI driver supports storage capacity tracking, then it must get deployed with a release of external-provisioner that supports the v1 API. ([kubernetes/kubernetes108445](https://github.com/kubernetes/kubernetes/pull/108445), [pohly](https://github.com/pohly))
- Custom resource requests with `fieldValidation=Strict` consistently require `apiVersion` and `kind`, matching non-strict requests ([kubernetes/kubernetes109019](https://github.com/kubernetes/kubernetes/pull/109019), [liggitt](https://github.com/liggitt))
- Feature of `DefaultPodTopologySpread` is graduated to GA ([kubernetes/kubernetes108278](https://github.com/kubernetes/kubernetes/pull/108278), [kerthcet](https://github.com/kerthcet))
- Feature of `NonPreemptingPriority` is graduated to GA ([kubernetes/kubernetes107432](https://github.com/kubernetes/kubernetes/pull/107432), [denkensk](https://github.com/denkensk))
- Feature of `PodOverhead` is graduated to GA ([kubernetes/kubernetes108441](https://github.com/kubernetes/kubernetes/pull/108441), [pacoxu](https://github.com/pacoxu))
- Fixed OpenAPI serialization of the x-kubernetes-validations field ([kubernetes/kubernetes107970](https://github.com/kubernetes/kubernetes/pull/107970), [liggitt](https://github.com/liggitt))
- Fixed failed flushing logs in defer function when kubelet cmd exit 1. ([kubernetes/kubernetes104774](https://github.com/kubernetes/kubernetes/pull/104774), [kerthcet](https://github.com/kerthcet))
- Fixes a regression in v1beta1 PodDisruptionBudget handling of `strategic merge patch`-type API requests for the `selector` field. Prior to 1.21, these requests would merge `matchLabels` content and replace `matchExpressions` content. In 1.21, patch requests touching the `selector` field started replacing the entire selector. This is consistent with server-side apply and the v1 PodDisruptionBudget behavior, but should not have been changed for v1beta1. ([kubernetes/kubernetes108138](https://github.com/kubernetes/kubernetes/pull/108138), [liggitt](https://github.com/liggitt))
- Improve kubectl's user help commands readability ([kubernetes/kubernetes104736](https://github.com/kubernetes/kubernetes/pull/104736), [lauchokyip](https://github.com/lauchokyip))
- Indexed Jobs graduated to stable. ([kubernetes/kubernetes107395](https://github.com/kubernetes/kubernetes/pull/107395), [alculquicondor](https://github.com/alculquicondor))
- Introduce a v1alpha1 networking API for ClusterCIDRConfig ([kubernetes/kubernetes108290](https://github.com/kubernetes/kubernetes/pull/108290), [sarveshr7](https://github.com/sarveshr7))
- Introduction of a new "sync_proxy_rules_no_local_endpoints_total" proxy metric. This metric represents the number of services with no internal endpoints. The "traffic_policy" label will contain both "internal" or "external". ([kubernetes/kubernetes108930](https://github.com/kubernetes/kubernetes/pull/108930), [MaxRenaud](https://github.com/MaxRenaud))
- JobReadyPods graduates to Beta and it's enabled by default. ([kubernetes/kubernetes107476](https://github.com/kubernetes/kubernetes/pull/107476), [alculquicondor](https://github.com/alculquicondor))
- Kube-apiserver: `--audit-log-version` and `--audit-webhook-version` now only support the default value of `audit.k8s.io/v1`. The v1alpha1 and v1beta1 audit log versions, deprecated since 1.13, have been removed. ([kubernetes/kubernetes108092](https://github.com/kubernetes/kubernetes/pull/108092), [carlory](https://github.com/carlory))
- Kube-apiserver: the `metadata.selfLink` field can no longer be populated by kube-apiserver; it was deprecated in 1.16 and has not been populated by default since 1.20+. ([kubernetes/kubernetes107527](https://github.com/kubernetes/kubernetes/pull/107527), [wojtek-t](https://github.com/wojtek-t))
- Kubelet external Credential Provider feature is moved to Beta. Credential Provider Plugin and Credential Provider Config API's updated from v1alpha1 to v1beta1 with no API changes. ([kubernetes/kubernetes108847](https://github.com/kubernetes/kubernetes/pull/108847), [adisky](https://github.com/adisky))
- Make STS available replicas optional again. ([kubernetes/kubernetes109241](https://github.com/kubernetes/kubernetes/pull/109241), [ravisantoshgudimetla](https://github.com/ravisantoshgudimetla))
- MaxUnavailable for StatefulSets, allows faster RollingUpdate by taking down more than 1 pod at a time. The number of pods you want to take down during a RollingUpdate is configurable using maxUnavailable parameter. ([kubernetes/kubernetes82162](https://github.com/kubernetes/kubernetes/pull/82162), [krmayankk](https://github.com/krmayankk))
- Non-graceful node shutdown handling is enabled for stateful workload failovers ([kubernetes/kubernetes108486](https://github.com/kubernetes/kubernetes/pull/108486), [sonasingh46](https://github.com/sonasingh46))
- Omit enum declarations from the static openapi file captured at https://git.k8s.io/kubernetes/api/openapi-spec. This file is used to generate API clients, and use of enums in those generated clients (rather than strings) can break forward compatibility with additional future values in those fields. See https://issue.k8s.io/109177 for details. ([kubernetes/kubernetes#109178](https://github.com/kubernetes/kubernetes/pull/109178), [liggitt](https://github.com/liggitt))
- OpenAPI V3 is turned on by default ([kubernetes/kubernetes109031](https://github.com/kubernetes/kubernetes/pull/109031), [Jefftree](https://github.com/Jefftree))
- Pod affinity namespace selector and cross-namespace quota graduated to GA. The feature gate `PodAffinityNamespaceSelector` is locked and will be removed in 1.26. ([kubernetes/kubernetes108136](https://github.com/kubernetes/kubernetes/pull/108136), [ahg-g](https://github.com/ahg-g))
- Promote IdentifyPodOS feature to beta. ([kubernetes/kubernetes107859](https://github.com/kubernetes/kubernetes/pull/107859), [ravisantoshgudimetla](https://github.com/ravisantoshgudimetla))
- Remove a v1alpha1 networking API for ClusterCIDRConfig ([kubernetes/kubernetes109436](https://github.com/kubernetes/kubernetes/pull/109436), [JamesLaverack](https://github.com/JamesLaverack))
- Renamed metrics `evictions_number` to `evictions_total` and mark it as stable. The original `evictions_number` metrics name is marked as "Deprecated" and has been removed in kubernetes 1.23 . ([kubernetes/kubernetes106366](https://github.com/kubernetes/kubernetes/pull/106366), [cyclinder](https://github.com/cyclinder))
- Skip x-kubernetes-validations rules if having fundamental error against the OpenAPIv3 schema. ([kubernetes/kubernetes108859](https://github.com/kubernetes/kubernetes/pull/108859), [cici37](https://github.com/cici37))
- Support for gRPC probes is now in beta. GRPCContainerProbe feature gate is enabled by default. ([kubernetes/kubernetes108522](https://github.com/kubernetes/kubernetes/pull/108522), [SergeyKanzhelev](https://github.com/SergeyKanzhelev))
- Suspend job to GA. The feature gate `SuspendJob` is locked and will be removed in 1.26. ([kubernetes/kubernetes108129](https://github.com/kubernetes/kubernetes/pull/108129), [ahg-g](https://github.com/ahg-g))
- The AnyVolumeDataSource feature is now beta, and the feature gate is enabled by default. In order to provide user feedback on PVCs with data sources, deployers must install the VolumePopulators CRD and the data-source-validator controller. ([kubernetes/kubernetes108736](https://github.com/kubernetes/kubernetes/pull/108736), [bswartz](https://github.com/bswartz))
- The CertificateSigningRequest `spec.expirationSeconds` API field has graduated to GA. The `CSRDuration` feature gate for the field is now unconditionally enabled and will be removed in 1.26. ([kubernetes/kubernetes108782](https://github.com/kubernetes/kubernetes/pull/108782), [cfryanr](https://github.com/cfryanr))
- The `ServerSideFieldValidation` feature has graduated to beta and is now enabled by default. Kubectl 1.24 and newer will use server-side validation instead of client-side validation when writing to API servers with the feature enabled. ([kubernetes/kubernetes108889](https://github.com/kubernetes/kubernetes/pull/108889), [kevindelgado](https://github.com/kevindelgado))
- The `ServiceLBNodePortControl` feature has graduated to GA. The feature gate will be removed in 1.26. ([kubernetes/kubernetes107027](https://github.com/kubernetes/kubernetes/pull/107027), [uablrek](https://github.com/uablrek))
- The deprecated kube-controller-manager flag '--deployment-controller-sync-period' has been removed, it is not used by the deployment controller. ([kubernetes/kubernetes107178](https://github.com/kubernetes/kubernetes/pull/107178), [SataQiu](https://github.com/SataQiu))
- The feature `DynamicKubeletConfig` has been removed from the kubelet. ([kubernetes/kubernetes106932](https://github.com/kubernetes/kubernetes/pull/106932), [SergeyKanzhelev](https://github.com/SergeyKanzhelev))
- The infrastructure for contextual logging is complete (feature gate implemented, JSON backend ready). ([kubernetes/kubernetes108995](https://github.com/kubernetes/kubernetes/pull/108995), [pohly](https://github.com/pohly))
- This adds an optional `timeZone` field as part of the CronJob spec to support running cron jobs in a specific time zone. ([kubernetes/kubernetes108032](https://github.com/kubernetes/kubernetes/pull/108032), [deejross](https://github.com/deejross))
- Updated the default API priority-and-fairness config to avoid endpoint/configmaps operations from controller-manager to all match leader-election priority level. ([kubernetes/kubernetes106725](https://github.com/kubernetes/kubernetes/pull/106725), [wojtek-t](https://github.com/wojtek-t))
- `topologySpreadConstraints` includes `minDomains` field to limit the minimum number of topology domains. ([kubernetes/kubernetes107674](https://github.com/kubernetes/kubernetes/pull/107674), [sanposhiho](https://github.com/sanposhiho))
- Introduce a v1alpha1 networking API for ClusterCIDRConfig ([kubernetes/kubernetes108290](https://github.com/kubernetes/kubernetes/pull/108290), [sarveshr7](https://github.com/sarveshr7)) [SIG API Machinery, Apps, Auth, CLI, Cloud Provider, Instrumentation, Network and Testing]
- Introduction of a new "sync_proxy_rules_no_local_endpoints_total" proxy metric. This metric represents the number of services with no internal endpoints. The "traffic_policy" label will contain both "internal" or "external". ([kubernetes/kubernetes108930](https://github.com/kubernetes/kubernetes/pull/108930), [MaxRenaud](https://github.com/MaxRenaud)) [SIG API Machinery, Apps, Architecture, Auth, Autoscaling, CLI, Cloud Provider, Instrumentation, Network, Node, Release, Scheduling, Storage, Testing and Windows]
- Make STS available replicas optional again, ([kubernetes/kubernetes109241](https://github.com/kubernetes/kubernetes/pull/109241), [ravisantoshgudimetla](https://github.com/ravisantoshgudimetla)) [SIG API Machinery and Apps]
- Omit enum declarations from the static openapi file captured at https://git.k8s.io/kubernetes/api/openapi-spec. This file is used to generate API clients, and use of enums in those generated clients (rather than strings) can break forward compatibility with additional future values in those fields. See https://issue.k8s.io/109177 for details. ([kubernetes/kubernetes#109178](https://github.com/kubernetes/kubernetes/pull/109178), [liggitt](https://github.com/liggitt)) [SIG API Machinery and Auth]
- Remove a v1alpha1 networking API for ClusterCIDRConfig ([kubernetes/kubernetes109436](https://github.com/kubernetes/kubernetes/pull/109436), [JamesLaverack](https://github.com/JamesLaverack)) [SIG API Machinery, Apps, Auth, CLI, Network and Testing]
- The deprecated kube-controller-manager flag '--deployment-controller-sync-period' has been removed, it is not used by the deployment controller. ([kubernetes/kubernetes107178](https://github.com/kubernetes/kubernetes/pull/107178), [SataQiu](https://github.com/SataQiu)) [SIG API Machinery and Apps]
- Adds a new Status subresource in Network Policy objects ([kubernetes/kubernetes107963](https://github.com/kubernetes/kubernetes/pull/107963), [rikatz](https://github.com/rikatz)) [SIG API Machinery, Apps, Network and Testing]
- Adds support for "InterfaceNamePrefix" and "BridgeInterface" as arguments to --detect-local-mode option and also introduces a new optional `--pod-interface-name-prefix` and `--pod-bridge-interface` flags to kube-proxy. ([kubernetes/kubernetes95400](https://github.com/kubernetes/kubernetes/pull/95400), [tssurya](https://github.com/tssurya)) [SIG API Machinery and Network]
- CEL CRD validation expressions may now reference existing object state using the identifier `oldSelf`. ([kubernetes/kubernetes108073](https://github.com/kubernetes/kubernetes/pull/108073), [benluddy](https://github.com/benluddy)) [SIG API Machinery and Testing]
- CSIStorageCapacity.storage.k8s.io: The v1beta1 version of this API is deprecated in favor of v1, and will be removed in v1.27. If a CSI driver supports storage capacity tracking, then it must get deployed with a release of external-provisioner that supports the v1 API. ([kubernetes/kubernetes108445](https://github.com/kubernetes/kubernetes/pull/108445), [pohly](https://github.com/pohly)) [SIG API Machinery, Architecture, Auth, Scheduling, Storage and Testing]
- Custom resource requests with fieldValidation=Strict consistently require apiVersion and kind, matching non-strict requests ([kubernetes/kubernetes109019](https://github.com/kubernetes/kubernetes/pull/109019), [liggitt](https://github.com/liggitt)) [SIG API Machinery]
- Improve kubectl's user help commands readability ([kubernetes/kubernetes104736](https://github.com/kubernetes/kubernetes/pull/104736), [lauchokyip](https://github.com/lauchokyip)) [SIG API Machinery, Apps, Architecture, Auth, Autoscaling, CLI, Cloud Provider, Cluster Lifecycle, Contributor Experience, Instrumentation, Network, Node, Release, Scalability, Scheduling, Security, Storage, Testing and Windows]
- Indexed Jobs graduates to stable ([kubernetes/kubernetes107395](https://github.com/kubernetes/kubernetes/pull/107395), [alculquicondor](https://github.com/alculquicondor)) [SIG Apps, Architecture and Testing]
- Introduce a v1alpha1 networking API for ClusterCIDRConfig ([kubernetes/kubernetes108290](https://github.com/kubernetes/kubernetes/pull/108290), [sarveshr7](https://github.com/sarveshr7)) [SIG API Machinery, Apps, Auth, CLI, Cloud Provider, Instrumentation, Network and Testing]
- JobReadyPods graduates to Beta and it's enabled by default. ([kubernetes/kubernetes107476](https://github.com/kubernetes/kubernetes/pull/107476), [alculquicondor](https://github.com/alculquicondor)) [SIG API Machinery, Apps and Testing]
- Kubelet external Credential Provider feature is moved to Beta. Credential Provider Plugin and Credential Provider Config API's updated from v1alpha1 to v1beta1 with no API changes. ([kubernetes/kubernetes108847](https://github.com/kubernetes/kubernetes/pull/108847), [adisky](https://github.com/adisky)) [SIG API Machinery and Node]
- MaxUnavailable for StatefulSets, allows faster RollingUpdate by taking down more than 1 pod at a time. The number of pods you want to take down during a RollingUpdate is configurable using maxUnavailable parameter. ([kubernetes/kubernetes82162](https://github.com/kubernetes/kubernetes/pull/82162), [krmayankk](https://github.com/krmayankk)) [SIG API Machinery and Apps]
- Non graceful node shutdown handling. ([kubernetes/kubernetes108486](https://github.com/kubernetes/kubernetes/pull/108486), [sonasingh46](https://github.com/sonasingh46)) [SIG Apps, Node and Storage]
- OpenAPI V3 is turned on by default ([kubernetes/kubernetes109031](https://github.com/kubernetes/kubernetes/pull/109031), [Jefftree](https://github.com/Jefftree)) [SIG API Machinery, Apps, Architecture, Auth, Autoscaling, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Network, Node, Scheduling, Storage and Testing]
- Promote IdentifyPodOS feature to beta. ([kubernetes/kubernetes107859](https://github.com/kubernetes/kubernetes/pull/107859), [ravisantoshgudimetla](https://github.com/ravisantoshgudimetla)) [SIG API Machinery, Apps, Node, Testing and Windows]
- Skip x-kubernetes-validations rules if having fundamental error against OpenAPIv3 schema. ([kubernetes/kubernetes108859](https://github.com/kubernetes/kubernetes/pull/108859), [cici37](https://github.com/cici37)) [SIG API Machinery and Testing]
- Support for gRPC probes is now in beta. GRPCContainerProbe feature gate is enabled by default. ([kubernetes/kubernetes108522](https://github.com/kubernetes/kubernetes/pull/108522), [SergeyKanzhelev](https://github.com/SergeyKanzhelev)) [SIG API Machinery, Apps, Node and Testing]
- The AnyVolumeDataSource feature is now beta, and the feature gate is enabled by default. In order to provide user feedback on PVCs with data sources, deployers must install the VolumePopulators CRD and the data-source-validator controller. ([kubernetes/kubernetes108736](https://github.com/kubernetes/kubernetes/pull/108736), [bswartz](https://github.com/bswartz)) [SIG Apps, Storage and Testing]
- The `ServerSideFieldValidation` feature has graduated to beta and is now enabled by default. Kubectl 1.24 and newer will use server-side validation instead of client-side validation when writing to API servers with the feature enabled. ([kubernetes/kubernetes108889](https://github.com/kubernetes/kubernetes/pull/108889), [kevindelgado](https://github.com/kevindelgado)) [SIG API Machinery, Architecture, CLI and Testing]
- The infrastructure for contextual logging is complete (feature gate implemented, JSON backend ready). ([kubernetes/kubernetes108995](https://github.com/kubernetes/kubernetes/pull/108995), [pohly](https://github.com/pohly)) [SIG API Machinery, Architecture, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Network, Node, Scheduling and Testing]
- This adds an optional `timeZone` field as part of the CronJob spec to support running cron jobs in a specific time zone. ([kubernetes/kubernetes108032](https://github.com/kubernetes/kubernetes/pull/108032), [deejross](https://github.com/deejross)) [SIG API Machinery and Apps]
- Add 2 new options for kube-proxy running in winkernel mode.
`--forward-healthcheck-vip`, if specified as true, health check traffic whose destination is service VIP will be forwarded to kube-proxy's healthcheck service. `--root-hnsendpoint-name` specifies the name of the hns endpoint for the root network namespace.
This option enables the pass-through load balancers like Google's GCLB to correctly health check the backend services. Without this change, the health check packets is dropped, and Windows node will be considered to be unhealthy by those load balancers. ([kubernetes/kubernetes99287](https://github.com/kubernetes/kubernetes/pull/99287), [anfernee](https://github.com/anfernee)) [SIG API Machinery, Cloud Provider, Network, Testing and Windows]
- Added CEL runtime cost calculation into CustomerResource validation. CustomerResource validation will fail if runtime cost exceeds the budget. ([kubernetes/kubernetes108482](https://github.com/kubernetes/kubernetes/pull/108482), [cici37](https://github.com/cici37)) [SIG API Machinery]
- CRD writes will generate validation errors if a CEL validation rule references the identifier "oldSelf" on a part of the schema that does not support it. ([kubernetes/kubernetes108013](https://github.com/kubernetes/kubernetes/pull/108013), [benluddy](https://github.com/benluddy)) [SIG API Machinery]
- Feature of `DefaultPodTopologySpread` is graduated to GA ([kubernetes/kubernetes108278](https://github.com/kubernetes/kubernetes/pull/108278), [kerthcet](https://github.com/kerthcet)) [SIG Scheduling]
- Feature of `PodOverhead` is graduated to GA ([kubernetes/kubernetes108441](https://github.com/kubernetes/kubernetes/pull/108441), [pacoxu](https://github.com/pacoxu)) [SIG API Machinery, Apps, Node and Scheduling]
- Fixes a regression in v1beta1 PodDisruptionBudget handling of "strategic merge patch"-type API requests for the `selector` field. Prior to 1.21, these requests would merge `matchLabels` content and replace `matchExpressions` content. In 1.21, patch requests touching the `selector` field started replacing the entire selector. This is consistent with server-side apply and the v1 PodDisruptionBudget behavior, but should not have been changed for v1beta1. ([kubernetes/kubernetes108138](https://github.com/kubernetes/kubernetes/pull/108138), [liggitt](https://github.com/liggitt)) [SIG Apps, Auth and Testing]
- Kube-apiserver: --audit-log-version and --audit-webhook-version now only support the default value of audit.k8s.io/v1. The v1alpha1 and v1beta1 audit log versions, deprecated since 1.13, have been removed. ([kubernetes/kubernetes108092](https://github.com/kubernetes/kubernetes/pull/108092), [carlory](https://github.com/carlory)) [SIG API Machinery, Auth and Testing]
- Pod-affinity namespace selector and cross-namespace quota graduated to GA. The feature gate PodAffinityNamespaceSelector is locked and will be removed in 1.26. ([kubernetes/kubernetes108136](https://github.com/kubernetes/kubernetes/pull/108136), [ahg-g](https://github.com/ahg-g)) [SIG API Machinery, Apps, Scheduling and Testing]
- Suspend job to GA. The feature gate SuspendJob is locked and will be removed in 1.26. ([kubernetes/kubernetes108129](https://github.com/kubernetes/kubernetes/pull/108129), [ahg-g](https://github.com/ahg-g)) [SIG Apps and Testing]
- The CertificateSigningRequest `spec.expirationSeconds` API field has graduated to GA. The `CSRDuration` feature gate for the field is now unconditionally enabled and will be removed in 1.26. ([kubernetes/kubernetes108782](https://github.com/kubernetes/kubernetes/pull/108782), [cfryanr](https://github.com/cfryanr)) [SIG API Machinery, Apps, Auth, Instrumentation and Testing]
- TopologySpreadConstraints includes minDomains field to limit the minimum number of topology domains. ([kubernetes/kubernetes107674](https://github.com/kubernetes/kubernetes/pull/107674), [sanposhiho](https://github.com/sanposhiho)) [SIG API Machinery, Apps and Scheduling]
- CRD deep copies should no longer contain shallow copies of JSONSchemaProps.XValidations. ([kubernetes/kubernetes107956](https://github.com/kubernetes/kubernetes/pull/107956), [benluddy](https://github.com/benluddy)) [SIG API Machinery]
- Feature of `NonPreemptingPriority` is graduated to GA ([kubernetes/kubernetes107432](https://github.com/kubernetes/kubernetes/pull/107432), [denkensk](https://github.com/denkensk)) [SIG Apps, Scheduling and Testing]
- Fix OpenAPI serialization of the x-kubernetes-validations field ([kubernetes/kubernetes107970](https://github.com/kubernetes/kubernetes/pull/107970), [liggitt](https://github.com/liggitt)) [SIG API Machinery]
- Kube-apiserver: the `metadata.selfLink` field can no longer be populated by kube-apiserver; it was deprecated in 1.16 and has not been populated by default in 1.20+. ([kubernetes/kubernetes107527](https://github.com/kubernetes/kubernetes/pull/107527), [wojtek-t](https://github.com/wojtek-t)) [SIG API Machinery, Apps, Auth, Autoscaling, CLI, Cloud Provider, Network, Scheduling, Storage and Testing]
- Add a new metric `webhook_fail_open_count` to monitor webhooks that fail open ([kubernetes/kubernetes107171](https://github.com/kubernetes/kubernetes/pull/107171), [ltagliamonte-dd](https://github.com/ltagliamonte-dd)) [SIG API Machinery and Instrumentation]
- Fix failed flushing logs in defer function when kubelet cmd exit 1. ([kubernetes/kubernetes104774](https://github.com/kubernetes/kubernetes/pull/104774), [kerthcet](https://github.com/kerthcet)) [SIG Node and Scheduling]
- Rename metrics `evictions_number` to `evictions_total` and mark it as stable. The original `evictions_number` metrics name is marked as "Deprecated" and will be removed in kubernetes 1.23 ([kubernetes/kubernetes106366](https://github.com/kubernetes/kubernetes/pull/106366), [cyclinder](https://github.com/cyclinder)) [SIG API Machinery, Apps, Architecture, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Network, Node, Release, Scalability, Scheduling, Storage, Testing and Windows]
- The `ServiceLBNodePortControl` feature graduates to GA. The feature gate will be removed in 1.26. ([kubernetes/kubernetes107027](https://github.com/kubernetes/kubernetes/pull/107027), [uablrek](https://github.com/uablrek)) [SIG Network and Testing]
- The feature DynamicKubeletConfig is removed from the kubelet. ([kubernetes/kubernetes106932](https://github.com/kubernetes/kubernetes/pull/106932), [SergeyKanzhelev](https://github.com/SergeyKanzhelev)) [SIG Apps, Auth, Instrumentation, Node and Testing]
- Update default API priority-and-fairness config to avoid endpoint/configmaps operations from controller-manager to all match leader-election priority level. ([kubernetes/kubernetes106725](https://github.com/kubernetes/kubernetes/pull/106725), [wojtek-t](https://github.com/wojtek-t)) [SIG API Machinery]

API Change
- Omits alpha-level enums from the static openapi file captured in api/openapi-spec ([kubernetes/kubernetes109179](https://github.com/kubernetes/kubernetes/pull/109179), [liggitt](https://github.com/liggitt)) [SIG Apps and Auth]
- Fixes a regression in v1beta1 PodDisruptionBudget handling of "strategic merge patch"-type API requests for the `selector` field. Prior to 1.21, these requests would merge `matchLabels` content and replace `matchExpressions` content. In 1.21, patch requests touching the `selector` field started replacing the entire selector. This is consistent with server-side apply and the v1 PodDisruptionBudget behavior, but should not have been changed for v1beta1. ([kubernetes/kubernetes108139](https://github.com/kubernetes/kubernetes/pull/108139), [liggitt](https://github.com/liggitt)) [SIG Auth and Testing]