[gardener]

🐛 Bug Fixes

• [OPERATOR] Node names are always lower case. (gardener/gardener#5578, @stoyanr)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.42.2
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.42.2
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.42.2
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.42.2
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.42.2
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.42.2
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.42.2

[gardener]

🐛 Bug Fixes

• [OPERATOR] Node names are always lower case. (gardener/gardener#5579, @stoyanr)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.41.5
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.41.5
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.41.5
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.41.5
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.41.5
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.41.5
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.41.5

[gardener]

🐛 Bug Fixes

• [OPERATOR] Node names are always lower case. (gardener/gardener#5580, @stoyanr)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.40.4
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.40.4
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.40.4
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.40.4
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.40.4
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.40.4
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.40.4

[gardener]

🐛 Bug Fixes

• [USER] A race condition has been fixed which can lead to pods without any projected token volumes for newly created shoots. (gardener/gardener#5571, @rfranzke)
• [USER] A bug causing shoot reconciliations or deletions to fail with "no matches for kind" errors has been fixed. (gardener/gardener#5568, @rfranzke)
• [OPERATOR] An issue has been fixed leading to shoot namespaces in the seed blocking deletion due to referenced objects with finalizers. (gardener/gardener#5564, @rfranzke)
• [OPERATOR] An issue causing Shoot deletion to fail in a rare case when the corresponding Shoot Namespace in the Seed is already terminating is now fixed. (gardener/gardener#5559, @ialidzhikov)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.41.4
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.41.4
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.41.4
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.41.4
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.41.4
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.41.4
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.41.4

[gardener]

🐛 Bug Fixes

• [USER] A race condition has been fixed which can lead to pods without any projected token volumes for newly created shoots. (gardener/gardener#5570, @rfranzke)
• [USER] A bug causing shoot reconciliations or deletions to fail with "no matches for kind" errors has been fixed. (gardener/gardener#5567, @rfranzke)
• [OPERATOR] An issue has been fixed leading to shoot namespaces in the seed blocking deletion due to referenced objects with finalizers. (gardener/gardener#5563, @rfranzke)
• [OPERATOR] An issue causing Shoot deletion to fail in a rare case when the corresponding Shoot Namespace in the Seed is already terminating is now fixed. (gardener/gardener#5558, @ialidzhikov)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.42.1
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.42.1
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.42.1
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.42.1
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.42.1
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.42.1
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.42.1

[gardener]

🐛 Bug Fixes

• [USER] A race condition has been fixed which can lead to pods without any projected token volumes for newly created shoots. (gardener/gardener#5572, @rfranzke)
• [USER] A bug causing shoot reconciliations or deletions to fail with "no matches for kind" errors has been fixed. (gardener/gardener#5569, @rfranzke)
• [OPERATOR] An issue has been fixed leading to shoot namespaces in the seed blocking deletion due to referenced objects with finalizers. (gardener/gardener#5565, @rfranzke)
• [OPERATOR] An issue causing Shoot deletion to fail in a rare case when the corresponding Shoot Namespace in the Seed is already terminating is now fixed. (gardener/gardener#5560, @ialidzhikov)
• [OPERATOR] An error has been fixed that sporadically occurred during shoot deletion and was related to volumesnapshots, volumesnapshotcontents not being found. (gardener/gardener#5491, @timuthy)
• [OPERATOR] An issue preventing the nginx-ingress addon to be disabled is now fixed. (gardener/gardener#5485, @ialidzhikov)

🏃 Others

• [OPERATOR] A bug has been fixed causing gardenlet to panic when a Shoot with multiple worker pools specifies eviction settings for the kubelet configuration in .spec.kubernetes.kubelet while .spec.provider.workers[].kubernetes.kubelet=nil. (gardener/gardener#5518, @rfranzke)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.40.3
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.40.3
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.40.3
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.40.3
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.40.3
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.40.3
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.40.3

[gardener]

⚠️ Breaking Changes

• [OPERATOR] It is no longer possible to disable the audit policy configmap protection of the Shoot reference controller of gardener-controller-manager. The audit policy configmap protection is enabled by default since v1.41 and was mainly introduced for backwards-compatibility reasons related to the Shoot deletion. (gardener/gardener#5525, @ialidzhikov)
• [OPERATOR] The AdminKubeconfigRequest feature gate in the gardener-apiserver has been promoted to GA. (gardener/gardener#5511, @acumino)
• [OPERATOR] The deprecated error code ERR_INFRA_INSUFFICIENT_PRIVILEGES is removed in favour of ERR_INFRA_UNAUTHORIZED. (gardener/gardener#5501, @shafeeqes)
• [OPERATOR] Switch default leader election resource lock for dependency-watchdog from endpoints to endpointsleases (gardener/gardener#5497, @ashwani2k)
• [DEPENDENCY] Use ginkgo v2 Report structures and drop usage of deprecated custom reporter. To adapt, replace the call of reporters.ReportViaDeprecatedReporter within any ReportAfterSuite node with reporter.ReportResults(*reportFilePath, *esIndex, report) (gardener/gardener#5504, @hendrikKahl)
• [DEPENDENCY] The packages test/ and extensions/test/ have been restructured. You might need to adapt your imports accordingly. (gardener/gardener#5469, @timebertt)

✨ New Features

• [USER] It is now possible to configure multiple accepted issuers for a shoot's kube-apiserver by setting .kubernetes.kubeAPIServer.serviceAccountConfig.acceptedIssuers in the shoot spec. This list of issuers will not be used to generate new service account tokens but will be used to determine if a service account token is accepted by asserting the value in the iss claim. This also allows a non-disruptive change of the current issuer of a kube-apiserver. (gardener/gardener#5498, @dimityrmirchev)
• [OPERATOR] Operators can now provide a scaleUpDelaySeconds or|and scaleDownDelaySeconds for individual dependent resources for dependency-watchdog probe to consider while scaling. (gardener/gardener#5497, @ashwani2k)
  • In addition to the delay, for each resource managed by dependency-watchdog probe one can also specify additional dependent resources via a new field scaleRefDependsOn. This ensures that dependency-watchdog probe applies scaling operation on a resource only if the dependents for this resource defined under scaleRefDependsOn are available in the desired state as per the applicable scaling operation.
• [OPERATOR] Add VPA-recommender scrape config to seed-prometheus (gardener/gardener#5467, @voelzmo)
• [DEVELOPER] Enhance package structure to isolate APIs (gardener/gardener#5497, @ashwani2k)
  • Export types in pkg/restarter and pkg/scaler to make them reusable for other packages.

🐛 Bug Fixes

• [OPERATOR] A bug has been fixed that caused the monitoring data to falsely display the API server as unavailable from shoots. (gardener/gardener#5543, @timebertt)
• [OPERATOR] Fix panic during shoot spec and status check. (gardener/gardener#5497, @ashwani2k)
• [OPERATOR] An error has been fixed that sporadically occurred during shoot deletion and was related to volumesnapshots, volumesnapshotcontents not being found. (gardener/gardener#5484, @timuthy)
• [OPERATOR] An issue preventing the nginx-ingress addon to be disabled is now fixed. (gardener/gardener#5482, @ialidzhikov)
• [OPERATOR] When SUSE OS node is restarted and "/run/log/journal" the promtail service continue to read from "/var/log/journal". (gardener/gardener#5470, @vlvasilev)

🏃 Others

• [OPERATOR] The proxy container of the apiserver-proxy now has a liveness probe ensuring that failing containers get restarted. (gardener/gardener#5544, @ScheererJ)
• [OPERATOR] Adds a new label to the cloudprovider secret so that it can be filtered by controllers. (gardener/gardener#5527, @kon-angelo)
  • The cloudprovider webhook now filters secrets using the new label of the cloudprovider secret.
• [OPERATOR] A bug has been fixed causing gardenlet to panic when a Shoot with multiple worker pools specifies eviction settings for the kubelet configuration in .spec.kubernetes.kubelet while .spec.provider.workers[].kubernetes.kubelet=nil. (gardener/gardener#5516, @rfranzke)
• [OPERATOR] Promote gardenlet feature gate ReversedVPN to beta. (gardener/gardener#5515, @ScheererJ)
• [OPERATOR] The DenyInvalidExtensionResources feature gate in the seed-admission-controller has been promoted to beta and is now enabled by default. (gardener/gardener#5512, @ary1992)
• [OPERATOR] gardener-controller-manager's SecretBinding provider controller is now enabled by default. (gardener/gardener#5499, @ialidzhikov)
• [OPERATOR] ManagedSeeds can now specify whether updates to the ManagedSeed spec are applied with a jitter. It can configured via the flag jitterUpdates in the managed seed controller configuration. (gardener/gardener#5483, @ary1992)
• [OPERATOR] Improve Grafana panels regarding Request latency (gardener/gardener#5477, @istvanballok)
• [OPERATOR] The Golang version was bumped to 1.17.7. (gardener/gardener#5476, @ialidzhikov)
• [OPERATOR] A mutating admission plugin is added which adds labels for the extension types specified in the spec of the objects (Seeds, Shoots, CloudProfiles, BackupBuckets, BackupEntrys). Extensions can make use of this label as object selector in their admission webhooks to filter out the resources, which they are responsible for. (gardener/gardener#5472, @shafeeqes)
• [OPERATOR] The gardener-resource-manager is now destroyed early in the deletion flow. In this way there will not be a failing deployment with PDB that prevents graceful termination of seed nodes. (gardener/gardener#5466, @vpnachev)
• [DEVELOPER] License and copyright information is now specified in REUSE format. (gardener/gardener#5497, @ashwani2k)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.42.0
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.42.0
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.42.0
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.42.0
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.42.0
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.42.0
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.42.0

[gardener]

🐛 Bug Fixes

• [OPERATOR] A bug has been fixed that caused the monitoring data to falsely display the API server as unavailable from shoots. (gardener/gardener#5545, @kris94)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.41.3
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.41.3
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.41.3
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.41.3
landscaper-gardenlet: eu.gcr.io/gardener-project/gardener/landscaper-gardenlet:v1.41.3
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.41.3
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.41.3
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.41.3
landscaper-controlplane: eu.gcr.io/gardener-project/gardener/landscaper-controlplane:v1.41.3

[gardener]

🏃 Others

• [OPERATOR] A bug has been fixed causing gardenlet to panic when a Shoot with multiple worker pools specifies eviction settings for the kubelet configuration in .spec.kubernetes.kubelet while .spec.provider.workers[].kubernetes.kubelet=nil. (gardener/gardener#5517, @rfranzke)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.41.2
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.41.2
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.41.2
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.41.2
landscaper-gardenlet: eu.gcr.io/gardener-project/gardener/landscaper-gardenlet:v1.41.2
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.41.2
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.41.2
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.41.2
landscaper-controlplane: eu.gcr.io/gardener-project/gardener/landscaper-controlplane:v1.41.2

[gardener]

🐛 Bug Fixes

• [OPERATOR] An issue preventing the nginx-ingress addon to be disabled is now fixed. (gardener/gardener#5493, @ialidzhikov)
• [OPERATOR] An error has been fixed that sporadically occurred during shoot deletion and was related to volumesnapshots, volumesnapshotcontents not being found. (gardener/gardener#5490, @timuthy)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.41.1
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.41.1
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.41.1
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.41.1
landscaper-gardenlet: eu.gcr.io/gardener-project/gardener/landscaper-gardenlet:v1.41.1
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.41.1
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.41.1
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.41.1
landscaper-controlplane: eu.gcr.io/gardener-project/gardener/landscaper-controlplane:v1.41.1

[gardener]

⚠️ Breaking Changes

• [OPERATOR] The Logging feature gate is removed, now logging is enabled/disabled/configured via the gardenlet configuration. (gardener/gardener#5337, @acumino)
• [OPERATOR] A new field Enabled is introduced in Logging field of the gardenlet configuration to enable/disable logging. By default it is set to false. (gardener/gardener#5337, @acumino)
• [DEPENDENCY] The default leader election of extensions has been changed from configmapsleases to leases. Please make sure, that you had at least [email protected] in your go.mod before upgrading to this version so that it has successfully acquired leadership with the hybrid resource lock (configmapsleases) at least once. (gardener/gardener#5456, @acumino)
• [DEPENDENCY] The controllercmd.LogErrAndExit and controller.*EventLogger helper functions have been dropped in favor of proper error handling and structured logging, as their usage was not aligned with our logging guideline. (gardener/gardener#5442, @timebertt)

✨ New Features

• [OPERATOR] The kube-apiserver's Prometheus metrics have been extended with some metrics that describe the costs of handling LIST requests. They are as follows. (gardener/gardener#5445, @acumino)
  • apiserver_cache_list_total: Counter of LIST requests served from watch cache, broken down by resource_prefix and index_name
  • apiserver_cache_list_fetched_objects_total: Counter of objects read from watch cache in the course of serving a LIST request, broken down by resource_prefix and index_name
  • apiserver_cache_list_evaluated_objects_total: Counter of objects tested in the course of serving a LIST request from watch cache, broken down by resource_prefix
  • apiserver_cache_list_returned_objects_total: Counter of objects returned for a LIST request from watch cache, broken down by resource_prefix
  • apiserver_storage_list_total: Counter of LIST requests served from etcd, broken down by resource
  • apiserver_storage_list_fetched_objects_total: Counter of objects read from etcd in the course of serving a LIST request, broken down by resource
  • apiserver_storage_list_evaluated_objects_total: Counter of objects tested in the course of serving a LIST request from etcd, broken down by resource
  • apiserver_storage_list_returned_objects_total: Counter of objects returned for a LIST request from etcd, broken down by resource
• [OPERATOR] Gardener API Server now supports configuration for enabling service account token volume projection. It is exposed through the .Values.global.apiserver.serviceAccountTokenVolumeProjection section in the respective chart's values. (gardener/gardener#5431, @dimityrmirchev)
• [OPERATOR] It is now possible to configure a user instead of a serviceaccount subject in the clusterrolebinding for the Gardener API Server when using virtual garden setup by setting .Values.global.virtualGarden.apiserver.user.name. (gardener/gardener#5431, @dimityrmirchev)
• [OPERATOR] Gardener Scheduler now supports configuration for enabling service account token volume projection. It is exposed through the .Values.global.scheduler.serviceAccountTokenVolumeProjection section in the respective chart's values. (gardener/gardener#5430, @dimityrmirchev)
• [OPERATOR] It is now possible to configure a user instead of a serviceaccount subject in the clusterrolebinding for the Gardener Scheduler when using virtual garden setup by setting .Values.global.virtualGarden.scheduler.user.name. (gardener/gardener#5430, @dimityrmirchev)
• [OPERATOR] Gardener Controller Manager now supports configuration for enabling service account token volume projection. It is exposed through the .Values.global.controller.serviceAccountTokenVolumeProjection section in the respective chart's values. (gardener/gardener#5429, @dimityrmirchev)
• [OPERATOR] It is now possible to configure a user instead of a serviceaccount subject in the clusterrolebinding for the Gardener Controller Manager when using virtual garden setup by setting .Values.global.virtualGarden.controller.user.name. (gardener/gardener#5429, @dimityrmirchev)
• [OPERATOR] The unused static ServiceAccount tokens for the controllers part of kube-controller-manager in the kube-system namespace of shoot clusters are now invalidated. Note that the tokens for the {node,route,service} controllers will only be invalidated for Kubernetes 1.21+ clusters since the cloud-controller-managers of prior versions still rely on them. (gardener/gardener#5422, @rfranzke)
• [OPERATOR] Gardener Admission Controller now supports configuration for enabling service account token volume projection. It is exposed through the .Values.global.admission.serviceAccountTokenVolumeProjection section in the respective chart's values. (gardener/gardener#5386, @dimityrmirchev)
• [OPERATOR] It is now possible to configure a user instead of a serviceaccount subject in the clusterrolebinding for the Gardener Admission Controller when using virtual garden setup by setting .Values.global.virtualGarden.admission.user.name. (gardener/gardener#5386, @dimityrmirchev)
• [DEVELOPER] A new logcheck tool has been added: it aims at making logs across Gardener components more consistent and help detect programmer-level errors early on. Read more about it in the tool's documentation. (gardener/gardener#5442, @timebertt)
• [DEVELOPER] Functions RESTConfigFromKubeconfig and RESTConfigFromClientConnectionConfiguration in package /pkg/client/kubernetes now support an allowedFields parameter which can be used to allow additional fields in the kubeconfig when creating clients. (gardener/gardener#5386, @dimityrmirchev)

🐛 Bug Fixes

• [USER] Fixed a bug, that broken Shoot system components didn't cause failing Shoot health checks. (gardener/gardener#5453, @timebertt)
• [USER] The EveryNodeReady shoot condition is now correctly computed even if a worker pool overwrites the Kubernetes version. (gardener/gardener#5437, @rfranzke)
• [OPERATOR] Increase the ginkgo timeout for default shoot serial test suite to prevent timeouts on tests (gardener/gardener#5428, @BeckerMax)
• [DEPENDENCY] Fixes a bug that caused only one Machine object to be restored, and all others to be recreated during control plane migration. (gardener/gardener#5471, @plkokanov)
• [DEPENDENCY] The generic Worker actuator is now more resilient to status updates that fail because of conflicts. (gardener/gardener#5451, @ialidzhikov)

📖 Documentation

• [OPERATOR] Add a GEP for Shoot cluster CA rotation (gardener/gardener#5395, @BeckerMax)

🏃 Others

• [USER] The used PriorityClass for kube-proxy was changed from system-cluster-critical to system-node-critical. (gardener/gardener#5438, @rfranzke)
• [OPERATOR] Add a size based retention policy to the aggregate prometheus (gardener/gardener#5450, @istvanballok)
• [OPERATOR] Federate vpn and api server availability metrics (gardener/gardener#5448, @wyb1)
• [OPERATOR] Error messages containing duplicate zones and overlapping zones in their description that can happen when reconciling DNSProviders are now classified as ERR_CONFIGURATION_PROBLEM. (gardener/gardener#5447, @plkokanov)
• [OPERATOR] Change the Throttle factor metric to Throttle % in the Kubernetes Pods dashboard (gardener/gardener#5446, @istvanballok)
• [OPERATOR] These metrics are only available for the shoot with k8s v1.23 and upwards. (gardener/gardener#5445, @acumino)
• [OPERATOR] The audit policy configmap protection by the Shoot reference controller of gardener-controller-manager is now enabled by default (but still configurable). (gardener/gardener#5426, @ialidzhikov)
• [OPERATOR] Keep the _count and _sum series of the api server histogram metrics (gardener/gardener#5424, @istvanballok)
• [OPERATOR] The following golang dependencies have been upgraded, please consult the upstream release notes and this issue for guidance on upgrading your golang dependencies when vendoring this gardener version: (gardener/gardener#5421, @acumino)
  • k8s.io/* to v0.23.3
  • sigs.k8s.io/controller-runtime to v0.11.0
  • sigs.k8s.io/controller-tools to v0.8.0
• [OPERATOR] Add a dashboard for API Server request duration and response size (gardener/gardener#5419, @wyb1)
• [OPERATOR] Use max instead of sum when counting etcd objects (gardener/gardener#5418, @wyb1)
• [OPERATOR] Add new collector configs to node exporter (gardener/gardener#5396, @teturou8001)
• [OPERATOR] Topology spread constraints and anti affinity are now defined in the coredns deployment for zones to better spread coredns pods across multiple zones. (gardener/gardener#5393, @DockToFuture)
• [OPERATOR] A validation is added to validate that the configured worker pools maximum nodes count do not exceed maximum nodes count allowed by the Pods CIDR (spec.networking.pods). (gardener/gardener#5389, @shafeeqes)
• [OPERATOR] gardener-apiserver does now support a field selector for Bastions by spec.shootRef.name (gardener/gardener#5382, @shafeeqes)
• [OPERATOR] Add feature flag DisableDNSProviderManagement. This is part of Move DNSProvider capabilities out of g/g #5270. (gardener/gardener#5349, @MartinWeindel)
• [OPERATOR] Allows cluster owner to switch between horizontal and cluster-proportional autoscaling of coredns (gardener/gardener#5275, @ScheererJ)

📰 Noteworthy

• [DEVELOPER] A new document has been added describing the development tasks for supporting a new minor Kubernetes version. (gardener/gardener#5461, @rfranzke)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.41.0
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.41.0
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.41.0
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.41.0
landscaper-gardenlet: eu.gcr.io/gardener-project/gardener/landscaper-gardenlet:v1.41.0
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.41.0
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.41.0
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.41.0
landscaper-controlplane: eu.gcr.io/gardener-project/gardener/landscaper-controlplane:v1.41.0

[gardener]

🐛 Bug Fixes

• [DEPENDENCY] Fixes a bug that caused only one Machine object to be restored, and all others to be recreated during control plane migration. (gardener/gardener#5474, @plkokanov)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.40.2
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.40.2
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.40.2
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.40.2
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.40.2
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.40.2
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.40.2

[gardener]

🐛 Bug Fixes

• [OPERATOR] Increase the ginkgo timeout for default shoot serial test suite to prevent timeouts on tests (gardener/gardener#5436, @BeckerMax)
• [DEPENDENCY] The generic Worker actuator is now more resilient to status updates that fail because of conflicts. (gardener/gardener#5454, @ialidzhikov)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.40.1
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.40.1
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.40.1
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.40.1
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.40.1
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.40.1
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.40.1

[gardener]

🐛 Bug Fixes

• [OPERATOR] The finalizer on the Bastion resource in the garden cluster can now be updated even if the referenced Shoot is in deletion (gardener/gardener#5433, @ialidzhikov)
• [DEPENDENCY] The generic Worker actuator is now more resilient to status updates that fail because of conflicts. (gardener/gardener#5455, @ialidzhikov)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.39.5
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.39.5
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.39.5
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.39.5
landscaper-gardenlet: eu.gcr.io/gardener-project/gardener/landscaper-gardenlet:v1.39.5
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.39.5
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.39.5

[gardener]

⚠️ Breaking Changes

• [USER] Gardener does no longer automatically reconcile Shoot clusters if their referenced Audit Policy ConfigMap was changed. If users want to immediately rollout Audit Policy changes, they can manually trigger a Shoot reconciliation as described in triggering an immediate reconciliation. (gardener/gardener#5392, @timebertt)
• [OPERATOR] ⚠️ Gardener does no longer support shoot clusters with Kubernetes versions < 1.17. Make sure to upgrade all existing clusters before upgrading to this Gardener version. (gardener/gardener#5272, @rfranzke)
• [OPERATOR] ⚠️ The minimum Kubernetes version for garden clusters has been raised from v1.16 to v1.17. Make sure that your garden cluster meets this requirement before upgrading to this Gardener version. (gardener/gardener#5272, @rfranzke)
• [DEPENDENCY] The field status.ingress of bastions.extensions.gardener.cloud is now optional and thus changed to a pointer (gardener/gardener#5416, @petersutter)
• [DEPENDENCY] Two validation helpers have moved to dedicated packages to allow reusing them without importing the entire Core API validation package and its dependencies: (gardener/gardener#5347, @timebertt)
  • ValidateFeatureGates to pkg/utils/validation/features
  • ShouldEnforceImmutability to pkg/utils/validation
• [DEPENDENCY] The ginkgo dependency has been upgraded to v2.1.0 (major version upgrade). You can consult gardener/gardener#5311 and the migration guide for the steps required by this. (gardener/gardener#5313, @timebertt)

✨ New Features

• [USER] Importing "exported packages" (e.g. Gardener API packages or extensions library) in other Go projects, will now pull in only a minimal set of transitive dependencies, which will simplify dependency management for dependent projects. If you still face some difficulties, please report an issue. (gardener/gardener#5347, @timebertt)
• [DEVELOPER] make check now verifies a set of import restrictions. Read more about it here. (gardener/gardener#5347, @timebertt)
• [DEVELOPER] More e2e tests will be executed on your PRs and periodically on master now. You can also run them on your development machine using make test-e2e-local. (gardener/gardener#5320, @timebertt)

🐛 Bug Fixes

• [USER] Gardener now validates AuditPolicy ConfigMaps, even if operators have disabled the reference protection feature for them. (gardener/gardener#5399, @timebertt)
• [OPERATOR] Fixed issue with "Required value" error for status.ingress field of bastions.extensions.gardener.cloud custom resource. This field is not required anymore. (gardener/gardener#5416, @petersutter)
• [OPERATOR] Prevent gardener-controller-manager from "piling up" a high amount of shoot reconciliations until the next restart because of updated AuditPolicy ConfigMaps. (gardener/gardener#5400, @timebertt)
• [OPERATOR] Manually applied annotations or labels are now preserved on shoot Namespaces in the seed clusters. (gardener/gardener#5384, @rfranzke)
• [OPERATOR] Fixes an issue that prevents the status of Machine objects to be modified during the restore phase of control plane migration. Patch is now used to do the modification instead of an update call. (gardener/gardener#5381, @plkokanov)
• [OPERATOR] An issue causing external DNSRecord's Secret data to be not updated on Shoot deletion is now fixed. (gardener/gardener#5375, @ialidzhikov)
• [OPERATOR] Deletion of Shoot is no longer wrongly blocked because of Bastion in the same Project that is not related to this Shoot. (gardener/gardener#5361, @ialidzhikov)
• [OPERATOR] When the ReversedVPN feature gate is disabled, the kube-apiserver-http-proxy secret is properly removed from the ShootState and the shoot's control plane. (gardener/gardener#5354, @plkokanov)
• [OPERATOR] The finalizer on the Bastion resource in the garden cluster can now be updated even if the referenced Shoot is in deletion (gardener/gardener#5345, @petersutter)
• [OPERATOR] A bug has been fixed which caused clusters which are being hibernated from succeeding because of a Gardener-Resource-Manager deployment issue. (gardener/gardener#5329, @timuthy)
• [OPERATOR] An issue causing the namespace.gardener.cloud/keep-after-project-deletion="true" annotation to be wrongly added to the Project Namespace when the Project controller does not adopt existing Project Namespace is now fixed. (gardener/gardener#5322, @acumino)
• [OPERATOR] Fixed migration flow for shoots on providers that require the cloud-provider-config secret to be injected in some control plane components. (gardener/gardener#4584, @plkokanov)
• [OPERATOR] Fixed migration flow for hibernated clusters on azure where resources managed by the remedy controller were not removed due to the remedy controller being scaled down to 0. (gardener/gardener#4584, @plkokanov)
• [DEVELOPER] Make logging integration tests on flatcar OS more stable by increasing the memory limit and request of the logger application (gardener/gardener#5351, @vlvasilev)
• [DEVELOPER] Fix logging integration test to remove the IPs from loki.Spec.ClusterIPs (gardener/gardener#5327, @vlvasilev)
• [DEPENDENCY] A bug regarding the usage of the token requestor in the generic ControlPlane actuator package has been fixed. (gardener/gardener#5338, @rfranzke)

📖 Documentation

• [USER] Gardener API reference does now contain information if a field is immutable. (gardener/gardener#5348, @ialidzhikov)

🏃 Others

• [OPERATOR] The number of max connections for envoy-proxy sidecar in vpn-seed-server is now increased to 8192. (gardener/gardener#5379, @DockToFuture)
• [OPERATOR] Min allowed vpa cpu requests for the vpn-shoot pod are set to 100m. (gardener/gardener#5372, @DockToFuture)
• [OPERATOR] The following image is updated: (gardener/gardener#5360, @ialidzhikov)
  • grafana/grafana: 7.5.12 -> 7.5.13
• [OPERATOR] Fix kube scheduler metrics collection (gardener/gardener#5318, @wyb1)
• [OPERATOR] Grafana's default timezone is now set to UTC (previously it was browser). (gardener/gardener#5312, @ialidzhikov)
• [OPERATOR] The gardenlet blueprint is now auto-generated and is executed as part of running make generate (gardener/gardener#5276, @danielfoehrKn)
• [OPERATOR] Added the blueprint for the landscaper-controlplane component (gardener/gardener#5233, @danielfoehrKn)
• [OPERATOR] Added a blueprint and OpenAPI generator for landscaper components (gardener/gardener#5233, @danielfoehrKn)

📰 Noteworthy

• [USER] Shoot clusters using Kubernetes 1.23 or above will continue to use the cgroupfs cgroup driver (the change to use systemd was reverted). (gardener/gardener#5324, @rfranzke)
• [OPERATOR] In case gardenlet runs on a cluster with at least Kubernetes 1.20 then it will use a projected ServiceAccount token only valid for 12h for communicating with the (seed) cluster's API Server. (gardener/gardener#5280, @rfranzke)
• [OPERATOR] promtail does no longer use a static token but an auto-rotated ServiceAccount token which is only valid for 12h. (gardener/gardener#5153, @rfranzke)

[hvpa-controller]

⚠️ Breaking Changes

• [DEVELOPER] Switch to github.com/gardener/hvpa-controller/api if you vendor only the API of hvpa-controller. (gardener/hvpa-controller#85, @amshuman-kr)

🏃 Others

• [OPERATOR] Check if OOMKilled pod has latest resource values before overriding stabilisation (gardener/hvpa-controller#78, @ggaurav10)
• [OPERATOR] consider hpa scale out limited if hpa is not deployed (gardener/hvpa-controller#77, @ggaurav10)
• [DEVELOPER] Golang is upgraded to 1.15.3. (gardener/hvpa-controller#81, @mvladev)

📰 Noteworthy

• [OPERATOR] A new module github.com/gardener/hvpa-controller/api can be used to get the API definitions. (gardener/hvpa-controller#85, @amshuman-kr)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.40.0
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.40.0
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.40.0
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.40.0
landscaper-gardenlet: eu.gcr.io/gardener-project/gardener/landscaper-gardenlet:v1.40.0
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.40.0
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.40.0
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.40.0
landscaper-controlplane: eu.gcr.io/gardener-project/gardener/landscaper-controlplane:v1.40.0

[gardener]

⚠️ Breaking Changes

• [DEPENDENCY] The field status.ingress of bastions.extensions.gardener.cloud is now optional and thus changed to a pointer (gardener/gardener#5417, @ialidzhikov)

🐛 Bug Fixes

• [OPERATOR] Fixed issue with "Required value" error for status.ingress field of bastions.extensions.gardener.cloud custom resource. This field is not required anymore. (gardener/gardener#5417, @ialidzhikov)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.39.4
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.39.4
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.39.4
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.39.4
landscaper-gardenlet: eu.gcr.io/gardener-project/gardener/landscaper-gardenlet:v1.39.4
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.39.4
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.39.4
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.39.4

[gardener]

🐛 Bug Fixes

• [USER] Gardener now validates AuditPolicy ConfigMaps, even if operators have disabled the reference protection feature for them. (gardener/gardener#5402, @timebertt)
• [OPERATOR] Prevent gardener-controller-manager from "piling up" a high amount of shoot reconciliations until the next restart because of updated AuditPolicy ConfigMaps. (gardener/gardener#5405, @timebertt)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.38.6
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.38.6
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.38.6
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.38.6
landscaper-gardenlet: eu.gcr.io/gardener-project/gardener/landscaper-gardenlet:v1.38.6
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.38.6
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.38.6
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.38.6

[gardener]

🐛 Bug Fixes

• [USER] Gardener now validates AuditPolicy ConfigMaps, even if operators have disabled the reference protection feature for them. (gardener/gardener#5403, @timebertt)
• [OPERATOR] Prevent gardener-controller-manager from "piling up" a high amount of shoot reconciliations until the next restart because of updated AuditPolicy ConfigMaps. (gardener/gardener#5406, @timebertt)
• [OPERATOR] Deletion of Shoot is no longer wrongly blocked because of Bastion in the same Project that is not related to this Shoot. (gardener/gardener#5371, @ialidzhikov)
• [OPERATOR] When the ReversedVPN feature gate is disabled, the kube-apiserver-http-proxy secret is properly removed from the ShootState and the shoot's control plane. (gardener/gardener#5365, @plkokanov)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.37.7
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.37.7
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.37.7
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.37.7
landscaper-gardenlet: eu.gcr.io/gardener-project/gardener/landscaper-gardenlet:v1.37.7
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.37.7
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.37.7
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.37.7

[gardener]

🐛 Bug Fixes

• [USER] Gardener now validates AuditPolicy ConfigMaps, even if operators have disabled the reference protection feature for them. (gardener/gardener#5401, @timebertt)
• [OPERATOR] Fixes an issue that prevents the status of Machine objects to be modified during the restore phase of control plane migration. Patch is now used to do the modification instead of an update call. (gardener/gardener#5408, @timebertt)
• [OPERATOR] Prevent gardener-controller-manager from "piling up" a high amount of shoot reconciliations until the next restart because of updated AuditPolicy ConfigMaps. (gardener/gardener#5404, @timebertt)
• [DEVELOPER] Make logging integration tests on flatcar OS more stable by increasing the memory limit and request of the logger application (gardener/gardener#5409, @timebertt)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.39.3
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.39.3
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.39.3
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.39.3
landscaper-gardenlet: eu.gcr.io/gardener-project/gardener/landscaper-gardenlet:v1.39.3
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.39.3
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.39.3
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.39.3

[gardener]

🐛 Bug Fixes

• [OPERATOR] Deletion of Shoot is no longer wrongly blocked because of Bastion in the same Project that is not related to this Shoot. (gardener/gardener#5370, @ialidzhikov)
• [OPERATOR] When the ReversedVPN feature gate is disabled, the kube-apiserver-http-proxy secret is properly removed from the ShootState and the shoot's control plane. (gardener/gardener#5366, @plkokanov)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.38.5
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.38.5
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.38.5
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.38.5
landscaper-gardenlet: eu.gcr.io/gardener-project/gardener/landscaper-gardenlet:v1.38.5
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.38.5
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.38.5
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.38.5