[gardener/gardener]

🐛 Bug Fixes

• [OPERATOR] A bug has been fixed for Istio-Ingress Gateways for seeds that use ExposureClassHandlers. Earlier, annotations in seed.spec.settings.loadBalancerServices caused an override of the ones specified in gardenletConfiguration.exposureClassHandler[].loadBalancerService for zonal Istios. Now, annotations in gardenletConfiguration.exposureClassHandler[].loadBalancerService are given priority, like it was already the case of the global Istio. by @gardener-ci-robot [#8180]

🏃 Others

• [OPERATOR] Adapt vpa-updater QPS limits such that it doesn't get throttled on large clusters by @gardener-ci-robot [#8176]

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.72.3
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.72.3
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.72.3
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.72.3
operator: eu.gcr.io/gardener-project/gardener/operator:v1.72.3
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.72.3
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.72.3

[gardener]

🐛 Bug Fixes

• [OPERATOR] Fix network annotations to allow fluent-bit connecting to shoot Valis. (gardener/gardener#8200, @gardener-ci-robot)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.74.1
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.74.1
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.74.1
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.74.1
operator: eu.gcr.io/gardener-project/gardener/operator:v1.74.1
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.74.1
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.74.1

[gardener]

⚠️ Breaking Changes

• [USER] Annotation alpha.featuregates.shoot.gardener.cloud/node-local-dns is deprecated and will be removed in future releases. Use field .spec.systemComponents.nodeLocalDNS.enabled in Shoot instead. Switching on node-local-dns via shoot specification will roll the nodes even if node-local-dns was enabled beforehand via annotation. (gardener/gardener#8067, @acumino)
• [USER] Annotation alpha.featuregates.shoot.gardener.cloud/node-local-dns-force-tcp-to-{cluster-dns, upstream-dns} is deprecated and will be removed in future releases. Use field .spec.systemComponents.{nodeLocalDNSforceTCPToClusterDNS, nodeLocalDNSforceTCPToUpstreamDNS} in Shoot instead. (gardener/gardener#8067, @acumino)
• [OPERATOR] The Seed's .spec.settings.ownerChecks field is now no-op - the gardener-apiserver no longer defaults this field and no longer validates it. The field will be set always to nil on CREATE/UPDATE request. (gardener/gardener#7951, @dimitar-kostadinov)
  • Gardener landscape operators specifying this field should no longer specify it. The field will be removed in a future version of Gardener.
• [OPERATOR] The GA-ed feature gates HAControlPlanes and FullNetworkPoliciesInRuntimeCluster have been removed. (gardener/gardener#8083, @rfranzke)
• [OPERATOR] ⚠️ Gardener does no longer support garden, seed, or shoot clusters with Kubernetes versions < 1.22. Make sure to upgrade all existing clusters before upgrading to this Gardener version. (gardener/gardener#8087, @shafeeqes)
• [OPERATOR] The shootstate-extensions and shootstate-secret controllers have been dropped. The gardenlet's component config file should be updated to no longer specify related configuration (.controllers.{shootSecret,shootStateSync}). (gardener/gardener#8136, @rfranzke)
• [OPERATOR] gardener.cloud/operation annotation was introduced to seeds. This includes a verification of its value. Please check your seeds for this annotation and remove it if necessary prior to the update. (gardener/gardener#8152, @timebertt)
• [OPERATOR] A new field .spec.virtualCluster.dns.domains was added to the Garden API. This field allows to expose the kube-apiserver of the virtual cluster via multiple domains. Earlier, the API only accepted one domain name via .spec.virtualCluster.dns.domain. (gardener/gardener#8173, @gardener-ci-robot)
  • ⚠️ With this change .spec.virtualCluster.dns.domain is deprecated and will be removed in the next release. Please update your Garden resource to the new .spec.virtualCluster.dns.domains field by removing the existing domain configuration from dns.domain and add it as the first entry of dns.domains.
• [DEVELOPER] The deprecated local development setups have been removed. From now on, only the kind-based setups are supported. Please refer to this guide for all information. (gardener/gardener#8075, @oliver-goetz)
• [DEVELOPER] The deprecated allow-to-seed-apiserver NetworkPolicy is no longer available in garden or seed clusters. Use allow-to-runtime-apiserver instead. (gardener/gardener#8083, @rfranzke)

✨ New Features

• [USER] The VerticalPodAutoscaler resources for kube-proxys is no longer recreated when the Kubernetes patch version of the Shoot or the respective worker pools is updated. This ensures updated kube-proxys keep the same CPU/memory resource requirements as before the patch version update. In order to put this change into effect, all existing VerticalPodAutoscalers for kube-proxys are getting recreated. (gardener/gardener#8071, @rfranzke)
• [USER] Shoot addon nginx-ingress-controller image is updated to v1.8.0 for Kubernetes v1.24+ clusters, to v1.6.4 for Kubernetes v1.23 clusters, and to v1.4.0 for Kubernetes v1.22 clusters. (gardener/gardener#8096, @shafeeqes)
• [OPERATOR] Gardener uses an InternalSecret per Shoot for syncing the client CA to the project namespace in the garden cluster (named <shoot-name>.ca-client). The shoots/adminkubeconfig subresource signs short-lived client certificates by retrieving the CA from the InternalSecret. (gardener/gardener#8088, @timebertt)
• [OPERATOR] A new controller in gardenlet for periodically backing up the ShootState for Shoots has been introduced. This controller is only activated when gardenlet is responsible for an unmanaged Seed (i.e., one not backed by a ManagedSeed object). By default, backups are taken roughly each 6h. (gardener/gardener#8112, @rfranzke)
• [OPERATOR] If gardenlet is responsible for a managed Seed, it will delete all ShootState resources for its Shoots that are not currently in migration. See also GEP-22 for further details about the motivation. (gardener/gardener#8144, @rfranzke)

🐛 Bug Fixes

• [USER] A regression was fixed that prevented deletions for shoot clusters which were created with a wrong configuration (e.g. with an unavailable domain name). (gardener/gardener#8122, @timuthy)
• [OPERATOR] Missing permissions were added for the Gardenlet service account for Machine objects. This fix is relevant if feature gate MachineControllerManagerDeployment is enabled in your landscape. (gardener/gardener#8121, @timuthy)
• [OPERATOR] An issue causing deletion of a legacy (wrongly configured) Shoot cluster to be denied because of network ranges overlapping with the default VPN network is now fixed. (gardener/gardener#8129, @ialidzhikov)
• [OPERATOR] gardener-resource-manager's system-components-config webhook no longer adds the toleration for the ToBeDeletedByClusterAutoscaler taint to system components in shoot clusters. The ToBeDeletedByClusterAutoscaler taint is maintained by the cluster-autoscaler. This was breaking cluster-autoscaler's drain mechanism when scaling down an under-utilized node. It was causing just evicted system components from to be deleted node to be scheduled again on the to be deleted node. (gardener/gardener#8172, @gardener-ci-robot)
• [OPERATOR] A bug has been fixed for Istio-Ingress Gateways for seeds that use ExposureClassHandlers. Earlier, annotations in seed.spec.settings.loadBalancerServices caused an override of the ones specified in gardenletConfiguration.exposureClassHandler[].loadBalancerService for zonal Istios. Now, annotations in gardenletConfiguration.exposureClassHandler[].loadBalancerService are given priority, like it was already the case of the global Istio. (gardener/gardener#8178, @gardener-ci-robot)
• [DEVELOPER] On deletion, the generic ControlPlane actuator will now redeploy the cloud config chart to allow provider extensions update the content with the most up-to-date information. (gardener/gardener#8106, @kon-angelo)

🏃 Others

• [OPERATOR] Plutono is now updated to v7.5.22 (gardener/gardener#8081, @nickytd)
• [OPERATOR] The reconciliation time limit for the controller resource reconciliation, e.g. for ManagedResource, has been increased from 1m to 3m. (gardener/gardener#8085, @ScheererJ)
• [OPERATOR] Fixed flaky operator behaviour with regards to istio deployment caused by concurrent update of garden object (gardener/gardener#8103, @ScheererJ)
• [OPERATOR] Vali is now updated to version v2.2.6 (gardener/gardener#8111, @nickytd)
• [OPERATOR] The Deploying Shoot namespace in Seed step was slightly improved. Earlier it failed at some occasions when it tried to read zone information for volumes that have not been created yet. This was a transient error that dissolved in subsequent reconcile runs. (gardener/gardener#8115, @timuthy)
• [OPERATOR] All components in the gardener logging stack are now updated to the following respective versions. Fluent-bit to 2.1.4, Fluent-operator to 2.3.0 and logging to 0.55.3 (gardener/gardener#8133, @nickytd)
• [OPERATOR] Decouple progess update of gardener operator from task flow logic and thereby prevent concurrency bugs. (gardener/gardener#8145, @ScheererJ)
• [OPERATOR] Adapt vpa-updater QPS limits such that it doesn't get throttled on large clusters (gardener/gardener#8174, @gardener-ci-robot)
• [OPERATOR] The kind cluster used in local setup does now use the new way in containerd to configure registry mirrors. (gardener/gardener#8047, @ialidzhikov)
• [DEVELOPER] extensions.gardener.cloud/v1alpha1.ControlPlane is now deployed after kube-apiserver in the Shoot reconciliation flow. (gardener/gardener#8182, @gardener-ci-robot)

[etcd-druid]

🏃 Others

• [OPERATOR] Bumped up the custom image version to v3.4.13-bootstrap-11 (gardener/etcd-druid#624, @abdasgupta)
• [OPERATOR] Druid now exposes metrics related to snapshot compaction, on default port 8080. Please expose the desired metrics port via the etcd-druid service to allow metrics to be scraped by a Prometheus instance. (gardener/etcd-druid#625, @abdasgupta)

[logging]

🏃 Others

• [OPERATOR] The logging e2e event logger test is now adapted to vali logging stack. (gardener/logging#199, @nickytd)
• [OPERATOR] Now git revision and commit ids are properly propagated through build variables. These are showed in the fluent-bit plugin logs during start. (gardener/logging#200, @nickytd)
• [OPERATOR] Base image on telegraf and tune2fs is upgraded from 3.17.2 to 3.18.0 (gardener/logging#201, @nickytd)
• [OPERATOR] Gardener-based e2e test for the event-logger. (gardener/logging#191, @vlvasilev)
• [DEVELOPER] Introduces a skaffold local development pipeline to fluent-bit-vali-plugin (gardener/logging#202, @nickytd)
• [DEVELOPER] The project vendors the latest released gardener version - v1.73.0 (gardener/logging#204, @nickytd)
• [DEVELOPER] The fluent-bit-vali-plugin now supports fluent-bit v2.1.0 and above. (gardener/logging#205, @nickytd)

[machine-controller-manager]

🐛 Bug Fixes

• [OPERATOR] Included UnavailableReplicas in determining if a machine deployment status update is needed (gardener/machine-controller-manager#834, @ialidzhikov)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.74.0
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.74.0
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.74.0
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.74.0
operator: eu.gcr.io/gardener-project/gardener/operator:v1.74.0
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.74.0
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.74.0

[gardener]

🏃 Others

• [OPERATOR] Adapt vpa-updater QPS limits such that it doesn't get throttled on large clusters (gardener/gardener#8177, @gardener-ci-robot)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.71.6
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.71.6
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.71.6
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.71.6
operator: eu.gcr.io/gardener-project/gardener/operator:v1.71.6
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.71.6
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.71.6

[gardener]

🐛 Bug Fixes

• [OPERATOR] An issue causing deletion of a legacy (wrongly configured) Shoot cluster to be denied because of network ranges overlapping with the default VPN network is now fixed. (gardener/gardener#8137, @oliver-goetz)
• [OPERATOR] Missing permissions were added for the Gardenlet service account for Machine objects. This fix is relevant if feature gate MachineControllerManagerDeployment is enabled in your landscape. (gardener/gardener#8123, @gardener-ci-robot)

🏃 Others

• [OPERATOR] Plutono is now updated to v7.5.22 (gardener/gardener#8100, @gardener-ci-robot)
• [OPERATOR] Fixed flaky operator behaviour with regards to istio deployment caused by concurrent update of garden object (gardener/gardener#8105, @gardener-ci-robot)
• [OPERATOR] The Deploying Shoot namespace in Seed step was slightly improved. Earlier it failed at some occasions when it tried to read zone information for volumes that have not been created yet. This was a transient error that dissolved in subsequent reconcile runs. (gardener/gardener#8118, @gardener-ci-robot)
• [OPERATOR] The reconciliation time limit for the controller resource reconciliation, e.g. for ManagedResource, has been increased from 1m to 3m. (gardener/gardener#8090, @gardener-ci-robot)

Docker Images

apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.73.1
admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.73.1
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.73.1
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.73.1
operator: eu.gcr.io/gardener-project/gardener/operator:v1.73.1
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.73.1
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.73.1

[gardener]

🐛 Bug Fixes

• [OPERATOR] An issue causing deletion of a legacy (wrongly configured) Shoot cluster to be denied because of network ranges overlapping with the default VPN network is now fixed. (gardener/gardener#8138, @oliver-goetz)
• [OPERATOR] The terraformer library will now skip deletion of the Terraformer pod when the request context has been canceled. This change aims to prevent inconsistencies in Terraform state by attempting to allow uninterrupted execution of healthy Terraformer pods. (gardener/gardener#8107, @gardener-ci-robot)
• [OPERATOR] A bug has been fixed in the garden/fluent-bit that caused a failure in creating networkpolicies for scraping metrics. (gardener/gardener#8074, @gardener-ci-robot)

🏃 Others

• [OPERATOR] Plutono is now updated to v7.5.22 (gardener/gardener#8101, @ialidzhikov)
• [OPERATOR] Fixed flaky operator behaviour with regards to istio deployment caused by concurrent update of garden object (gardener/gardener#8104, @gardener-ci-robot)
• [OPERATOR] The Deploying Shoot namespace in Seed step was slightly improved. Earlier it failed at some occasions when it tried to read zone information for volumes that have not been created yet. This was a transient error that dissolved in subsequent reconcile runs. (gardener/gardener#8119, @gardener-ci-robot)
• [OPERATOR] The reconciliation time limit for the controller resource reconciliation, e.g. for ManagedResource, has been increased from 1m to 3m. (gardener/gardener#8091, @gardener-ci-robot)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.72.2
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.72.2
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.72.2
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.72.2
operator: eu.gcr.io/gardener-project/gardener/operator:v1.72.2
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.72.2
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.72.2

[gardener]

🐛 Bug Fixes

• [OPERATOR] The terraformer library will now skip deletion of the Terraformer pod when the request context has been canceled. This change aims to prevent inconsistencies in Terraform state by attempting to allow uninterrupted execution of healthy Terraformer pods. (gardener/gardener#8108, @gardener-ci-robot)
• [OPERATOR] A bug has been fixed in the garden/fluent-bit that caused a failure in creating networkpolicies for scraping metrics. (gardener/gardener#8077, @timuthy)

🏃 Others

• [OPERATOR] Plutono is now updated to v7.5.22 (gardener/gardener#8102, @ialidzhikov)
• [OPERATOR] The Deploying Shoot namespace in Seed step was slightly improved. Earlier it failed at some occasions when it tried to read zone information for volumes that have not been created yet. This was a transient error that dissolved in subsequent reconcile runs. (gardener/gardener#8117, @gardener-ci-robot)
• [OPERATOR] The reconciliation time limit for the controller resource reconciliation, e.g. for ManagedResource, has been increased from 1m to 3m. (gardener/gardener#8092, @gardener-ci-robot)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.71.5
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.71.5
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.71.5
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.71.5
operator: eu.gcr.io/gardener-project/gardener/operator:v1.71.5
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.71.5
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.71.5

[gardener]

⚠️ Breaking Changes

• [OPERATOR] The field .spec.secretRef in the Seed API has been deprecated and will be removed in a future release of Gardener. (gardener/gardener#8064, @acumino)
• [OPERATOR] Before upgrading to this gardener version, operators should configure gardener-apiserver to encrypt the internalsecrets.core.gardener.cloud resource in etcd. (gardener/gardener#8078, @timebertt)
• [OPERATOR] The GA-ed feature gates SeedChange and CopyEtcdBackupsDuringControlPlaneMigration have been removed. (gardener/gardener#8008, @rfranzke)
• [OPERATOR] The feature gates FullNetworkPolicies and HAControlPlanes have been promoted to GA and are now locked to "unconditionally enabled". (gardener/gardener#8008, @rfranzke)
• [OPERATOR] The deprecated feature gate APIServerSNI has been removed. (gardener/gardener#8062, @rfranzke)
• [DEVELOPER] Functions controllerutils.GetAndCreateOrMergePatch, controllerutils.GetAndCreateOrStrategicMergePatch, controllerutils.CreateOrGetAndMergePatch and controllerutils.CreateOrGetAndStrategicMergePatch were incompatibly changed and now accept a controllerutils.PatchOption instead of client.MergeFromOption. (gardener/gardener#8043, @timuthy)
  • If your controllers use one of these functions with client.MergeFromOption, you should update it to controllerutils.PatchOption.
  • The controllerutils.PatchOption can hold two options today:
  • client.MergeFromOption which is passed to the underlying patch function.
  • controllerutils.SkipEmptyPatch which prevents sending empty patches ({}).

✨ New Features

• [OPERATOR] A new alpha feature gate DisableScalingClassesForShoots has been introduced on gardenlet. If turned on, initial resource requests for kube-apiservers of shoot clusters running on seed clusters which enable the HVPA feature gate are assigned statically and no longer by a scaling class determined by maximum node count. This helps to reduce resource waste for clusters with little usage. (gardener/gardener#8003, @voelzmo)
• [OPERATOR] A new alpha feature gate named MachineControllerManagerDeployment has been introduced in gardenlet. Only enable it when all registered provider extensions in your landscape support this feature. (gardener/gardener#8018, @rfranzke)
• [OPERATOR] gardener-apiserver now exposes a new core.gardener.cloud/v1beta1.InternalSecret API, see the documentation for more information. (gardener/gardener#8025, @timebertt)
• [OPERATOR] The gardenlet's ManagedSeed controller now cleans up the referred seed secret when .spec.secretRef is unset in the seed template. (gardener/gardener#8039, @shafeeqes)
• [DEVELOPER] It is now easier to annotate Services related to extensions serving webhook handlers that must be reached by kube-apiservers running in separate namespaces such that the respective network traffic gets allowed. Please refer to this guide for all information. Extensions serving shoot webhook should make use of this new approach - the old functionality deploying dedicated NetworkPolicys is deprecated and will be removed in the future. (gardener/gardener#8076, @rfranzke)
• [DEVELOPER] gardenlet's ControllerInstallation controller now populates the feature gate of gardenlet via the Helm values to extensions when they are getting installed. The information is populated via the .gardener.gardenlet.featureGates key. It contains a map whose keys are feature gates names and whose values are booleans (depicting the enablement status). (gardener/gardener#8011, @rfranzke)
• [DEVELOPER] Provider extensions should be adapted such that they only inject their provider-specific machine-controller-manager sidecar container into the machine-controller-manager deployment instead of managing the full deployment themselves. In the future, gardenlet will take over managing it. Please see https://github.com/gardener/gardener/pull/8019 for an example how provider-local was adapted and replicate it for your provider extensions. (gardener/gardener#8018, @rfranzke)
• [DEVELOPER] Provider extensions should be adapted such that they no longer perform health checks specific to the machine-controller-manager deployment or the machines/nodes. In the future, gardenlet will take over performing these checks. Please see https://github.com/gardener/gardener/pull/8019 for an example how provider-local was adapted and replicate it for your provider extensions. (gardener/gardener#8056, @rfranzke)

🐛 Bug Fixes

• [USER] A bug causing the shoot provider label in the infrastructure secret to not get cleaned up is now fixed. (gardener/gardener#7994, @shafeeqes)
• [USER] Webhooks remediator sets the timeoutSeonds to 3 seconds for webhook affecting lease resources in kube-system namespace only if there is no objectSelector provided in webhook. (gardener/gardener#8034, @acumino)
• [OPERATOR] A bug has been fixed in the garden/fluent-bit that caused a failure in creating networkpolicies for scraping metrics. (gardener/gardener#8069, @timuthy)
• [OPERATOR] A bug has been fixed in the HighAvailabilityConfig-Webhook which caused duplicated entries for zone affinities. (gardener/gardener#8042, @timuthy)
• [OPERATOR] The terraformer library will now skip deletion of the Terraformer pod when the request context has been canceled. This change aims to prevent inconsistencies in Terraform state by attempting to allow uninterrupted execution of healthy Terraformer pods. (gardener/gardener#8059, @kon-angelo)
• [DEVELOPER] pkg/resourcemanager/controller/garbagecollector/references.InjectAnnotations now also handles pods.spec.imagePullSecrets. (gardener/gardener#8028, @vpnachev)

🏃 Others

• [OPERATOR] The shoot namespace in seeds is redeployed during shoot deletion to update the zones in use. (gardener/gardener#8079, @timuthy)
• [OPERATOR] nginx-ingress-controller-seed image is updated to v1.8.0 for 1.24.x+ seeds. (gardener/gardener#8021, @shafeeqes)
• [OPERATOR] The following image is updated: (gardener/gardener#8029, @nickytd)
  • quay.io/brancz/kube-rbac-proxy: v0.14.0 -> v0.14.2
• [OPERATOR] The worker count for the NetworkPolicy controller in GRM was increased to 20. This is necessary to create and update NetworkPolicies in time, esp. on larger seed clusters. (gardener/gardener#8035, @timuthy)
• [DEVELOPER] gardenlet is taking over management of the CustomResourceDefinitions for the machine.sapcloud.io/v1alpha1 API group, hence extensions do no longer need to take care. Consequently, the extensions/pkg/controller/worker.Options struct as well as the extensions/pkg/controller/worker.ApplyMachineResources{ForConfig} functions are deprecated and will be removed in a future release. (gardener/gardener#8015, @rfranzke)
• [DEVELOPER] Go version is updated to 1.20.5. (gardener/gardener#8037, @shafeeqes)
• [DEVELOPER] The kind clusters are now unified to use garden.local.gardener.cloud DNS name in the containerd config when configuring registry mirror hostnames. Previously, to access the pull through registry cache some kind clusters were configured to use garden.local.gardener.cloud, others - the Node name of the control plane Node. (gardener/gardener#8063, @ialidzhikov)admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.73.0

Docker Images

apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.73.0
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.73.0
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.73.0
operator: eu.gcr.io/gardener-project/gardener/operator:v1.73.0
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.73.0
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.73.0

[gardener]

🐛 Bug Fixes

• [OPERATOR] A bug has been fixed in the HighAvailabilityConfig-Webhook which caused duplicated entries for zone affinities. (gardener/gardener#8050, @gardener-ci-robot)

🏃 Others

• [OPERATOR] The worker count for the NetworkPolicy controller in GRM was increased to 20. This is necessary to create and update NetworkPolicies in time, esp. on larger seed clusters. (gardener/gardener#8046, @gardener-ci-robot)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.71.4
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.71.4
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.71.4
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.71.4
operator: eu.gcr.io/gardener-project/gardener/operator:v1.71.4
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.71.4
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.71.4

[gardener]

🐛 Bug Fixes

• [OPERATOR] A bug has been fixed in the HighAvailabilityConfig-Webhook which caused duplicated entries for zone affinities. (gardener/gardener#8051, @timuthy)

🏃 Others

• [OPERATOR] The worker count for the NetworkPolicy controller in GRM was increased to 20. This is necessary to create and update NetworkPolicies in time, esp. on larger seed clusters. (gardener/gardener#8055, @timuthy)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.70.3
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.70.3
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.70.3
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.70.3
operator: eu.gcr.io/gardener-project/gardener/operator:v1.70.3
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.70.3
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.70.3

[gardener]

🐛 Bug Fixes

• [USER] Webhooks remediator sets the timeoutSeonds to 3 seconds for webhook affecting lease resources in kube-system namespace only if there is no objectSelector provided in webhook. (gardener/gardener#8045, @gardener-ci-robot)
• [OPERATOR] A bug has been fixed in the HighAvailabilityConfig-Webhook which caused duplicated entries for zone affinities. (gardener/gardener#8049, @gardener-ci-robot)

🏃 Others

• [OPERATOR] The worker count for the NetworkPolicy controller in GRM was increased to 20. This is necessary to create and update NetworkPolicies in time, esp. on larger seed clusters. (gardener/gardener#8044, @timuthy)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.72.1
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.72.1
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.72.1
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.72.1
operator: eu.gcr.io/gardener-project/gardener/operator:v1.72.1
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.72.1
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.72.1

[gardener]

⚠️ Breaking Changes

• [USER] The core/v1alpha1 API version is dropped. Make sure that you don't use the core/v1alpha1 API version in your machinery. (gardener/gardener#7965, @ary1992)
• [USER] The alpha.featuregates.shoot.gardener.cloud/apiserver-sni-pod-injector annotation has been dropped and is no longer available for Shoots. It should be removed from all existing Shoot resources. (gardener/gardener#7980, @rfranzke)
• [USER] Any resource with a kind other than ConfigMap or Secret in .spec.resources in Shoots is now forcefully removed. New validation has been introduced to prevent adding other resources in the future. (gardener/gardener#7995, @acumino)
• [USER] Webhooks remediator now sets the timeoutSeonds to 3 seconds for webhook affecting lease resources in kube-system namespace. (gardener/gardener#7902, @acumino)
• [OPERATOR] The core/v1alpha1 API version is dropped. Before upgrading to this version, make sure that there are no resources in the etcd stored in the core/v1alpha1 API version. Otherwise, the [email protected] will fail to start.
• [OPERATOR] ⚠️ Seeds' .spec.settings.ownerChecks.enabled field is locked to false (i.e. if the field value is true a validation error will be returned). Before updating to this version of Gardener, set .spec.settings.ownerChecks.enabled field to false for you Seeds and ManagedSeeds. (gardener/gardener#7909, @dimitar-kostadinov)
• [OPERATOR] It is required to have ControllerRegistrationss for Kinds ControlPlane, Infrastructure and Worker with the same types used for seeds (seed.spec.provider.type). This is already the case if seeds and shoots share the same cloud provider. The seed reconciliation flow waits for the associated ControllerInstallation to be ready before continuing rolling out seed system components. It allows Gardener provider extensions to ship components that not only act on shoot control-plane but also on seed system components. (gardener/gardener#7928, @timuthy)
• [DEPENDENCY] The {github.com/gardener/gardener/pkg/apis/core/helper,github.com/gardener/gardener/pkg/apis/core/v1beta1/helper}.SeedSettingOwnerChecksEnabled will now return false if the corresponding Seed setting is nil. Previously, the func was returning true when the Seed setting is nil. (gardener/gardener#7909, @dimitar-kostadinov)
• [DEPENDENCY] The unused github.com/gardener/gardener/pkg/controllerutils/predicate.IsBeingMigratedPredicate, github.com/gardener/gardener/pkg/controllerutils/predicate.IsObjectBeingMigrated and github.com/gardener/gardener/pkg/utils/gardener.IsObjectBeingMigrated funcs are now removed. (gardener/gardener#7909, @dimitar-kostadinov)

✨ New Features

• [USER] The certificate chains served by kube-apiservers does now include the CA certificates used to sign their server certificates. (gardener/gardener#7961, @rfranzke)
• [OPERATOR] gardener-operator configures SNI components in order to expose the virtual-garden-kube-apiserver via the istio-ingressgateway in the Garden cluster. (gardener/gardener#7953, @timuthy)
  • With this change, operators can start to switch DNS records from the virtual-garden-kube-apiserver service to the istio-ingress service endpoint. The type of the virtual-garden-kube-apiserver service will soon be switched from LoadBalancer to ClusterIP.
• [DEVELOPER] When performing control plane migration with provider-local, the full migration and restoration logic implemented in the extensions library (generic Worker actuator) is now executed (previously, it was skipped). This improves the accuracy of the e2e tests for control plane migration. (gardener/gardener#7981, @rfranzke)

🐛 Bug Fixes

• [USER] A bug that prevented referencing ConfigMaps in .spec.resources in Shoots has been fixed. (gardener/gardener#7995, @acumino)
• [USER] A bug that prevented finalizers from being added to referenced Secrets or ConfigMaps in .spec.resources in Shoots has been fixed. (gardener/gardener#7995, @acumino)
• [OPERATOR] The NetworkPolicy reconciler is only added to gardener-operator if the .spec.runtimeCluster.networking.{pods,services} fields of the Garden are set. (gardener/gardener#7983, @shafeeqes)
• [OPERATOR] Several low timeouts (30s) that were introduced in v1.71.0 for several steps are now reverted as in some cases the Network/ControlPlane reconciliation cannot succeed for 30s. (gardener/gardener#8005, @ialidzhikov)

🏃 Others

• [OPERATOR] A configuration issue that resulted in a relatively slow startup and termination of the vali pods is fixed. (gardener/gardener#7979, @istvanballok)
• [OPERATOR] Add new grafana dashboard of seed deployment replicas (gardener/gardener#7896, @Sallyan)

[dependency-watchdog]

🏃 Others

• [OPERATOR] More categories are added to label a release note for a PR on DWD. (gardener/dependency-watchdog#75, @himanshu-kun)
  • Release notifications would now be sent to gardener-dwd channel (private) on releases.
• [OPERATOR] Probes will not be created for shoots with no workers. (gardener/dependency-watchdog#82, @unmarshall)
• [OPERATOR] Fixes for make check target (gardener/dependency-watchdog#87, @unmarshall)
• [DEPENDENCY] Following dependencies are updated: (gardener/dependency-watchdog#84, @unmarshall)
  • Go - 1.20.3
  • client-go - v0.26.2
  • controller-runtime - v0.14.5
  • gomega - v1.27.1
  • zap - v1.24.0
  • gardener/gardener v1.69.0
  • k8s (api and apimachinery) - v0.26.2

[etcd-backup-restore]

🐛 Bug Fixes

• [OPERATOR] Fixes a bug in backup-restore which falsely detects scale-up scenario incase of rolling update of statefulset. (gardener/etcd-backup-restore#614, @ishan16696)

🏃 Others

• [OPERATOR] Base alpine image upgraded from 3.15.7 to 3.15.8 (gardener/etcd-backup-restore#612, @aaronfern)
• [OPERATOR] Add a learner with backoff in case of scale-up feature is triggered. (gardener/etcd-backup-restore#617, @ishan16696)
• [OPERATOR] Added a safety check before adding a learner(non-voting) member in etcd cluster. (gardener/etcd-backup-restore#605, @ishan16696)
• [DEVELOPER] Upgrade to go 1.20.3 (gardener/etcd-backup-restore#613, @shreyas-s-rao)
• [DEVELOPER] Block public access for S3 buckets created by integration tests. (gardener/etcd-backup-restore#615, @shreyas-s-rao)

[etcd-custom-image]

🏃 Others

• [OPERATOR] Base alpine image for etcd-custom-image upgraded from 3.15.7 to 3.15.8 (gardener/etcd-custom-image#32, @aaronfern)

[etcd-druid]

✨ New Features

• [DEVELOPER] Run make ci-e2e-kind to run the e2e tests on local machine (gardener/etcd-druid#547, @abdasgupta)
• [DEVELOPER] Eliminated Role helm charts and converted into Golang component with added unit tests. (gardener/etcd-druid#538, @seshachalam-yv)
• [DEVELOPER] Eliminated RoleBinding helm charts and converted into Golang component with added unit tests. (gardener/etcd-druid#539, @seshachalam-yv)

🐛 Bug Fixes

• [OPERATOR] Added check to ensure that the scale up annotation is removed from the etcd statefulset only when scale-up succeeds (gardener/etcd-druid#587, @ishan16696)

🏃 Others

• [OPERATOR] When scaling from single-node to multi-node etcd cluster, Etcd Druid will now first ensure that any change to the peer URL (e.g TLS enablement) is seen by the existing etcd process running within the etcd member pod. Once that is confirmed then it will scale up the Etcd StatefulSet and add relevant annotations. (gardener/etcd-druid#598, @unmarshall)
• [OPERATOR] Backup-restore waits for its etcd to be ready before attempting to update peerUrl (gardener/etcd-druid#602, @abdasgupta)
• [OPERATOR] When scaling from single-node to multi-node etcd cluster, Etcd Druid will now first ensure that any change to the peer URL (e.g TLS enablement) is seen by the existing etcd process running within the etcd member pod. Once that is confirmed then it will scale up the Etcd StatefulSet and add relevant annotations. (gardener/etcd-druid#602, @abdasgupta)
• [OPERATOR] etcd-custom-image updates from v3.4.13-bootstrap-9 to v3.4.13-bootstrap-10 (gardener/etcd-druid#575, @aaronfern)
• [DEVELOPER] Upgrade to go 1.20.3. (gardener/etcd-druid#579, @shreyas-s-rao)
• [DEVELOPER] Block public access for S3 buckets created by e2e tests. (gardener/etcd-druid#581, @shreyas-s-rao)

[logging]

🐛 Bug Fixes

• [OPERATOR] Remove lastOperation check in fluent-bit-to-vali plugin. (gardener/logging#197, @vlvasilev)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.72.0
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.72.0
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.72.0
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.72.0
operator: eu.gcr.io/gardener-project/gardener/operator:v1.72.0
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.72.0
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.72.0

[gardener]

🐛 Bug Fixes

• [OPERATOR] A bug causing gardenlet to panic when admission-controller is upgraded to v1.71 but gardenlet is still on v1.70. (gardener/gardener#7989, @acumino)
• [OPERATOR] Several low timeouts (30s) that were introduced in v1.71.0 for several steps are now reverted as in some cases the Network/ControlPlane reconciliation cannot succeed for 30s. (gardener/gardener#8006, @gardener-ci-robot)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.71.3
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.71.3
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.71.3
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.71.3
operator: eu.gcr.io/gardener-project/gardener/operator:v1.71.3
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.71.3
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.71.3

[gardener]

🐛 Bug Fixes

• [OPERATOR] The NetworkPolicy reconciler is only added to gardener-operator if the .spec.runtimeCluster.networking.{pods,services} fields of the Garden are set. (gardener/gardener#7986, @gardener-ci-robot)

🏃 Others

• [OPERATOR] A configuration issue that resulted in a relatively slow startup and termination of the vali pods is fixed. (gardener/gardener#7982, @gardener-ci-robot)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.71.2
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.71.2
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.71.2
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.71.2
operator: eu.gcr.io/gardener-project/gardener/operator:v1.71.2
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.71.2
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.71.2

[logging]

🐛 Bug Fixes

• [OPERATOR] Remove lastOperation check in fluent-bit-to-vali plugin. (gardener/logging#197, @vlvasilev)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.71.1
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.71.1
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.71.1
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.71.1
operator: eu.gcr.io/gardener-project/gardener/operator:v1.71.1
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.71.1
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.71.1

[gardener]

⚠️ Breaking Changes

• [USER] Gardener denies setting Shoot.Spec.ControlPlane.HighAvailability.FailureTolerance.Type if shoot is hibernated. (gardener/gardener#7922, @gardener-ci-robot)

🐛 Bug Fixes

• [USER] A bug has been fixed which could cause kube-proxys from being missing after a Shoot has been woken up from hibernation. (gardener/gardener#7919, @gardener-ci-robot)

🏃 Others

• [OPERATOR] An issue causing panic in the health check for extension is fixed. (gardener/gardener#7916, @gardener-ci-robot)
• [OPERATOR] The nested kubelet in the Gardener e2e tests (in prow/kind) now work on hosts using cgroupsv2 (gardener/gardener#7800, @gardener-ci-robot)
• [DEVELOPER] Set cgroupDriver of provider-local to systemd. (gardener/gardener#7805, @gardener-ci-robot)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.68.1
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.68.1
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.68.1
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.68.1
operator: eu.gcr.io/gardener-project/gardener/operator:v1.68.1
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.68.1
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.68.1

[gardener]

⚠️ Breaking Changes

• [USER] Since Namespaces are no longer deleted (and forcefully finalized after some grace period), the shoot.gardener.cloud/cleanup-namespaces-finalize-grace-period-seconds annotation does no longer have any effect. Relevant Kubernetes resources are still cleaned up (see this document) for more information. (gardener/gardener#7864, @rfranzke)
• [USER] Using internal API versions in providerConfig fields is no longer permitted (deprecated since more than 2y). Ensure that you always use a versioned API. (gardener/gardener#7868, @rfranzke)
• [USER] As of Kubernetes v1.27, Gardener enforces a worker.maximum configuration for system component worker pools. The value must be greater or equal to the number of zones configured for this pool. This ensures, that the pool has the minimum required nodes to schedule system component across nodes. (gardener/gardener#7878, @timuthy)
• [USER] The static token kubeconfig can no longer be enabled for Shoot clusters using Kubernetes version 1.27 and higher. (gardener/gardener#7883, @ary1992)
• [USER] For Shoot clusters using Kubernetes version 1.27 and higher, the .spec.kubernetes.kubeControllerManager.podEvictionTimeout field has no effect anymore since the backing --pod-eviction-timeout CLI flag has been removed. (gardener/gardener#7883, @ary1992)
• [USER] ⚠️ The deprecated field .spec.kubernetes.kubeAPIServer.enableBasicAuthentication has been removed from the Shoot API. Please check your Shoots manifests and remove the .spec.kubernetes.kubeAPIServer.enableBasicAuthentication field. (gardener/gardener#7886, @dimitar-kostadinov)
• [USER] Gardener denies setting Shoot.Spec.ControlPlane.HighAvailability.FailureTolerance.Type if shoot is hibernated. (gardener/gardener#7894, @aaronfern)
• [OPERATOR] All fluent-bit-related configuration options have been removed from gardenlet's component configuration. (gardener/gardener#7568, @Kristian-ZH)
• [OPERATOR] The FullNetworkPoliciesInRuntimeCluster feature gate has been promoted to beta and is now turned on by default. Before deploying this Gardener version, make sure that all your registered extensions support this feature gate. (gardener/gardener#7866, @rfranzke)
• [OPERATOR] The HAControlPlanes feature gate has been promoted to beta and is now turned on by default. (gardener/gardener#7867, @timuthy)
• [OPERATOR] The deprecated allow-{to,from}-shoot-apiserver NetworkPolicys have been dropped. Ensure that all registered extensions have been adapted. (gardener/gardener#7868, @rfranzke)
• [OPERATOR] The deprecated identity value is no longer passed when ControllerInstallation Helm charts are deployed. (gardener/gardener#7868, @rfranzke)
• [OPERATOR] The lastUpdateTime of extension conditions is no longer considered. Ensure that all registered extensions populate the lastHeartbeatTime field instead. (gardener/gardener#7868, @rfranzke)
• [DEVELOPER] The pkg/operation/botanist/component/* resources have been moved to pkg/component/*. (gardener/gardener#7938, @rfranzke)
• [DEVELOPER] gardenlet will no longer respect ConfigMaps labeled with extensions.gardener.cloud/configuration=logging. The way to deploy a new filter or parser configuration is to create ClusterFilters or ClusterParsers custom resources in the seed cluster. (gardener/gardener#7568, @Kristian-ZH)
• [DEVELOPER] Extensions vendoring this gardener/gardener version need to provide RBAC privileges for PATCH apps/depoyments/scale. (gardener/gardener#7868, @rfranzke)
• [DEPENDENCY] Extensions that wish to be scraped by the seed-prometheus must annotate their pods with prometheus.io/scrape=true along with prometheus.io/name=<name>. See https://github.com/gardener/gardener/blob/master/docs/monitoring/README.md#seed-prometheus for more details. (gardener/gardener#7885, @shafeeqes)

✨ New Features

• [USER] It is possible now to create a workerless shoot cluster when the WorkerlessShoots feature gate in the gardener-apiserver is enabled. Please see this document for more details. (gardener/gardener#7882, @shafeeqes)
• [OPERATOR] fluent-operator is now installed in the garden namespace of seed clusters and will take care of the entire lifecycle of the fluent-bit DaemonSet. (gardener/gardener#7568, @Kristian-ZH)
• [OPERATOR] The gardener-operator now enables full NetworkPolicy protection for the garden cluster. In case your garden cluster is a seed at the same time, make sure to keep the values of the FullNetworkPoliciesInRuntimeCluster feature gate in sync for both gardener-operator and gardenlet. (gardener/gardener#7859, @rfranzke)
• [OPERATOR] gardenlet and gardener-operator managed deployments and statefulsets can now be equipped with toleration seconds for taints node.kubernetes.io/not-ready and node.kubernetes.io/unreachable. (gardener/gardener#7861, @timuthy)
  • Please consult the respective component config examples (gardenlet, gardener-operator) for more information.
• [OPERATOR] The gardenlet and gardener-operator Helm charts allow to define toleration seconds for node.kubernetes.io/not-ready and node.kubernetes.io/unreachable. This configuration considered for their own Deployment as well as the Gardenlet's or Operator's config. The values are set to 60s by default. (gardener/gardener#7861, @timuthy)
• [OPERATOR] An optional field workerlessSupported is added under spec.resources in the ControllerRegistration API. (gardener/gardener#7863, @ary1992)
• [OPERATOR] gardener-operator is now managing the gardener-resource-manager instance as part of the virtual garden cluster control plane. It provides a TokenRequest API-based kubeconfig for gardener-operator to access the virtual garden cluster. The static token kubeconfig is now unconditionally disabled. (gardener/gardener#7881, @oliver-goetz)
• [OPERATOR] It is now possible to provide namespace selectors for additional namespaces which should be covered by the NetworkPolicy controllers of gardener-operator or gardenlet. The selectors must be provided via their component configs. Please consult this document for further insights. (gardener/gardener#7929, @rfranzke)
• [OPERATOR] gardener-operator is now managing the kube-controller-manager instance as part of the virtual garden cluster control plane. (gardener/gardener#7931, @rfranzke)
• [DEVELOPER] In order to allow kube-apiserver pods of shoot or garden clusters to reach webhook servers, they must no longer be explicitly labeled with networking.resources.gardener.cloud/to-<service-name>-<protocol>-<port>=allowed. Instead, it is enough to annotate the Service of the webhook server with networking.resources.gardener.cloud/from-all-webhook-targets-allowed-ports=<ports>. (gardener/gardener#7907, @rfranzke)
• [DEPENDENCY] To support workerless Shoots, extensions reconciling extensions.gardener.cloud/v1alpha1.Extension resources need to make adaptions if needed and then set spec.resources[].workerlessSupported to true in the ControllerRegistration for their respective extension type. (gardener/gardener#7863, @ary1992)

🐛 Bug Fixes

• [USER] An issue has been fixed which might have caused the deletion of Shoot clusters to stuck when a namespace was forcefully removed before all relevant resources have been cleaned up. (gardener/gardener#7864, @rfranzke)
• [USER] A bug has been fixed which could cause kube-proxys from being missing after a Shoot has been woken up from hibernation. (gardener/gardener#7912, @rfranzke)
• [OPERATOR] An issue causing VPN Seed (CPU| Memory) Usage dashboards not showing data is now fixed. (gardener/gardener#7865, @Sallyan)
• [OPERATOR] A bug has been fixed which prevented components using the networking.resources.gardener.cloud/from-world-to-ports annotation from being reached from internal IP addresses when the cluster was using Cilium as CNI. (gardener/gardener#7884, @ScheererJ)
• [OPERATOR] A bug which was causing race conditions to occur during reconciliation of extension resources was fixed. (gardener/gardener#7906, @dimityrmirchev)
• [OPERATOR] An issue causing panic in the health check for extension, when the health check result is empty, is fixed. (gardener/gardener#7908, @acumino)
• [OPERATOR] An issue has been fixed that caused traffic from outside of the cluster to Istio-Ingress being blocked. This is only relevant if seed(s) specify additional load balancer annotations via seed.spec.settings.loadBalancerServices.annotations. (gardener/gardener#7910, @timuthy)

📖 Documentation

• [DEVELOPER] A guideline for developers regarding TODO statements has been introduced. (gardener/gardener#7939, @rfranzke)

🏃 Others

• [USER] The --node-monitor-grace-period flag of kube-controller-manager is now defaulted to 40s for Shoot clusters using Kubernetes version 1.27 and higher. (gardener/gardener#7883, @ary1992)
• [USER] The following images are updated: (gardener/gardener#7897, @himanshu-kun)
  • eu.gcr.io/gardener-project/gardener/autoscaler/cluster-autoscaler: v1.21.5 -> v1.21.6 (for Kubernetes 1.21)
  • eu.gcr.io/gardener-project/gardener/autoscaler/cluster-autoscaler: v1.22.5 -> v1.22.6 (for Kubernetes 1.22)
  • eu.gcr.io/gardener-project/gardener/autoscaler/cluster-autoscaler: v1.23.3 -> v1.23.4 (for Kubernetes 1.23)
  • eu.gcr.io/gardener-project/gardener/autoscaler/cluster-autoscaler: v1.24.2 -> v1.24.3 (for Kubernetes 1.24)
  • eu.gcr.io/gardener-project/gardener/autoscaler/cluster-autoscaler: v1.25.2 -> v1.25.3 (for Kubernetes 1.24)
  • eu.gcr.io/gardener-project/gardener/autoscaler/cluster-autoscaler: v1.26.1 -> v1.26.2 (for Kubernetes 1.26)
• [OPERATOR] Default log level in fluent-bit is changed from info to error (gardener/gardener#7942, @nickytd)
• [OPERATOR] Grafana and Loki are replaced with the fork of their last Apache 2.0 licensed releases: Plutono and Vali, that will continue to receive security updates. (gardener/gardener#7318, @istvanballok)
• [OPERATOR] The following image is updated: (gardener/gardener#7892, @rickardsjp)
  • quay.io/prometheus/prometheus: v2.41.0 -> v2.43.1
• [OPERATOR] nginx-ingress-controller-seed image is updated to v1.7.1 for 1.24.x+ seeds. (gardener/gardener#7904, @shafeeqes)
• [OPERATOR] Allow the kubelet configuration to define swap behaviour {LimitedSwap / UnlimitedSwap} for k8s >= 1.22 (gardener/gardener#7913, @danielfoehrKn)
• [OPERATOR] Updated cluster-proportional-autoscaler to v1.8.8 (gardener/gardener#7927, @ScheererJ)
• [OPERATOR] The gardenlet and the gardener-operator will now use the new service.kubernetes.io/topology-mode=auto annotation when enabling topology-aware routing for a Service when the Kubernetes version of the runtime cluster is >= 1.27. In Kubernetes 1.27, the service.kubernetes.io/topology-aware-hints=auto annotation is deprecated in favor of the newly introduced service.kubernetes.io/topology-mode=auto (gardener/gardener#7933, @ialidzhikov)
• [DEVELOPER] The check-apidiff check was changed to only report incompatible and critical changes which need inspection from the developer's side. (gardener/gardener#7936, @timuthy)
• [DEVELOPER] The networking.resources.gardener.cloud/from-policy-pod-label-selector and networking.resources.gardener.cloud/from-policy-allowed-ports annotations are now deprecated and will be removed in the future. Use networking.resources.gardener.cloud/from-<pod-label-selector>-allowed-ports=<ports> instead. (gardener/gardener#7907, @rfranzke)
• [DEPENDENCY] Shoot addon nginx-ingress-controller image is updated to v1.3.0 for v1.22+ shoots. (gardener/gardener#7932, @shafeeqes)

[apiserver-proxy]

🏃 Others

• [OPERATOR] Use admission v1 instead of v1beta1 for apiserver-proxy webhook. (gardener/apiserver-proxy#35, @ScheererJ)
• [OPERATOR] Fix verification. (gardener/apiserver-proxy#38, @axel7born)

📰 Noteworthy

• [OPERATOR] Update golang 1.19.5 -> 1.20.4 (gardener/apiserver-proxy#37, @axel7born)

[ext-authz-server]

✨ New Features

• [OPERATOR] Update golang 1.19.5 -> 1.20.4 (gardener/ext-authz-server#19, @axel7born)

[logging]

🏃 Others

• [OPERATOR] Prevent fluent-bit-to-vali plugin panic when Cluster is updated and its Shoot has no lastOperation set (gardener/logging#192, @vlvasilev)
• [OPERATOR] Improves client recreate during cluster reconcile. (gardener/logging#195, @nickytd)
• [OPERATOR] Update k8s.io/client-go from v0.17.0 to v0.26.2 (gardener/logging#188, @vlvasilev)
• [OPERATOR] Updated golang container image build version to 1.20.4 (gardener/logging#190, @nickytd)
• [DEVELOPER] This PR aligns container build targets with project CI supporting multi-platform builds and simplifies overall Makefile structure. (gardener/logging#189, @nickytd)

[vpn2]

🏃 Others

• [OPERATOR] Bump builder image golang from 1.20.2 to 1.20.4 (gardener/vpn2#32, @MartinWeindel)
• [OPERATOR] Bump builder image golang from 1.19.5 to 1.20.2 (gardener/vpn2#30, @MartinWeindel)
• [OPERATOR] Bump alpine base image from 1.16.3 to 1.16.5 (gardener/vpn2#30, @MartinWeindel)
• [OPERATOR] Updated kubernetes dependencies from 1.25.0 to 1.26.2 (gardener/vpn2#30, @MartinWeindel)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.71.0
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.71.0
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.71.0
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.71.0
operator: eu.gcr.io/gardener-project/gardener/operator:v1.71.0
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.71.0
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.71.0

[gardener]

⚠️ Breaking Changes

• [USER] Gardener denies setting Shoot.Spec.ControlPlane.HighAvailability.FailureTolerance.Type if shoot is hibernated. (gardener/gardener#7921, @gardener-ci-robot)

🐛 Bug Fixes

• [USER] A bug has been fixed which could cause kube-proxys from being missing after a Shoot has been woken up from hibernation. (gardener/gardener#7918, @gardener-ci-robot)

🏃 Others

• [OPERATOR] An issue causing panic in the health check for extension is fixed. (gardener/gardener#7915, @gardener-ci-robot)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.69.3
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.69.3
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.69.3
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.69.3
operator: eu.gcr.io/gardener-project/gardener/operator:v1.69.3
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.69.3
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.69.3

[gardener]

⚠️ Breaking Changes

• [USER] Gardener denies setting Shoot.Spec.ControlPlane.HighAvailability.FailureTolerance.Type if shoot is hibernated. (gardener/gardener#7920, @gardener-ci-robot)

🐛 Bug Fixes

• [USER] A bug has been fixed which could cause kube-proxys from being missing after a Shoot has been woken up from hibernation. (gardener/gardener#7917, @gardener-ci-robot)
• [OPERATOR] An issue has been fixed that caused traffic from outside of the cluster to Istio-Ingress being blocked. This is only relevant if seed(s) specify additional load balancer annotations via seed.spec.settings.loadBalancerServices.annotations. (gardener/gardener#7911, @gardener-ci-robot)

🏃 Others

• [OPERATOR] An issue causing panic in the health check for extension is fixed. (gardener/gardener#7914, @gardener-ci-robot)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.70.2
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.70.2
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.70.2
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.70.2
operator: eu.gcr.io/gardener-project/gardener/operator:v1.70.2
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.70.2
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.70.2

[gardener]

🐛 Bug Fixes

• [OPERATOR] A bug has been fixed which prevented components using the networking.resources.gardener.cloud/from-world-to-ports annotation from being reached from internal IP addresses when the cluster was using Cilium as CNI. (gardener/gardener#7890, @gardener-ci-robot)

🏃 Others

• [USER] The following images are updated: (gardener/gardener#7900, @gardener-ci-robot)
  • eu.gcr.io/gardener-project/gardener/autoscaler/cluster-autoscaler: v1.21.5 -> v1.21.6 (for Kubernetes 1.21)
  • eu.gcr.io/gardener-project/gardener/autoscaler/cluster-autoscaler: v1.22.5 -> v1.22.6 (for Kubernetes 1.22)
  • eu.gcr.io/gardener-project/gardener/autoscaler/cluster-autoscaler: v1.23.3 -> v1.23.4 (for Kubernetes 1.23)
  • eu.gcr.io/gardener-project/gardener/autoscaler/cluster-autoscaler: v1.24.2 -> v1.24.3 (for Kubernetes 1.24)
  • eu.gcr.io/gardener-project/gardener/autoscaler/cluster-autoscaler: v1.25.2 -> v1.25.3 (for Kubernetes 1.24)
  • eu.gcr.io/gardener-project/gardener/autoscaler/cluster-autoscaler: v1.26.1 -> v1.26.2 (for Kubernetes 1.26)
• [OPERATOR] Fixed an issue that would cause the gardenlet to run into CrashLoopBackoff when following the docs/development/getting_started_locally.md#remote-local-setup guide. (gardener/gardener#7843, @gardener-ci-robot)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.69.2
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.69.2
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.69.2
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.69.2
operator: eu.gcr.io/gardener-project/gardener/operator:v1.69.2
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.69.2
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.69.2