[gardener]

🐛 Bug Fixes

• [OPERATOR] A bug in the sshd-ensurer script running on all shoot worker nodes has been fixed which was causing it to also kill processes other than sshd when SSHAccess for worker nodes is disabled. (gardener/gardener#7427, @gardener-ci-robot)
• [OPERATOR] Fixed bug that cause HA VPN to fail in case the seed's apiserver was not targeted by kube-apiserver's vpn-client via a public address (gardener/gardener#7465, @MartinWeindel)
• [OPERATOR] Fix a bug in the etcd deploy flow that erroneously unsets etcd.Spec.Etcd.PeerUrlTls in the ETCD CRs of high available shoots when marked for hibernation. (gardener/gardener#7532, @gardener-ci-robot)
  • Before this change, high availability clusters failed to be deleted while being hibernated.
• [OPERATOR] An issue has been fixed that caused hibernated shoots with HA control-planes being stuck in deletion. (gardener/gardener#7565, @gardener-ci-robot)
• [DEVELOPER] The Gardener upgrade tests have been updated to use the previous minor version of Gardener instead of the latest release tag when the environment variable GARDENER_PREVIOUS_RELEASE is not specified. (gardener/gardener#7500, @gardener-ci-robot)

🏃 Others

• [USER] The PodSecurity kube-apiserver admission plugin config in the Shoot, if provided, is now validated. (gardener/gardener#7486, @gardener-ci-robot)
• [OPERATOR] An issue causing a nil pointer error in the seed-lifecycle controller is fixed. (gardener/gardener#7544, @gardener-ci-robot)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.63.2
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.63.2
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.63.2
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.63.2
operator: eu.gcr.io/gardener-project/gardener/operator:v1.63.2
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.63.2
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.63.2

[gardener]

⚠️ Breaking Changes

• [USER] The core.gardener.cloud/v1alpha1 API is deprecated and will be removed soon. The core.gardener.cloud/v1beta1 API is already available since a very long time and should be used instead. (gardener/gardener#7443, @ary1992)
• [USER] Support for shoot annotation alpha.control-plane.shoot.gardener.cloud/high-availability has been dropped. Existing shoot clusters have already been migrated to the respective spec fields since Gardener v1.60.0. Starting with this release, the annotation is not respected and the migration will not happen anymore. Please make sure to use shoot.spec.controlPlane.highAvailability.failureTolerance: {node, zone} instead. (gardener/gardener#7493, @timuthy)
• [OPERATOR] Before upgrading to this Gardener version, Seeds using .spec.dns.ingressDomain must now finally be switched to using .spec.ingress and .spec.dns.provider (as changed with https://github.com/gardener/gardener/pull/3131 back in 2020). Please find more information about it here. The .spec.dns.ingressDomain field is deprecated since more than 2 years and will be removed in a future version. (gardener/gardener#7515, @rfranzke)
• [DEPENDENCY] Extensions which deploy components that need to be scraped by the Prometheis in the shoot namespaces need to adapt to the new NetworkPolicys. For more information, read this section. (gardener/gardener#7484, @rfranzke)
• [DEPENDENCY] Extensions which deploy components to shoot namespaces need to adapt to the new NetworkPolicys. Concretely, the following labels related to NetworkPolicies are deprecated and should be replaced: (gardener/gardener#7515, @rfranzke)
  • networking.gardener.cloud/to-shoot-apiserver=allowed, replace it with networking.resources.gardener.cloud/to-kube-apiserver-tcp-443=allowed.
  • networking.gardener.cloud/from-shoot-apiserver=allowed, replace it with the label networking.resources.gardener.cloud/to-<service-name>-tcp-<container-port>=allowed on kube-apiserver pods.

✨ New Features

• [USER] A taint is added to all Node objects on registration by the kubelet. Gardener removes the taint once all node-critical pods are ready. This makes sure that user workload is only scheduled to nodes where all node-critical components are ready. Please refer to the documentation for more details. (gardener/gardener#7406, @timebertt)
• [DEVELOPER] Now by default, Gardener performs health check for all the ManagedResources with .spec.class=nil created in the shoot namespaces. Extensions using Gardener v1.65.0 onwards can drop the health check for the MangedResource. (gardener/gardener#7462, @acumino)
• [DEVELOPER] Extensions can label node-critical pods that they manage with node.gardener.cloud/critical-component=true to ensure user workload is only scheduled to nodes where all node-critical components are ready. Please refer to the documentation for more details. (gardener/gardener#7406, @timebertt)
• [DEPENDENCY] The goimports-reviser is updated to a version that properly ignores generated files. (gardener/gardener#7492, @vpnachev)

🐛 Bug Fixes

• [OPERATOR] Fix a bug in the etcd deploy flow that erroneously unsets etcd.Spec.Etcd.PeerUrlTls in the ETCD CRs of high available shoots when marked for hibernation. (gardener/gardener#7514, @aaronfern)
  • Before this change, high availability clusters failed to be deleted while being hibernated.
• [OPERATOR] An issues has been fixed that caused outdated Envoy stats filters not being cleaned up in Istio-Ingress namespaces. (gardener/gardener#7397, @timuthy)
• [DEVELOPER] The Gardener upgrade tests have been updated to use the previous minor version of Gardener instead of the latest release tag when the environment variable GARDENER_PREVIOUS_RELEASE is not specified. (gardener/gardener#7491, @seshachalam-yv)

🏃 Others

• [USER] The PodSecurity kube-apiserver admission plugin config in the Shoot, if provided, is now validated. (gardener/gardener#7472, @shafeeqes)
• [OPERATOR] Fluent-bit daemon set memory limit increased to 650MB and request to 200MB. (gardener/gardener#7564, @vlvasilev)
• [OPERATOR] The ExposureClass and ShootState resources have been promoted to v1beta1. (gardener/gardener#7443, @ary1992)
• [OPERATOR] Add response rewrite to dns-search-path-optimization, as some clients require matching hostnames in a DNS query and the answer. (gardener/gardener#7478, @axel7born)
• [OPERATOR] nginx-ingress-controller-seed image is updated to v1.6.4 for 1.23+ seeds and v1.4.0 for 1.22.x seeds. (gardener/gardener#7490, @shafeeqes)
• [OPERATOR] Remove limit defaults from helm charts for controlplane components (gardener/gardener#7494, @voelzmo)
• [OPERATOR] The resource-manager now recreates immutable Secrets/ConfigMaps on invalid update error. (gardener/gardener#7516, @shafeeqes)
• [OPERATOR] Loki user tenant is removed. (gardener/gardener#7523, @vlvasilev)
• [OPERATOR] An issue causing a nil pointer error in the seed-lifecycle controller is fixed. (gardener/gardener#7539, @acumino)
• [DEVELOPER] The Shoot creation integration test now saves the kubeconfig obtained from the shoot/adminkubeconfig to $TM_KUBECONFIG_PATH/shoot.config. Previously, it was saving the static token kubeconfig. (gardener/gardener#7495, @ialidzhikov)
• [DEVELOPER] golangci-lint has been updated to v1.51.2. (gardener/gardener#7537, @vpnachev)
• [DEVELOPER] Update to Go 1.19.6. (gardener/gardener#7542, @oliver-goetz)
• [DEPENDENCY] hack/format.sh now can run goimports-reviser with custom options set via the environment variable GOIMPORTS_REVISER_OPTIONS. (gardener/gardener#7502, @vpnachev)

[etcd-backup-restore]

🐛 Bug Fixes

• [OPERATOR] Fixes bug of false wrong annotation added to etcd-member lease of TLS not enabled. (gardener/etcd-backup-restore#564, @ishan16696)

🏃 Others

• [USER] Better error message if setting in etcd config is missing (gardener/etcd-backup-restore#582, @mxmxchere)
• [USER] Update alpine base image to 3.15.7. (gardener/etcd-backup-restore#590, @shreyas-s-rao)
• [OPERATOR] making chunk-size configurable by introducing flag: --min-chunk-size (default value 5MB), it will be helpful in fine tuning the multi-part chunk upload size for different storage provider. (gardener/etcd-backup-restore#545, @louisportay)
• [OPERATOR] Removed owner checks that were used to restart the etcd process that runs in the source Seed cluster during "bad case" control plane migration. (gardener/etcd-backup-restore#555, @plkokanov)
• [OPERATOR] Enhances the decision to take full snapshot during startup of etcd-backup-restore to avoid missing of any full-snapshot. (gardener/etcd-backup-restore#574, @ishan16696)

📰 Noteworthy

• [OPERATOR] Added support for Application credentials to authenticate Openstack client for Openstack backup buckets. (gardener/etcd-backup-restore#580, @ishan16696)
• [OPERATOR] Update golang version for Docker image build to v1.19.3. (gardener/etcd-backup-restore#561, @ishan16696)
• [DEVELOPER] Update golang build version to 1.19.5. (gardener/etcd-backup-restore#590, @shreyas-s-rao)
• [DEVELOPER] Update golang version for dependency vendoring to v1.19. (gardener/etcd-backup-restore#561, @ishan16696)

[etcd-druid]

✨ New Features

• [OPERATOR] Enhance kubectl printer columns for Etcd resource. (gardener/etcd-druid#490, @shreyas-s-rao)

🏃 Others

• [USER] Explicitly set logging options to use JSON logging and ISO8601 timestamp format. (gardener/etcd-druid#525, @shreyas-s-rao)
• [OPERATOR] --etcd-process-name has been deprecated and is now not added to the statefulset (gardener/etcd-druid#514, @aaronfern)
• [OPERATOR] The Etcd resource now allows specify etcd client Service labels via the spec.etcd.clientService.labels field. (gardener/etcd-druid#485, @ialidzhikov)
• [OPERATOR] Removed ability to set owner checks that were used to restart the etcd process that runs in the source Seed cluster during "bad case" control plane migration. (gardener/etcd-druid#461, @plkokanov)
• [DEVELOPER] Update golang build version to v1.19.4. (gardener/etcd-druid#495, @shreyas-s-rao)
• [DEPENDENCY] Dependency github.com/gardener/gardener is updated v1.36.0 -> v1.57.1 (gardener/etcd-druid#450, @AleksandarSavchev)
• [DEPENDENCY] Dependency github.com/onsi/ginkgo is upgraded to github.com/onsi/ginkgo/v2 (gardener/etcd-druid#450, @AleksandarSavchev)
• [DEPENDENCY] The dependency of sigs.k8s.io/controller-runtime/pkg/envtest/printer package in etcd-druid is removed. (gardener/etcd-druid#493, @shafeeqes)

[logging]

🏃 Others

• [OPERATOR] Loki label docker_id is replaced by container_id. (gardener/logging#172, @vlvasilev)
• [OPERATOR] Logging Gardener-specific multi-tenancy can be switched off by EnableMultiTenancy. (gardener/logging#172, @vlvasilev)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.65.0
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.65.0
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.65.0
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.65.0
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.65.0
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.65.0
operator: eu.gcr.io/gardener-project/gardener/operator:v1.65.0

[gardener]

🐛 Bug Fixes

• [OPERATOR] Fix a bug in the etcd deploy flow that erroneously unsets etcd.Spec.Etcd.PeerUrlTls in the ETCD CRs of high available shoots when marked for hibernation. (gardener/gardener#7531, @gardener-ci-robot)
  • Before this change, high availability clusters failed to be deleted while being hibernated.
• [DEVELOPER] The Gardener upgrade tests have been updated to use the previous minor version of Gardener instead of the latest release tag when the environment variable GARDENER_PREVIOUS_RELEASE is not specified. (gardener/gardener#7499, @gardener-ci-robot)

🏃 Others

• [USER] The PodSecurity kube-apiserver admission plugin config in the Shoot, if provided, is now validated. (gardener/gardener#7487, @gardener-ci-robot)
• [OPERATOR] An issue causing a nil pointer error in the seed-lifecycle controller is fixed. (gardener/gardener#7545, @gardener-ci-robot)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.64.1
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.64.1
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.64.1
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.64.1
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.64.1
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.64.1
operator: eu.gcr.io/gardener-project/gardener/operator:v1.64.1

[gardener]

🐛 Bug Fixes

• [OPERATOR] Fixed bug that cause HA VPN to fail in case the seed's apiserver was not targeted by kube-apiserver's vpn-client via a public address (gardener/gardener#7466, @MartinWeindel)
• [OPERATOR] A bug has been fixed which caused the conditions of Shoots to be set to Unknown too fast in case the responsible gardenlet is no longer posting its heartbeat. (gardener/gardener#7414, @gardener-ci-robot)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.62.3
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.62.3
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.62.3
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.62.3
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.62.3
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.62.3
operator: eu.gcr.io/gardener-project/gardener/operator:v1.62.3

[gardener]

⚠️ Breaking Changes

• [OPERATOR] The rootcapublisher controller has been dropped from gardener-resource-manager since Gardener no longer supports Kubernetes clusters below v1.20. For higher versions, the controller is no longer needed. (gardener/gardener#7367, @rfranzke)
• [OPERATOR] The configuration for gardenlet's NetworkPolicy controller must now be submitted via .controllers.networkPolicy instead of controllers.seedAPIServerNetworkPolicy. (gardener/gardener#7389, @rfranzke)
• [OPERATOR] The deprecated field seed.spec.settings.shootDNS has been removed from the Seed API. Please check your Seeds and remove any usage before upgrading to this Gardener version. (gardener/gardener#7399, @acumino)
• [DEPENDENCY] The allow-to-seed-apiserver NetworkPolicy is now deprecated and replaced by the new allow-to-runtime-apiserver NetworkPolicy. Components running in the seed cluster that need to talk to the kube-apiserver should be labeled with networking.gardener.cloud/to-runtime-apiserver=allowed. (gardener/gardener#7389, @rfranzke)

✨ New Features

• [OPERATOR] Enabling the new feature gate MutableShootSpecNetworkingNodes in gardener-apiserver makes .spec.networking.nodes in Shoots mutable. This allows increasing the network space for nodes. (gardener/gardener#7368, @axel7born)
• [OPERATOR] The istio-system namespace in seed clusters is now labeled with gardener.cloud/role=istio-system. All istio-ingress* namespaces are now labeled with gardener.cloud/role=istio-ingress. (gardener/gardener#7389, @rfranzke)
• [OPERATOR] Shoot owners have the same access to Grafana and Prometheus as Gardener operators. (gardener/gardener#7007, @istvanballok)
  • Previously, Gardener used to manage two Grafana deployments in the control plane of a shoot cluster. One for shoot owners and another for Gardener operators. Now there is only a single Grafana deployment in the control plane.
  • Previously, the shoot owner Grafana was configured in a fairly restricted way. Shoot owners can now access the Explore feature of Grafana to browse observability data in Prometheus and Loki with interactive queries. They can inspect the definition of dashboard panels and perform temporary edits to facilitate interactive investigations.
  • Previously, shoot owners could not access Prometheus that was restricted to Gardener operators. Now shoot owners can access the Prometheus UI as well to facilitate interactive investigations.
  • Shoot owners can programmatically access the Prometheus API to federate control plane metrics.
• [DEVELOPER] make start-envtest brings up a test environment that can simplify debugging of integration tests. See the docs for more information. (gardener/gardener#7431, @timebertt)
• [DEVELOPER] Add bootstrapping a local IPv6 KinD cluster with make kind-up IPFAMILY=ipv6. (gardener/gardener#7388, @breuerfelix)
• [DEVELOPER] A new NetworkPolicy controller (disabled by default) has been introduced in gardener-resource-manager. You can read all about it in this document. (gardener/gardener#7392, @rfranzke)

🐛 Bug Fixes

• [USER] A PR which was breaking the creation of clusters with custom workerpool labels, for eg: node-role.kubernetes.io/default, due to restrictions for the kubelet on applying them, is now reverted. (gardener/gardener#7424, @shafeeqes)
• [OPERATOR] When deleting a seed the cluster-identity config map in kube-system namespace is not deleted anymore if it was already existing on seed creation. (gardener/gardener#7436, @oliver-goetz)
• [OPERATOR] Fixed bug that cause HA VPN to fail in case the seed's apiserver was not targeted by kube-apiserver's vpn-client via a public address (gardener/gardener#7440, @ScheererJ)
• [OPERATOR] Fixes an issue with a missing network policy in the namespace of exposure classes. (gardener/gardener#7459, @axel7born)
• [OPERATOR] A bug has been fixed which caused the conditions of Shoots to be set to Unknown too fast in case the responsible gardenlet is no longer posting its heartbeat. (gardener/gardener#7404, @rfranzke)
• [OPERATOR] A bug has been fixed which prevented the ssh-keypair secrets from being deleted when SSHAccess for worker nodes is disabled. (gardener/gardener#7411, @AleksandarSavchev)
• [OPERATOR] A bug in the sshd-ensurer script running on all shoot worker nodes has been fixed which was causing it to also kill processes other than sshd when SSHAccess for worker nodes is disabled. (gardener/gardener#7418, @AleksandarSavchev)
• [DEVELOPER] A bug has been fixed in the provider-local extension that causes the local-setup to not work with the runtime-gvisor extension. (gardener/gardener#7316, @danielfoehrKn)

🏃 Others

• [USER] User-specified and gardener-managed Node labels are added immediately on registration of new Nodes. Excluded from this are labels that kubelets are forbidden to add by the NodeRestriction admission plugin (they are still added asynchronously by machine-controller-manager). (gardener/gardener#7426, @timebertt)
• [OPERATOR] Update vertical-pod-autoscaler to v0.13.0. This deprecates autoscaling.k8s.io/v1beta2 API, switch to autoscaling.k8s.io/v1 instead. (gardener/gardener#7441, @voelzmo)
• [OPERATOR] apiserver-proxy now uses a dynamic base-id for shared memory segments and hence allows multiple envoy-proxy servers to run on the same node. (gardener/gardener#7446, @ScheererJ)
• [DEVELOPER] The logging integration tests which were failing for arm Shoots are now fixed. The Pods deployed by the test do now use multi-arch image. (gardener/gardener#7453, @ialidzhikov)
• [DEVELOPER] The following dependencies are updated: (gardener/gardener#7455, @shafeeqes)
  • sigs.k8s.io/controller-runtime: v0.14.1 -> v0.14.4
  • sigs.k8s.io/controller-tools: v0.11.1 -> v0.11.3
• [DEVELOPER] The following dependencies are updated: (gardener/gardener#7248, @shafeeqes)
  • k8s.io/* : v0.25.0 -> v0.26.1
  • sigs.k8s.io/controller-runtime: v0.13.0 -> v0.14.1
• [DEPENDENCY] When the MutableShootSpecNetworkingNodes feature gate is enabled in gardener-apiserver, Infrastructure extensions need to guarantee that the condition "infrastructure virtual network range >= nodes CIDR range >= worker CIDR range" is met. (gardener/gardener#7368, @axel7born)

[ext-authz-server]

🏃 Others

• [DEVELOPER] Add rules for verification, formatting, linting and testing to Makefile. (gardener/ext-authz-server#12, @axel7born)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.64.0
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.64.0
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.64.0
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.64.0
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.64.0
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.64.0
operator: eu.gcr.io/gardener-project/gardener/operator:v1.64.0

[gardener]

🐛 Bug Fixes

• [USER] A PR which was breaking the creation of clusters with custom workerpool labels, for eg: node-role.kubernetes.io/default, due to restrictions for the kubelet on applying them, is now reverted. (gardener/gardener#7425, @shafeeqes)
• [OPERATOR] A bug has been fixed which caused the conditions of Shoots to be set to Unknown too fast in case the responsible gardenlet is no longer posting its heartbeat. (gardener/gardener#7413, @gardener-ci-robot)
• [OPERATOR] A bug has been fixed which prevented the ssh-keypair secrets from being deleted when SSHAccess for worker nodes is disabled. (gardener/gardener#7417, @gardener-ci-robot)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.63.1
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.63.1
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.63.1
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.63.1
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.63.1
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.63.1
operator: eu.gcr.io/gardener-project/gardener/operator:v1.63.1

[gardener]

🐛 Bug Fixes

• [USER] On Debian-based operating systems, Gardener CAs are now added correctly to the CA bundle for shoot worker nodes. (gardener/gardener#7306, @gardener-ci-robot)
• [DEPENDENCY] The extension health check controller properly updates the conditions in case of timeouts. (gardener/gardener#7303, @gardener-ci-robot)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.60.7
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.60.7
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.60.7
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.60.7
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.60.7
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.60.7
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.60.7

[gardener]

🐛 Bug Fixes

• [USER] On Debian-based operating systems, Gardener CAs are now added correctly to the CA bundle for shoot worker nodes. (gardener/gardener#7307, @gardener-ci-robot)
• [OPERATOR] A bug causing the extension BackupEntry not to be correctly reconciled in case of secret rotation and a controller restart is now fixed. (gardener/gardener#7347, @shafeeqes)
• [OPERATOR] Fix Prometheus scrape config for Istio ingress gateway. (gardener/gardener#7384, @axel7born)
• [DEPENDENCY] The extension health check controller properly updates the conditions in case of timeouts. (gardener/gardener#7302, @gardener-ci-robot)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.61.6
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.61.6
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.61.6
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.61.6
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.61.6
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.61.6
operator: eu.gcr.io/gardener-project/gardener/operator:v1.61.6

[gardener]

⚠️ Breaking Changes

• [USER] The deprecated feature regarding the usage of a user-provided ServiceAccount signing key has been removed. This means that the Shoot API no longer supports the .spec.kubernetes.kubeAPIServer.serviceAccountConfig.signingKeySecret field. If you were using this field before then Gardener will now generate a new signing key secret. All existing ServiceAccount tokens will become invalid and must be regenerated. (gardener/gardener#7242, @rfranzke)
• [USER] The annotation alpha.featuregates.shoot.gardener.cloud/reversed-vpn on Shoots is no longer respected and should be removed from all resources. (gardener/gardener#7167, @axel7born)
• [OPERATOR] All the functionality related to the deprecated field seed.spec.settings.shootDNS has been removed and subsequently seed.spec.settings.shootDNS will be dropped from the Seed API in a later release of Gardener. Please check your Seeds and remove any usage before upgrading to this Gardener version. (gardener/gardener#7314, @acumino)
• [OPERATOR] The newly created or recreated basic authentication Secrets will no longer contain the basic_auth.csv field under the Secret's data. The basic_auth.csv field was only required for the kube-apiserver basic auth which can no longer be enabled for K8s >= 1.19 Shoot clusters. (gardener/gardener#7362, @dimitar-kostadinov)
• [OPERATOR] The maximum length of exposure class names has been reduced from 41 to 34 characters. (gardener/gardener#6997, @ScheererJ)
• [OPERATOR] The legacy VPN solution has been removed. The feature gates ReversedVPN, ManagedIstio and APIServerSNI are unconditionally enabled (locked to their default values) now. (gardener/gardener#7167, @axel7born)
• [DEPENDENCY] The controllerutils.{WithLogger,CreateWorker} functions have been removed. (gardener/gardener#7239, @rfranzke)

✨ New Features

• [USER] It is now possible to configure sshAccess field for the workers settings in the Shoot API via .spec.provider.workersSettings.sshAccess. It is set to true by default and when enabled it ensures that the sshd.service on the worker nodes is enabled and running. When set to false it ensures that the sshd.service on the worker nodes is disabled and stopped, and all already established SSH connections are terminated. (gardener/gardener#7188, @AleksandarSavchev)
• [USER] A new feature gate IPv6SingleStack has been added along with new ipFamilies fields in the Shoot, Seed, and Network APIs (defaulted to ["IPv4"]) in preparation for supporting IPv6 in gardener clusters. Please see the documentation for more information. (gardener/gardener#7288, @timebertt)
• [OPERATOR] Dashboards are added for controller-runtime metrics for all the controllers and webhooks in the seed and the shoot controlplanes. (gardener/gardener#7180, @shafeeqes)
• [OPERATOR] It is now possible to configure the control plane of the virtual garden cluster to run in a "highly available" mode by setting .spec.virtualCluster.controlPlane.highAvailability={}. As of today, this causes the ETCDs to run with 3 replicas each. (gardener/gardener#7225, @rfranzke)
• [OPERATOR] gardener-operator is now managing the load balancer Service for exposing the virtual-garden-kube-apiserver as part of the virtual garden cluster control plane. It is possible to specify annotations for it via .spec.runtimeCluster.settings.loadBalancerServices.annotations in the Garden resource. (gardener/gardener#7238, @rfranzke)
• [OPERATOR] A new field controllers.shootCare.managedResourceProgressingThreshold is introduced in GardenletConfiguration, which can be used to set the threshold time for tolerating long-running ManagedResources with Progressing=True condition. (gardener/gardener#7241, @acumino)
• [OPERATOR] The Gardener API Server now supports machine image versions to specify constraints for the supported kubelet versions. This is a feature needed in the context of Kubernetes 1.26 adoption. [email protected] removed support CRI v1alpha2. Hence, [email protected] is only compatible with containerd versions >= 1.6.0. Using this feature gardener operators can specify that a given machine image version with containerd < 1.6.0 is only usable by kubelet < 1.26. (gardener/gardener#7265, @ialidzhikov)
• [OPERATOR] A new shoot annotation shoot.gardener.cloud/skip-readiness has been added. Gardener skips most readiness checks in the shoot reconciliation flow when shoots have this annotation. It is meant to push through shoot spec changes in case of critical situations, e.g. an availability zone outage, in which various steps can never reach readiness. Once successfully reconciled, the annotation is automatically removed again. Using this annotation however, must be evaluated on a case-by-case basis since it can severely affect the availability of shoot control planes. (gardener/gardener#7268, @timuthy)
• [OPERATOR] Gardener can now support Shoot clusters (and Seed clusters with HVPA feature gate disabled) with Kubernetes version 1.26. In order to allow creation/update of 1.26 clusters you will have to update the version of your provider extension(s) to a version that supports 1.26 as well. Please consult the respective releases and notes in the provider extension's repository. (gardener/gardener#7275, @ialidzhikov)
• [OPERATOR] A new condition, ObservabilityComponentsHealthy has been introduced in the Shoot for tracking the status of observability components like Prometheus, Loki, Grafana, etc. The ControlPlaneHealthy condition now only tracks core control plane components like ETCD, KAPI, KCM etc. (gardener/gardener#7325, @shafeeqes)
• [DEVELOPER] A new webhook SystemComponentsConfig has been added to the Gardener Resource Manager. It automatically adds the system component node selector as well as any custom taints specified by the user to Pods not managed by DaemonSets. Hence, if you set the worker.gardener.cloud/system-components: true node selector manually, this can be dropped in favor of the webhook's automatic handling. (gardener/gardener#7204, @timuthy)
• [DEVELOPER] The HighAvailabilityConfig webhook now also mutates replica settings of HPA and HVPA resources. To make use of this handling, please label respective resources with the well known high-availability-config.resource.gardener.cloud/type label, see docs/development/high-availability.md for more information. (gardener/gardener#7226, @timuthy)
• [DEVELOPER] It is now possible to make secrets manager adopt existing secrets. Find out more in this document. (gardener/gardener#7243, @rfranzke)
• [DEVELOPER] Gardener can now support Shoot clusters (and Seed clusters with HVPA feature gate disabled) with Kubernetes version 1.26. Extension developers have to prepare individual extensions as well to work with 1.26. (gardener/gardener#7275, @ialidzhikov)
• [DEPENDENCY] The DNSRecord API now supports records of type AAAA. If you implement a DNSRecord controller, you can start implementing support for this record type. (gardener/gardener#7246, @nschad)

🐛 Bug Fixes

• [USER] On Debian-based operating systems, Gardener CAs are now added correctly to the CA bundle for shoot worker nodes. (gardener/gardener#7253, @oliver-goetz)
• [USER] A bug in the kubelet-monitor script running on all shoot worker nodes has been fixed which was causing to also kill processes other than kubelet only. (gardener/gardener#7278, @eric-garber)
• [USER] A bug has been fixed which prevented Shoots from being deleted if they have never been reconciled at least once. (gardener/gardener#7326, @rfranzke)
• [OPERATOR] A bug has been fixed which caused ETCD encryption secrets which were no longer in-use to not get auto-deleted. (gardener/gardener#7244, @rfranzke)
• [OPERATOR] When deploying kube-apiserver version v1.24, gardener will add the --shutdown-send-retry-after=true command line flag to the kube-apiserver command. This is necessary so that during the graceful termination of the kube-apiserver process, it responds to new requests with the Connection: close and Retry-After: N headers so that any active TCP connections are closed and they have a chance to be reopened to running kube-apiservers. This is a workaround for an issue in which if the --audit-log-mode=batch is set on the kube-apiserver, it can enter a deadlock during graceful termination. This deadlock can lead to kubelets not being able to update their corresponding Node's status as the TCP connection to the broken kube-apiserver will never be closed. (gardener/gardener#7250, @plkokanov)
• [OPERATOR] A bug causing the extension BackupEntry not to be correctly reconciled in case of secret rotation and a controller restart is now fixed. (gardener/gardener#7281, @shafeeqes)
• [OPERATOR] An issue causing grafana dashboard panel to show wrong count of pods per node is fixed. Previously the panel was only showing the count of pods in the kube-system namespace only. Now it shows the count of pods in all namespaces. (gardener/gardener#7291, @acumino)
• [OPERATOR] A bug preventing ManagedSeed bootstrapping for 1.24+ Kubernetes clusters has been fixed. (gardener/gardener#7315, @rfranzke)
• [OPERATOR] Fix Prometheus scrape config for Istio ingress gateway. (gardener/gardener#7375, @axel7born)
• [DEVELOPER] A typo in the command to update the /etc/hosts file that also prevented make test-e2e-local-simple to be executed successfully is now fixed. (gardener/gardener#7271, @ialidzhikov)
• [DEVELOPER] Ensures the correct set of permissions (0775) for the host-mounted bucket-directory during backup bucket reconciliation in the local-provider (gardener/gardener#7292, @danielfoehrKn)
• [DEPENDENCY] The extension health check controller properly updates the conditions in case of timeouts. (gardener/gardener#7296, @timebertt)

📖 Documentation

• [DEVELOPER] The Gardener project has introduced a policy for the number of supported Kubernetes versions read it here. (gardener/gardener#7300, @rfranzke)

🏃 Others

• [USER] User-specified and gardener-managed Node labels are added immediately on registration of new Nodes. (gardener/gardener#7202, @timebertt)
• [USER] The ServiceAccount signing key rotation procedure has been improved and should work better for clusters with lots of ServiceAccounts or intermittent creations/deletions of new/old ServiceAccount secrets. (gardener/gardener#7313, @rfranzke)
• [USER] Gardener refined the scope of the problematic webhook matcher for service objects. Earlier, shoot clusters were assigned a constraint reporting a problem with a failurePolicy: Fail webhook acting on these objects. Now, only services in the kube-system and defaults namespaces are considered for this check. (gardener/gardener#7324, @timuthy)
• [USER] Webhook configurations can now exclude namespaces (mostly kube-system) by using the well known and Kubernetes standard label key kubernetes.io/metadata.name. Earlier, Gardener reported such webhooks as problematic. (gardener/gardener#7324, @timuthy)
• [OPERATOR] Updated node-local-dns to v1.22.5. (gardener/gardener#7185, @ScheererJ)
• [OPERATOR] Update envoy-proxy to v1.24.1. (gardener/gardener#7186, @ScheererJ)
• [OPERATOR] A new webhook SystemComponentsConfig has been added to the Gardener Resource Manager. It makes sure that matching Pods can run on Nodes which allow system components. Please see ./docs/concepts/resource-manager.md#System-Components-Webhook for more information. (gardener/gardener#7204, @timuthy)
• [OPERATOR] The following image is updated: (gardener/gardener#7220, @rickardsjp)
  • ghcr.io/prometheus-operator/prometheus-config-reloader: v0.59.1 -> v0.61.1
• [OPERATOR] A validation for duplicate zones in the zones list of a worker has been added. (gardener/gardener#7221, @MartinWeindel)
• [OPERATOR] The following image is updated: (gardener/gardener#7223, @rickardsjp)
  • quay.io/prometheus/blackbox-exporter: v0.22.0 -> v0.23.0
• [OPERATOR] The following image is updated: (gardener/gardener#7234, @nickytd)
  • quay.io/brancz/kube-rbac-proxy: v0.13.1 -> v0.14.0
• [OPERATOR] Remove the configuration of kubelet's NodeStatusUpdateFrequency to use its default value. With this, shoots make full use of kubelet's node lease feature now instead of performing heartbeats on the Node objects. (gardener/gardener#7261, @breuerfelix)
• [OPERATOR] The following image is updated: (gardener/gardener#7282, @rickardsjp)
  • quay.io/prometheus/node-exporter: v1.4.0 -> v1.5.0
• [OPERATOR] The problematic webhook detector has been adapted to match webhooks acting on endpoint and lease objects in all namespaces. Under some circumstances, such webhooks were not reported as problematic before. (gardener/gardener#7355, @timuthy)
• [OPERATOR] The following image is updated: (gardener/gardener#7361, @istvanballok)
  • quay.io/prometheus/prometheus: v2.40.2 -> v2.41.0
• [OPERATOR] Added a watchdog container to kube-apiserver pods for kubernetes v1.24 which will monitor if the kube-apiserver process is working properly and use some heuristics to detect if it is stuck during shutdown. If that happens it will forcefully cause it to finish shutting down by sending a SIGTERM signal to it. (gardener/gardener#7366, @plkokanov)
• [OPERATOR] Single-zone control planes on multi-zone seed clusters will have a change in their externally visible address of the kube-apiserver. This will happen during the next reconciliation. (gardener/gardener#6997, @ScheererJ)
• [DEVELOPER] Update to Go 1.19.5. (gardener/gardener#7309, @oliver-goetz)
• [DEVELOPER] Gardener only adds additional addresses to the machine's loopback device if a local multi-zonal environment is used. Earlier this step was performed with all make kind-*up commands. (gardener/gardener#7369, @timuthy)

[apiserver-proxy]

🏃 Others

• [DEVELOPER] Add verification rules to makefile and verify step to pipeline. (gardener/apiserver-proxy#29, @axel7born)

[autoscaler]

🏃 Others

• [DEVELOPER] Docker images for cluster-autoscaler are published with multi-arch support for linux/amd64 and linux/arm64 now. (gardener/autoscaler#162, @oliver-goetz)
  • Update Go version to 1.19.5.

[etcd-druid]

🏃 Others

• [OPERATOR] Fix make manifests target. (gardener/etcd-druid#489, @shreyas-s-rao)
• [OPERATOR] The Etcd resource now allows specify etcd client Service labels via the spec.etcd.clientService.labels field. (gardener/etcd-druid#491, @ialidzhikov)
• [DEPENDENCY] Dependency github.com/gardener/gardener is updated v1.36.0 -> v1.57.1 (gardener/etcd-druid#450, @AleksandarSavchev)
• [DEPENDENCY] Dependency github.com/onsi/ginkgo is upgraded to github.com/onsi/ginkgo/v2 (gardener/etcd-druid#450, @AleksandarSavchev)
• [DEPENDENCY] The dependency of sigs.k8s.io/controller-runtime/pkg/envtest/printer package in etcd-druid is removed. (gardener/etcd-druid#493, @shafeeqes)

[hvpa-controller]

⚠️ Breaking Changes

• [OPERATOR] The parameters metrics-addr and enable-leader-election have been changed to metrics-bind-address and leader-elect respectively to be in-line with kubebuilder v3 scaffolding. (gardener/hvpa-controller#113, @voelzmo)

🏃 Others

• [OPERATOR] Bumped go to 1.19.4 (gardener/hvpa-controller#115, @voelzmo)
• [DEVELOPER] make test is adapted to automatically configure envtest to use k8s 1.24.2 binaries (gardener/hvpa-controller#113, @voelzmo)
• [DEVELOPER] Necessary binaries, such as controller-gen, kustomize and setup-envtest are automatically downloaded and installed into ./bin (gardener/hvpa-controller#113, @voelzmo)
• [DEVELOPER] Bump api module golang dep to 1.18 (gardener/hvpa-controller#111, @voelzmo)
• [DEVELOPER] Update k8s.io dependencies to include k8s 1.25 (gardener/hvpa-controller#111, @voelzmo)
• [DEVELOPER] Switch to klog/v2 (gardener/hvpa-controller#111, @voelzmo)

[logging]

🏃 Others

• [OPERATOR] upgrade golang version from 1.19.3 to 1.19.4. (gardener/logging#163, @vlvasilev)
• [OPERATOR] upgrade telegraf version from 1.24.2 to 1.25.0. (gardener/logging#163, @vlvasilev)
• [OPERATOR] Increase Golang version from 1.19.4 to 1.19.5 (gardener/logging#165, @vlvasilev)

[vpn2]

🏃 Others

• [OPERATOR] Set primary device for bond device to avoid partitioning of VPN clients into two groups (gardener/vpn2#25, @MartinWeindel)
• [OPERATOR] Update builder image to golang:1.19.5 (gardener/vpn2#25, @MartinWeindel)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.63.0
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.63.0
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.63.0
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.63.0
operator: eu.gcr.io/gardener-project/gardener/operator:v1.63.0
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.63.0
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.63.0

[gardener]

🐛 Bug Fixes

• [USER] A bug has been fixed which prevented Shoots from being deleted if they have never been reconciled at least once. (gardener/gardener#7328, @gardener-ci-robot)
• [USER] On Debian-based operating systems, Gardener CAs are now added correctly to the CA bundle for shoot worker nodes. (gardener/gardener#7308, @gardener-ci-robot)
• [USER] A bug in the kubelet-monitor script running on all shoot worker nodes has been fixed which was causing to also kill processes other than kubelet only. (gardener/gardener#7354, @gardener-ci-robot)
• [OPERATOR] A bug causing the extension BackupEntry not to be correctly reconciled in case of secret rotation and a controller restart is now fixed. (gardener/gardener#7346, @shafeeqes)
• [OPERATOR] Fix Prometheus scrape config for Istio ingress gateway. (gardener/gardener#7382, @axel7born)
• [DEPENDENCY] The extension health check controller properly updates the conditions in case of timeouts. (gardener/gardener#7301, @gardener-ci-robot)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.62.2
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.62.2
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.62.2
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.62.2
operator: eu.gcr.io/gardener-project/gardener/operator:v1.62.2
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.62.2
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.62.2

[gardener]

🐛 Bug Fixes

• [OPERATOR] A bug has been fixed which caused ETCD encryption secrets which were no longer in-use to not get auto-deleted. (gardener/gardener#7255, @gardener-ci-robot)
• [OPERATOR] When deploying kube-apiserver version v1.24, gardener will add the --shutdown-send-retry-after=true command line flag to the kube-apiserver command. This is necessary so that during the graceful termination of the kube-apiserver process, it responds to new requests with the Connection: close and Retry-After: N headers so that any active TCP connections are closed and they have a chance to be reopened to running kube-apiservers. This is a workaround for an issue in which if the --audit-log-mode=batch is set on the kube-apiserver, it can enter a deadlock during graceful termination. This deadlock can lead to kubelets not being able to update their corresponding Node's status as the TCP connection to the broken kube-apiserver will never be closed. (gardener/gardener#7263, @gardener-ci-robot)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.61.5
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.61.5
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.61.5
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.61.5
operator: eu.gcr.io/gardener-project/gardener/operator:v1.61.5
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.61.5
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.61.5

[gardener]

🐛 Bug Fixes

• [OPERATOR] A bug has been fixed which caused ETCD encryption secrets which were no longer in-use to not get auto-deleted. (gardener/gardener#7256, @gardener-ci-robot)
• [OPERATOR] When deploying kube-apiserver version v1.24, gardener will add the --shutdown-send-retry-after=true command line flag to the kube-apiserver command. This is necessary so that during the graceful termination of the kube-apiserver process, it responds to new requests with the Connection: close and Retry-After: N headers so that any active TCP connections are closed and they have a chance to be reopened to running kube-apiservers. This is a workaround for an issue in which if the --audit-log-mode=batch is set on the kube-apiserver, it can enter a deadlock during graceful termination. This deadlock can lead to kubelets not being able to update their corresponding Node's status as the TCP connection to the broken kube-apiserver will never be closed. (gardener/gardener#7262, @gardener-ci-robot)
• [DEVELOPER] A typo in the command to update the /etc/hosts file that also prevented make test-e2e-local-simple to be executed successfully is now fixed. (gardener/gardener#7274, @gardener-ci-robot)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.62.1
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.62.1
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.62.1
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.62.1
operator: eu.gcr.io/gardener-project/gardener/operator:v1.62.1
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.62.1
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.62.1

[gardener]

🐛 Bug Fixes

• [OPERATOR] When deploying kube-apiserver version v1.24, gardener will add the --shutdown-send-retry-after=true command line flag to the kube-apiserver command. This is necessary so that during the graceful termination of the kube-apiserver process, it responds to new requests with the Connection: close and Retry-After: N headers so that any active TCP connections are closed and they have a chance to be reopened to running kube-apiservers. This is a workaround for an issue in which if the --audit-log-mode=batch is set on the kube-apiserver, it can enter a deadlock during graceful termination. This deadlock can lead to kubelets not being able to update their corresponding Node's status as the TCP connection to the broken kube-apiserver will never be closed. (gardener/gardener#7264, @gardener-ci-robot)
• [OPERATOR] A bug has been fixed which caused ETCD encryption secrets which were no longer in-use to not get auto-deleted. (gardener/gardener#7254, @gardener-ci-robot)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.60.6
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.60.6
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.60.6
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.60.6
operator: eu.gcr.io/gardener-project/gardener/operator:v1.60.6
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.60.6
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.60.6

[gardener]

⚠️ Breaking Changes

• [OPERATOR] Remove DNSProvider from supported extension kinds. Make sure that the dns-external extension has been removed completely before installing this Gardener version. (gardener/gardener#7138, @MartinWeindel)
• [DEVELOPER] Functions CreateForShoot() and NewForShoot() in pkg/utils/managedresources had their signatures changed. They both now accept an additional parameter called origin. Gardener acts on resources with "origin=gardener" label. External callers of these functions should provide their own unique origin value when creating managedresources in order to prevent unwanted actions on these resources. (gardener/gardener#7162, @dimityrmirchev)

✨ New Features

• [USER] Allow all authenticated users to create TokenReviews and SelfSubjectaAccessReviews. This is a required for the new gardenctl ssh-patch command. (gardener/gardener#7201, @holgerkoser)
• [USER] gardener-admission-controller now validates Shoot Kubernetes version compatibility with Audit Policy API version on Shoot update request. (gardener/gardener#7205, @acumino)
• [USER] It is now possible to configure the general log verbosity and the verbosity for HTTP access logs for the kube-apiserver via the Shoot specification. (gardener/gardener#7094, @vlvasilev)
• [USER] High availability deployment for VPN has been added. It is enabled automatically whenever the high availability of the control plane is activated. Two separate VPN tunnels are setup in a mesh-like structure with automatic fail-over. Note, that open connections using the VPN may still be closed if either the used kube-apiserver pod on the seed side or the vpn-shoot pod on the shoot side are terminated. But new connections can be opened within seconds. For more details see this document. (gardener/gardener#6978, @MartinWeindel)
• [USER] From now on, summary of what was changed in the last maintenance time window in Shoot specification is kept in Shoot's .status.lastMaintenance field. (gardener/gardener#7035, @acumino)
• [OPERATOR] gardener-operator is now managing the two ETCD instances (main, events) as part of the virtual garden cluster control plane. (gardener/gardener#7067, @rfranzke)
• [OPERATOR] It is now possible to specify an extension lifecycle strategy that will determine when should be acted on Extension kinds during different control flows (reconciliation, deletion, migration). Please consult this document for more information. (gardener/gardener#6999, @dimityrmirchev)
• [OPERATOR] The gardener-operator now supports credentials rotation. The procedure is similar to how it works for Shoots. Please read this for more information. (gardener/gardener#7144, @rfranzke)
• [OPERATOR] Similar to Shoots are protected from accidental deletion, Gardens must now be annotated with confirmation.gardener.cloud/deletion=true before DELETE requests are accepted. (gardener/gardener#7144, @rfranzke)
• [OPERATOR] The gardener-operator now serves a validating webhook which ensures that there is only one Garden resource in the system at a time. (gardener/gardener#7144, @rfranzke)
• [OPERATOR] It is now possible to provide separate secrets for the ETCD CA/client certificate configuration for the gardener-apiserver when using the Gardener controlplane Helm chart. (gardener/gardener#7160, @rfranzke)
• [DEVELOPER] garden-local environment is extended by a scenario running Seeds and Shoots on a real infrastructure and registering provider-extensions. Please see docs/deployment/getting_started_locally_with_extensions.md for more information. (gardener/gardener#6678, @oliver-goetz)
• [DEVELOPER] A new document for developers has been added with a checklist for what to pay attention to when adding new components to garden, seed, or shoot clusters. Read it here. (gardener/gardener#7152, @rfranzke)

🐛 Bug Fixes

• [USER] An issue causing for K8s 1.25 Shoots the bin-packing-scheduler not to be added to the kube-scheduler configuration when the bin-packing profile is configured in the Shoot spec is now fixed. (gardener/gardener#7216, @ialidzhikov)
• [USER] A bug was fixed where nodes may get stuck in Pending state when kubelet's protectKernelDefaults is set to true. (gardener/gardener#7088, @AleksandarSavchev)
• [USER] A bug has been fixed which prevented adding deletion confirmation annotation for deleting Shoots whose domains were not unique in the system. (gardener/gardener#7134, @shafeeqes)
• [OPERATOR] An issue has been fixed which caused etcd-druid and hvpa-controller to be deleted on Seed deletion when the seed is the garden at the same time. (gardener/gardener#7172, @rfranzke)
• [OPERATOR] Fix replicas on hibernation for vpn-seed-server deployment. (gardener/gardener#7194, @MartinWeindel)
• [OPERATOR] Prevent updating Shoots which are scheduled to a Seed with less then 3 zones to spec.controlPlane.failureTolerance.type: zone (gardener/gardener#7195, @oliver-goetz)
• [OPERATOR] Fixed a bug where service, pod or node CIDRs that are private network (RFC1918) or carrier-grade NAT (RFC6598) IPv4 blocks would not be added as except clause to allow-to-private-networks networkpolicy. (gardener/gardener#7112, @axel7born)
• [OPERATOR] A bug has been fixed which prevented gardenlet from reconciling its Seed in case the seed cluster is the garden cluster at the same time. (gardener/gardener#7154, @rfranzke)
• [OPERATOR] gardener-resource-manager now only considers ConfigMaps/Secrets as garbage-collectable resources (i.e., other resources with the "garbage-collectable" label will not be kept even if removed from the ManagedResource or when it is deleted). (gardener/gardener#7161, @rfranzke)
• [OPERATOR] The HighAvailabilityConfig webhook part of gardener-resource-manager now ensures that the zone-pinning affinity is always respected. (gardener/gardener#7166, @rfranzke)

🏃 Others

• [OPERATOR] The generic Worker actuator will scale up machine-controller-manager Deployment when Shoot hibernates with .status.lastOperationType=Create or .status.lastOperationType=nil and a machine deployment exists with .status.Replicas > 0. (gardener/gardener#7179, @rishabh-11)
• [OPERATOR] The following image is updated: (gardener/gardener#7209, @rishabh-11)
  • eu.gcr.io/gardener-project/gardener/autoscaler/cluster-autoscaler: v1.24.0 -> v1.25.0 (for Kubernetes >= 1.25)
• [OPERATOR] Fix the error rate promql expression on the "Kubernetes Control Plane Status" Grafana dashboard, "API Server Request rates" panel. (gardener/gardener#7214, @istvanballok)
• [OPERATOR] Fluent operator CRDs are added to the seed clusters (gardener/gardener#7093, @Kristian-ZH)
• [OPERATOR] Promtail installed on each shoot node now scrapes log from pods when HOSTNAME contains capital letters (gardener/gardener#7132, @vlvasilev)
• [OPERATOR] CoreDNS updated to v1.10.0 (gardener/gardener#7133, @ScheererJ)
• [DEVELOPER] Update to Go 1.19.4. (gardener/gardener#7173, @oliver-goetz)
• [DEVELOPER] Gardener upgrade tests allow us to qualify/verify the current g/g revision/branch (not yet released) or a specific release whether it is compatible with latest or a specific release (gardener/gardener#7122, @seshachalam-yv)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.62.0
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.62.0
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.62.0
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.62.0
operator: eu.gcr.io/gardener-project/gardener/operator:v1.62.0
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.62.0
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.62.0

[gardener]

🐛 Bug Fixes

• [USER] An issue causing for K8s 1.25 Shoots the bin-packing-scheduler not to be added to the kube-scheduler configuration when the bin-packing profile is configured in the Shoot spec is now fixed. (gardener/gardener#7217, @gardener-ci-robot)
• [OPERATOR] Prevent updating Shoots which are scheduled to a Seed with less then 3 zones to spec.controlPlane.failureTolerance.type: zone (gardener/gardener#7198, @gardener-ci-robot)

🏃 Others

• [OPERATOR] The following image is updated: (gardener/gardener#7210, @gardener-ci-robot)
  • eu.gcr.io/gardener-project/gardener/autoscaler/cluster-autoscaler: v1.24.0 -> v1.25.0 (for Kubernetes >= 1.25)
• [OPERATOR] The generic Worker actuator will scale up machine-controller-manager Deployment when Shoot hibernates with .status.lastOperationType=Create or .status.lastOperationType=nil and a machine deployment exists with .status.Replicas > 0. (gardener/gardener#7181, @rishabh-11)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.61.4
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.61.4
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.61.4
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.61.4
operator: eu.gcr.io/gardener-project/gardener/operator:v1.61.4
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.61.4
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.61.4

[gardener]

🐛 Bug Fixes

• [USER] An issue causing for K8s 1.25 Shoots the bin-packing-scheduler not to be added to the kube-scheduler configuration when the bin-packing profile is configured in the Shoot spec is now fixed. (gardener/gardener#7218, @ialidzhikov)
• [OPERATOR] Prevent updating Shoots which are scheduled to a Seed with less then 3 zones to spec.controlPlane.failureTolerance.type: zone (gardener/gardener#7197, @gardener-ci-robot)

🏃 Others

• [OPERATOR] The following image is updated: (gardener/gardener#7211, @gardener-ci-robot)
  • eu.gcr.io/gardener-project/gardener/autoscaler/cluster-autoscaler: v1.24.0 -> v1.25.0 (for Kubernetes >= 1.25)
• [OPERATOR] The generic Worker actuator will scale up machine-controller-manager Deployment when Shoot hibernates with .status.lastOperationType=Create or .status.lastOperationType=nil and a machine deployment exists with .status.Replicas > 0. (gardener/gardener#7182, @gardener-ci-robot)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.60.5
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.60.5
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.60.5
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.60.5
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.60.5
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.60.5
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.60.5

[gardener]

🐛 Bug Fixes

• [USER] A bug has been fixed which prevented adding deletion confirmation annotation for deleting Shoots whose domains were not unique in the system. (gardener/gardener#7137, @gardener-ci-robot)
• [USER] An issue causing for K8s 1.25 Shoots the bin-packing-scheduler not to be added to the kube-scheduler configuration when the bin-packing profile is configured in the Shoot spec is now fixed. (gardener/gardener#7219, @ialidzhikov)
• [OPERATOR] Fixed an issue where the restoration phase of control plane can get stuck while waiting for the source BackupEntry to become ready. The issue could occur if the gardenlet configration specifies controllers.backupEntry.deletionGracePeriodHours larger than 0 and the Shoot's control plane is migrated twice within that timeframe. (gardener/gardener#7128, @gardener-ci-robot)
• [OPERATOR] Prevent updating Shoots which are scheduled to a Seed with less then 3 zones to spec.controlPlane.failureTolerance.type: zone (gardener/gardener#7196, @gardener-ci-robot)

🏃 Others

• [OPERATOR] The following image is updated: (gardener/gardener#7212, @gardener-ci-robot)
  • eu.gcr.io/gardener-project/gardener/autoscaler/cluster-autoscaler: v1.24.0 -> v1.25.0 (for Kubernetes >= 1.25)
• [OPERATOR] The generic Worker actuator will scale up machine-controller-manager Deployment when Shoot hibernates with .status.lastOperationType=Create or .status.lastOperationType=nil and a machine deployment exists with .status.Replicas > 0. (gardener/gardener#7183, @gardener-ci-robot)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.59.3
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.59.3
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.59.3
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.59.3
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.59.3
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.59.3
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.59.3

[gardener]

🐛 Bug Fixes

• [OPERATOR] An issue has been fixed which caused etcd-druid and hvpa-controller to be deleted on Seed deletion when the seed is the garden at the same time. (gardener/gardener#7176, @rfranzke)
• [OPERATOR] The HighAvailabilityConfig webhook part of gardener-resource-manager now ensures that the zone-pinning affinity is always respected. (gardener/gardener#7169, @gardener-ci-robot)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.61.3
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.61.3
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.61.3
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.61.3
operator: eu.gcr.io/gardener-project/gardener/operator:v1.61.3
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.61.3
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.61.3

[gardener]

🐛 Bug Fixes

• [OPERATOR] The HighAvailabilityConfig webhook part of gardener-resource-manager now ensures that the zone-pinning affinity is always respected. (gardener/gardener#7170, @gardener-ci-robot)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.60.4
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.60.4
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.60.4
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.60.4
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.60.4
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.60.4
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.60.4

[gardener]

🐛 Bug Fixes

• [OPERATOR] A bug has been fixed which prevented gardenlet from reconciling its Seed in case the seed cluster is the garden cluster at the same time. (gardener/gardener#7155, @gardener-ci-robot)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.61.2
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.61.2
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.61.2
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.61.2
operator: eu.gcr.io/gardener-project/gardener/operator:v1.61.2
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.61.2
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.61.2