[gardener]

🐛 Bug Fixes

• [USER] A bug has been fixed which prevented adding deletion confirmation annotation for deleting Shoots whose domains were not unique in the system. (gardener/gardener#7135, @gardener-ci-robot)
• [OPERATOR] Fixed an issue where the restoration phase of control plane can get stuck while waiting for the source BackupEntry to become ready. The issue could occur if the gardenlet configration specifies controllers.backupEntry.deletionGracePeriodHours larger than 0 and the Shoot's control plane is migrated twice within that timeframe. (gardener/gardener#7127, @gardener-ci-robot)

🏃 Others

• [OPERATOR] Cluster-proportional autoscaling of coredns now works with the high-availability handling. (gardener/gardener#7131, @gardener-ci-robot)
• [DEVELOPER] An issue causing the Seed logging integration test to always fail is now fixed. (gardener/gardener#7121, @gardener-ci-robot)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.60.3
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.60.3
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.60.3
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.60.3
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.60.3
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.60.3
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.60.3

[gardener]

🐛 Bug Fixes

• [USER] A bug has been fixed which prevented adding deletion confirmation annotation for deleting Shoots whose domains were not unique in the system. (gardener/gardener#7136, @gardener-ci-robot)
• [OPERATOR] The gardener-operator image is now successfully build and pushed with this release. With the 1.61.0 release the gardener-operator image was not build and pushed by the CI/CD.

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.61.1
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.61.1
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.61.1
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.61.1
operator: eu.gcr.io/gardener-project/gardener/operator:v1.61.1
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.61.1
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.61.1

[gardener]

🐛 Bug Fixes

• [USER] A bug has been fixed which prevented deleting Shoots whose domains were not unique in the system. (gardener/gardener#7090, @gardener-ci-robot)
• [OPERATOR] gardenlet is scraped again by seed-prometheus. (gardener/gardener#6984, @timebertt)
• [OPERATOR] Fix garden RBAC test to use TokenRequest API. (gardener/gardener#6995, @gardener-ci-robot)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.58.3
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.58.3
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.58.3
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.58.3
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.58.3
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.58.3
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.58.3

[gardener]

⚠️ Breaking Changes

• [USER] ⚠️ Gardener does no longer support shoot clusters with Kubernetes versions < 1.20. Make sure to upgrade all existing clusters before upgrading to this Gardener version. (gardener/gardener#6987, @dimitar-kostadinov)
• [USER] The Shoot kubeconfig Secret in the Project namespace no longer contains the token field. The token can be still fetched from the kubeconfig that is present in the kubeconfig field. (gardener/gardener#6987, @dimitar-kostadinov)
• [OPERATOR] The already deprecated SeedKubeScheduler feature gate is now removed. Before upgrading to this version, if you had the SeedKubeScheduler feature enabled, make sure to disable it and to run gardenlet to properly clean up any deployed resources related to the feature. Starting this version of Gardener, the feature gate and all related functionally is removed. Instead, use the bin-packing scheduling profile. (gardener/gardener#7052, @ialidzhikov)
• [OPERATOR] The deprecated field managedSeed.spec.seedTemplate has been removed from the ManagedSeed API. Please check your ManagedSeeds and ManagedSeedSets and remove any usage (switch to spec.gardenlet.config) before upgrading to this Gardener version. (gardener/gardener#6972, @timuthy)

✨ New Features

• [USER] The kube-apiserver is now verifying the server certificates presented by kubelets. (gardener/gardener#7047, @rfranzke)
• [OPERATOR] The gardener-operator does now also manage hvpa-controller (if HVPA feature gate is enabled) and etcd-druid. (gardener/gardener#7048, @rfranzke)
• [OPERATOR] The ResourceReferenceManager admission plugin in the gardener-apiserver now validates the BackupBuckets and BackupEntries for their resource references. Also, the deletion of BackupBucket is rejected if there are existing BackupEntries referencing it. (gardener/gardener#7065, @shafeeqes)
• [OPERATOR] There is a new gardener-operator component responsible for reconciling the new Garden CRD. Read more about it here. (gardener/gardener#7009, @rfranzke)

🐛 Bug Fixes

• [USER] A bug has been fixed which prevented deleting Shoots whose domains were not unique in the system. (gardener/gardener#7086, @rfranzke)
• [OPERATOR] nginx-ingress-controller now runs with 2 replicas to make it compatible with its pod disruption budget. (gardener/gardener#7042, @oliver-goetz)
• [OPERATOR] An issue has been fixed that caused Pods being stuck in Pending state when scheduled on seed clusters with multiple zones. (gardener/gardener#7061, @timuthy)
• [OPERATOR] Fixed an issue where the restoration phase of control plane can get stuck while waiting for the source BackupEntry to become ready. The issue could occur if the gardenlet configration specifies controllers.backupEntry.deletionGracePeriodHours larger than 0 and the Shoot's control plane is migrated twice within that timeframe. (gardener/gardener#7126, @plkokanov)

🏃 Others

• [USER] The following image is updated: (gardener/gardener#7084, @ialidzhikov)
  • registry.k8s.io/metrics-server/metrics-server: v0.6.1 -> v0.6.2
• [OPERATOR] Gardener explicitly configures the shoot worker nodes' kernels with net.ipv4.conf.{all,default}.forwarding = 1. (gardener/gardener#7046, @timebertt)
• [OPERATOR] The gardener-seed-admission-controller binary has been dropped from the code. Its logic has been merged into gardener-resource-manager. (gardener/gardener#7053, @rfranzke)
• [OPERATOR] The following image is updated: (gardener/gardener#7055, @rickardsjp)
  • quay.io/prometheus/prometheus: v2.39.1 -> v2.40.2
• [OPERATOR] The following image is updated: (gardener/gardener#7054, @rickardsjp)
  • quay.io/prometheus/node-exporter: v1.3.1 -> v1.4.0
• [OPERATOR] The blackbox-exporter tolerates the NoSchedule/NoExecute taints (gardener/gardener#7095, @istvanballok)
• [OPERATOR] The metrics-server tolerates the NoSchedule/NoExecute taints (gardener/gardener#7096, @istvanballok)
• [OPERATOR] Fix a "many-to-many matching" error case in a recording rule related to the monthly average resource usage calculation of the shoot control planes (gardener/gardener#7097, @istvanballok)
• [OPERATOR] Owner check settings are no longer configured for Etcd resources in all cases. Previously they were only configured for the etcd-main Etcd resource when the corresponding StatefulSet was deployed with 1 replica. (gardener/gardener#6988, @plkokanov)
• [OPERATOR] Cluster-proportional autoscaling of coredns now works with the high-availability handling. (gardener/gardener#7129, @ScheererJ)
• [OPERATOR] Reduced the timeout that EtcdCopyBackupsTask waits until a final snapshot of the ETCD backups is made before copying backups from the source Seed to the destination Seed during control plane migration to 5 minutes. (gardener/gardener#7018, @plkokanov)
• [OPERATOR] Add grafana dashboard and adapt prometheus alerts for monitoring multinode etcd clusters backing shoot clusters. (gardener/gardener#7023, @shreyas-s-rao)
• [OPERATOR] Remove status port 15021 from service istio-ingressgateway. (gardener/gardener#7027, @axel7born)
• [DEVELOPER] An issue causing the Seed logging integration test to always fail is now fixed. (gardener/gardener#7115, @ialidzhikov)
• [DEVELOPER] The deprecated label garden.sapcloud.io/role was finally removed from all Gardener components and from the API constants. (gardener/gardener#7036, @timuthy)

[apiserver-proxy]

🏃 Others

• [OPERATOR] Make apiserver proxy sidecar IPv6 aware (gardener/apiserver-proxy#27, @einfachnuralex)

[etcd-backup-restore]

🐛 Bug Fixes

• [OPERATOR] To avoid race-condition between closing of snapshotter and taking snapshot after defrag, removing the out-of-schedule full snapshot triggered after defragmentation of etcd. (gardener/etcd-backup-restore#554, @ishan16696)
• [OPERATOR] To avoid potential race-condition between go-routines updated probeEtcd func() to use shorter timeout. (gardener/etcd-backup-restore#532, @ishan16696)

🏃 Others

• [OPERATOR] Removes the redundant closing of snapshotter during initialization. (gardener/etcd-backup-restore#546, @ishan16696)
• [OPERATOR] Always update member peer URL, changed the way scale-up of etcd cluster is identified. (gardener/etcd-backup-restore#534, @unmarshall)
• [OPERATOR] Decreases the likelihood of potential race condition between the go-routines while closing the snapshotter. (gardener/etcd-backup-restore#537, @ishan16696)
• [OPERATOR] [bug-fix] backup-restore does not return error when it fails to update PeerURL of member. (gardener/etcd-backup-restore#540, @aaronfern)

📰 Noteworthy

• [OPERATOR] Updated golang version used to build images to 1.19.2 (gardener/etcd-backup-restore#551, @aaronfern)
• [OPERATOR] Updated golang version used to build images to 1.18.6 (gardener/etcd-backup-restore#535, @aaronfern)
• [OPERATOR] Updated golang version to 1.18 (gardener/etcd-backup-restore#535, @aaronfern)

[etcd-custom-image]

🏃 Others

• [OPERATOR] Etcd-custom-image will now retry fetching etcd configuration in case of any error (gardener/etcd-custom-image#26, @aaronfern)

[etcd-druid]

⚠️ Breaking Changes

• [OPERATOR] Claim logic for PodDisruptionBudgets have been removed. Already existing PodDisruptionBudget objects cannot be adopted anymore (gardener/etcd-druid#430, @aaronfern)
• [OPERATOR] Logic for deleting any leftover CronJobs created by etcd-druid:v0.6.0 has been removed. Please deploy a lower version of etcd-druid before upgrading if you still have any leftover CronJobs or manually delete them (gardener/etcd-druid#430, @aaronfern)

✨ New Features

• [OPERATOR] Applying the annotation druid.gardener.cloud/ignore-reconciliation on the ETCD CR will stop etcd-druid from reconciling it. (gardener/etcd-druid#446, @abdasgupta)
  • This is helpful for operators to apply any manual fixes to the ETCD components, such as manually fixing permanent quorum loss as per the playbook.

🐛 Bug Fixes

• [USER] Fix PDB permissions for druid chart. (gardener/etcd-druid#472, @shreyas-s-rao)
• [OPERATOR] An issue has been fixed that caused the BackupReady condition to show Unknown when the cluster is newly created. (gardener/etcd-druid#469, @timuthy)
• [OPERATOR] Updated condition used to decide when the gardener.cloud/scaled-to-multi-node annotation is added (gardener/etcd-druid#455, @aaronfern)
• [OPERATOR] A bug has been fixed that caused the wrong minAvailable configuration being calculated for multi-node etcd PodDisruptionBudget. (gardener/etcd-druid#441, @timuthy)

🏃 Others

• [OPERATOR] Updated golang version used to build images to 1.19.2 (gardener/etcd-druid#460, @aaronfern)
• [OPERATOR] The decision to add the scale-up annotation to the etcd sts now considers the etcd status if an existing sts is not present (gardener/etcd-druid#466, @aaronfern)
• [OPERATOR] Enables etcd-druid to trigger restarts of the etcd pods when peer URL is TLS enabled. (gardener/etcd-druid#421, @unmarshall)
• [OPERATOR] Etcd druid will now use policy/v1 for PodDisruptionBudgets for kubernetes >= 1.21. However, for kubernetes < 1.21, PodDisruptionBudgets will still default to policy/v1beta1 (gardener/etcd-druid#430, @aaronfern)
• [OPERATOR] Golang version used upgraded to 1.18.6 (gardener/etcd-druid#431, @aaronfern)
• [OPERATOR] etcd-custom-image upgraded to v3.4.13-bootstrap-8 (gardener/etcd-druid#432, @aaronfern)
• [OPERATOR] Added new field to the etcd CRD etcd.Spec.Etcd.clientService (gardener/etcd-druid#438, @aaronfern)
  • Users can now add annotations to the client service by specifying them in the above field of the etcd resource
• [OPERATOR] The BackupReady condition is not considered anymore when the PodDisruptionBudget configuration is calculated. This earlier blocked rolling out fixes that potentially solved problems with backup procedures. (gardener/etcd-druid#441, @timuthy)
• [OPERATOR] Enhance BackupReady condition to take into account statefulset being scaled down and the backup section not being defined (gardener/etcd-druid#415, @aaronfern)
• [DEVELOPER] Added e2e tests to check single member restoration in multi node setup (gardener/etcd-druid#451, @aaronfern)

[logging]

🏃 Others

• [OPERATOR] The logging components are now built using go version 1.19.3. (gardener/logging#160, @dimityrmirchev)
• [OPERATOR] The Event field which represents the first occurrence of the event is preserved. (gardener/logging#161, @vlvasilev)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.61.0
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.61.0
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.61.0
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.61.0
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.61.0
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.61.0

[gardener]

🐛 Bug Fixes

• [USER] A bug has been fixed which prevented deleting Shoots whose domains were not unique in the system. (gardener/gardener#7091, @gardener-ci-robot)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.60.2
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.60.2
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.60.2
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.60.2
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.60.2
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.60.2
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.60.2

[gardener]

🐛 Bug Fixes

• [USER] A bug has been fixed which prevented deleting Shoots whose domains were not unique in the system. (gardener/gardener#7092, @gardener-ci-robot)
• [OPERATOR] Fix garden RBAC test to use TokenRequest API. (gardener/gardener#6994, @gardener-ci-robot)
• [OPERATOR] gardenlet is scraped again by seed-prometheus. (gardener/gardener#6981, @gardener-ci-robot)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.59.2
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.59.2
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.59.2
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.59.2
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.59.2
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.59.2
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.59.2

[gardener]

🐛 Bug Fixes

• [OPERATOR] An issue has been fixed that caused Pods being stuck in Pending state when scheduled on seed clusters with multiple zones. (gardener/gardener#7063, @gardener-ci-robot)
• [OPERATOR] nginx-ingress-controller now runs with 2 replicas to make it compatible with its pod disruption budget. (gardener/gardener#7058, @gardener-ci-robot)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.60.1
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.60.1
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.60.1
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.60.1
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.60.1
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.60.1
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.60.1

[gardener]

⚠️ Breaking Changes

• [USER] The HA annotation alpha.control-plane.shoot.gardener.cloud/high-availability has been deprecated and is not respected anymore. We ask you to check your automation and to remove any usage of this annotation. (gardener/gardener#6998, @timuthy)
  • Gardener will automatically translate the annotation to the corresponding spec field for a few releases to not cause any disruptions for existing clusters that haven't been adjusted.
• [OPERATOR] The already deprecated .spec.highAvailability field has been removed from the Seed API. Instead, operators now MUST ensure to enter the names of all availability zones the seed worker nodes run in .spec.zones (ideally, before upgrading to this Gardener version). (gardener/gardener#6960, @rfranzke)
• [OPERATOR] The failureToleranceType field has been removed from .spec.gardenlet.deployment in the seedmanagement.gardener.cloud/v1alpha1.ManagedSeed API. (gardener/gardener#6967, @rfranzke)
• [OPERATOR] The field managedSeed.spec.seedTemplate has been deprecated and will be removed very soon in a future release of Gardener. Please adapt your ManagedSeedSet or ManagedSeed objects and transfer any seed configuration to managedSeed.spec.gardenlet.config (see example example/55-managedseed-gardenlet.yaml). (gardener/gardener#7006, @timuthy)
  • Please note that as a consequence, Gardenlet will be deployed and managed automatically (see docs/usage/managed_seed.md for more information).
• [DEVELOPER] The RBAC for DNSRecords is no longer required by extension controllers (other than DNSRecords) and should be removed where applicable. (gardener/gardener#6973, @plkokanov)
• [DEVELOPER] The default location for the kubeconfig of the local kind cluster has changed from example/gardener-local/kind/kubeconfig to example/gardener-local/kind/local/kubeconfig. (gardener/gardener#6976, @rfranzke)

✨ New Features

• [USER] Forwarding DNS queries to upstream DNS from node local DNS can be disabled to use custom DNS config in conjunction with node-local-dns. (gardener/gardener#6942, @axel7born)
• [OPERATOR] All non-observability-related seed system components are now running with configuration for high-availability according to the conventions. (gardener/gardener#6982, @rfranzke)
• [OPERATOR] All non-observability-related shoot system components are now running with configuration for high-availability according to the conventions. (gardener/gardener#6989, @rfranzke)
• [OPERATOR] All non-observability-related shoot control plane components are now running with configuration for high-availability according to the conventions. (gardener/gardener#6992, @rfranzke)
• [DEVELOPER] The gardener-resource-manager serves a new high-availability-config webhook for automatically mutating the HA-related configuration of Deployments and StatefulSets. Please refer to this and this document. (gardener/gardener#6967, @rfranzke)
• [DEPENDENCY] Deployments or StatefulSets deployed by extensions in seed or shoot clusters can now benefit from the new high-availability-config webhook for automatically mutating the HA-related configuration of these resources. Please refer to this document. (gardener/gardener#6967, @rfranzke)

🐛 Bug Fixes

• [USER] The CertificateSigningRequests created by kubelets for their server certificates are now also auto-approved when their Node object contains addresses of type InternalDNS, ExternalDNS, or ExternalIP. (gardener/gardener#6958, @rfranzke)
• [USER] A bug has been fixed which caused stuck Shoot on deletion because their Namespaces in the seed cluster were not cleaned up properly. It only affected clusters created prior gardener/[email protected]. (gardener/gardener#6964, @rfranzke)
• [OPERATOR] Fix garden RBAC test to use TokenRequest API. (gardener/gardener#6977, @vpnachev)
• [OPERATOR] gardenlet is scraped again by seed-prometheus. (gardener/gardener#6979, @timebertt)
• [OPERATOR] Gardener correctly identifies the logging directory for systemd-journald and does not rely on a directory existence check. (gardener/gardener#6980, @vlvasilev)
• [OPERATOR] A bug was fixed where sometimes the kube-apiserver was deleted during shoot deletion flow even though there were still shoot managed resources present. (gardener/gardener#7008, @dimityrmirchev)

🏃 Others

• [OPERATOR] Restrict the maximum amount of cpu/memory requests provided to node-local-dns pods to 100m/200Mi. (gardener/gardener#6913, @ScheererJ)
• [OPERATOR] The GA-ed Shoot{C,S}ARotation feature gates are now removed. (gardener/gardener#6930, @rfranzke)
• [OPERATOR] Calculate the monthly average resource usage of the shoot control planes (gardener/gardener#6944, @istvanballok)
  • Add recording rules to the "cache" Prometheus in the garden namespace of the seed and dashboards to the Grafana in the garden namespace to calculate and show the resource usage of the shoot control planes
• [OPERATOR] A bug has been fixed that can lead to containerd-shims getting SIGTERM signals by the containerd monitoring script. (gardener/gardener#6696, @danielfoehrKn)
• [OPERATOR] scheduler.alpha.kubernetes.io/critical-pod annotation is removed as pod priority (spec.priorityClassName) is used instead to mark pods as critical (gardener/gardener#6956, @dimitar-kostadinov)
• [OPERATOR] The GA-ed DisableDNSProviderManagement feature gate is now removed. (gardener/gardener#6959, @rfranzke)
• [OPERATOR] Added heartbeat controller to the common app package for os extensions. (gardener/gardener#6968, @AleksandarSavchev)
• [OPERATOR] Extension controllers no longer perform owner checks based on the owner DNSRecord at the start of their reconciliations. (gardener/gardener#6973, @plkokanov)
• [OPERATOR] Update istio to v1.15.3 (gardener/gardener#6983, @axel7born)
• [OPERATOR] A the seed vpa-recommender is no longer scaled by VPA. Instead, fixed resource request values are used. (gardener/gardener#7001, @andrerun)
• [OPERATOR] The blackbox-exporter respects the previously unnecessarily tolerated NoSchedule/NoExecute taints (gardener/gardener#7002, @istvanballok)
• [OPERATOR] A bug has been fixed which could prevent gardenlet pods from coming up in case the podtopologyspreadconstraints webhook served by gardener-resource-manager is unavailable or broken. (gardener/gardener#7015, @plkokanov)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.60.0
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.60.0
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.60.0
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.60.0
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.60.0
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.60.0
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.60.0

[gardener]

🐛 Bug Fixes

• [USER] The CertificateSigningRequests created by kubelets for their server certificates are now also auto-approved when their Node object contains addresses of type InternalDNS, ExternalDNS, or ExternalIP. (gardener/gardener#6963, @gardener-ci-robot)
• [USER] A bug has been fixed which caused stuck Shoot on deletion because their Namespaces in the seed cluster were not cleaned up properly. It only affected clusters created prior gardener/[email protected]. (gardener/gardener#6966, @gardener-ci-robot)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.59.1
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.59.1
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.59.1
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.59.1
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.59.1
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.59.1
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.59.1

[gardener]

🐛 Bug Fixes

• [USER] The CertificateSigningRequests created by kubelets for their server certificates are now also auto-approved when their Node object contains addresses of type InternalDNS, ExternalDNS, or ExternalIP. (gardener/gardener#6962, @gardener-ci-robot)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.58.2
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.58.2
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.58.2
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.58.2
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.58.2
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.58.2
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.58.2

[gardener]

⚠️ Breaking Changes

• [OPERATOR] The .spec.highAvailability field in the Seed is deprecated and no longer respected. It will be removed in a future release. The seed.gardener.cloud/multi-zonal label is removed and no longer respected. Instead, the Seed API now has .spec.provider.zones. Operators should enter the names of all availability zones the seed worker nodes run in. (gardener/gardener#6914, @rfranzke)
• [OPERATOR] HAControlPlanes feature gate is added to gardener-apiserver and removed from gardenlet. (gardener/gardener#6915, @oliver-goetz)
• [OPERATOR] The values for the gardenlet Helm chart are no longer put below .global.gardenlet. For example, before this PR the replica count was controlled via the global.gardenlet.replicaCount value while it's now controlled via replicaCount directly. Please adapt your values files accordingly. (gardener/gardener#6876, @rfranzke)
• [OPERATOR] The validate-namespace-deletion ValidatingWebhookConfiguration is renamed to gardener-admission-controller. You might need to cleanup the existing validate-namespace-deletion ValidatingWebhookConfiguration. (gardener/gardener#6894, @AleksandarSavchev)
• [OPERATOR] The gardener-shoot-controlplane PriorityClass is now deleted by gardenlet. Before updating to this version of Gardener, make sure that there are no extensions or external components still using this PriorityClass. Refer to this documentation to find out which PriorityClass should be used instead. (gardener/gardener#6899, @ialidzhikov)
• [DEVELOPER] The gardener-resource-manager component has been reworked entirely. It now uses a component config instead of CLI flags. Also, its Helm chart has been reworked entirely. (gardener/gardener#6865, @rfranzke)

✨ New Features

• [USER] It is now possible to configure the protectKernelDefaults field for the kubelet configuration in the Shoot API via .spec.{provider.workers[]}.kubernetes.kubelet.protectKernelDefaults. This will be unset by default for shoots with k8s version < 1.26 and will be defaulted to true for shoots with k8s version >= 1.26 once Gardener releases support for these versions. (gardener/gardener#6919, @dimityrmirchev)
• [USER] It is now possible to configure the streamingConnectionIdleTimeout field for the kubelet configuration in the Shoot API via .spec.{provider.workers[]}.kubernetes.kubelet.streamingConnectionIdleTimeout. This is implicitly defaulted to 4h for shoots with k8s version < 1.26 and will be defaulted to 5m for shoots with k8s version >= 1.26 once Gardener releases support for these versions. (gardener/gardener#6937, @dimityrmirchev)
• [USER] Kubelet configurations containerLogMaxSize and containerLogMaxFiles are now supported in the corresponding Shoot resource. Those properties manage rotation policy of the container logs. Under heavy load the default values may result in frequent log rotations. (gardener/gardener#6702, @nickytd)
• [OPERATOR] HAControlPlanes feature gate controls if it is possible to create shoots with a HighAvailability configuration in the landscape. (gardener/gardener#6915, @oliver-goetz)
• [DEVELOPER] log-level and log-format of provider-local can now be configured. (gardener/gardener#6875, @oliver-goetz)
• [DEPENDENCY] Extensions can now use the extensions/pkg/util.{DetermineError,DetermineErrorCodes} functions for conveniently handling errors with codes. (gardener/gardener#6912, @acumino)
• [DEPENDENCY] gardener-extensions-controller package includes CLI parameter for --log-level and --log-format now. (gardener/gardener#6875, @oliver-goetz)

🐛 Bug Fixes

• [OPERATOR] An issue has been fixed for shoot clusters on multi-zonal seeds that prevented control-plane pods from being scheduled, e.g. after hibernation. With this version of Gardener, zone-pinning for shoot control-planes will be suspended until a new version of the feature will be rolled out in a future release. (gardener/gardener#6934, @timuthy)
• [OPERATOR] A bug has been fixed which could prevent gardenlet pods from coming up in case the seccomp-profile webhook served by gardener-resource-manager is unavailable or broken. (gardener/gardener#6953, @dimityrmirchev)
• [OPERATOR] The KubeApiServerTooManyAuditlogFailures alert is now fixed to fire also when the audit plugins buffered and truncate are failing to process an audit event. (gardener/gardener#6871, @vpnachev)
• [OPERATOR] An issue causing the nginx-ingress-controller installed via the shoot's nginx-ingress addon to fail to start when cluster-wide seccomp defaulting is enabled is now fixed. (gardener/gardener#6895, @dimityrmirchev)

🏃 Others

• [USER] The rotation procedure of the ServiceAccount token signing key has been improved. (gardener/gardener#6943, @rfranzke)
• [OPERATOR] Profiling is now disabled for kube-controller-manager for shoots that have Kubernetes version >= 1.19. (gardener/gardener#6922, @dimityrmirchev)
• [OPERATOR] You should make sure that all Shoots are getting reconciled successfully or deleted in case they still have either the etcd-encryption-secret or service-account-key secrets in their namespaces in the seed cluster. (gardener/gardener#6929, @rfranzke)
• [OPERATOR] The zone-pinning feature for control-planes on multi-zonal seeds (introduced by https://github.com/gardener/gardener/pull/6579) has been removed. There will be a new version of the feature soon, that takes a different approach and fixes some bug and flaws along the way. (gardener/gardener#6934, @timuthy)
• [OPERATOR] Revert removal DNSProvider from supported extension kinds until v1.60.0 or later. (gardener/gardener#6951, @MartinWeindel)
• [OPERATOR] The ManagedResources related to seed system components are now labeled with gardener.cloud/role=system-component. (gardener/gardener#6836, @rfranzke)
• [OPERATOR] The gardenlet now waits for all managed resources referring the shoot to be deleted before continuing with the deletion of the shoot's kube-apiserver during shoot deletion or controlplane migration. (gardener/gardener#6853, @dimityrmirchev)
• [OPERATOR] Add new Prometheus alert ApiserverRequestsFailureRate for API Server failure rate. (gardener/gardener#6736, @cathyzhang05)
• [OPERATOR] gardenlet no longer tries to delete Ingress resources for a Seed via the extensions/v1beta1 API (no longer served as of K8s 1.22). As Gardener supports only Seed clusters with K8s >= 1.20, it is enough to delete the Ingress resources via the networking.k8s.io/v1 API (available since v1.19). (gardener/gardener#6866, @ialidzhikov)
• [OPERATOR] The Kubernetes Control Plane Status dashboard has been updated to show correct values for kube-controller-manager and kube-scheduler once they are deployed with multiple replicas for HA shoots. (gardener/gardener#6874, @timuthy)
• [DEVELOPER] Update golangci to v1.50.1. (gardener/gardener#6916, @oliver-goetz)
• [DEVELOPER] Go is updated to 1.19.3 (gardener/gardener#6941, @oliver-goetz)

[apiserver-proxy]

⚠️ Breaking Changes

• [DEVELOPER] bazel is no longer used for builds ands tests. As alternative a Makefile with equivalent targets is now provided. (gardener/apiserver-proxy#25, @ialidzhikov)

🐛 Bug Fixes

• [USER] An issue causing the apiserver-proxy-pod-webhook to wrongly remove the grpc field from livenessProbes, readinessProbes and startupProbes when defaulting a Pod is now fixed. (gardener/apiserver-proxy#24, @ialidzhikov)
• [OPERATOR] Native arm64 builds (builds on arm hosts) are now supported. Previously only arm64 builds with qemu were supported. (gardener/apiserver-proxy#25, @ialidzhikov)

🏃 Others

• [OPERATOR] Updated base image of apiserver-proxy to alpine 3.16.2 (gardener/apiserver-proxy#21, @ScheererJ)
• [OPERATOR] The golang version is updated to 1.19.2. (gardener/apiserver-proxy#22, @ialidzhikov)
• [OPERATOR] The following dependencies are updated: (gardener/apiserver-proxy#24, @ialidzhikov)
  • k8s.io/api: v0.19.2 -> v0.23.5
  • k8s.io/apimachinery: v0.19.2 -> v0.23.5
  • k8s.io/apiserver: v0.19.2 -> v0.23.5
  • k8s.io/client-go: v0.19.2 -> v0.23.5
  • sigs.k8s.io/controller-runtime: v0.7.0-alpha.4 -> v0.11.2

[logging]

🏃 Others

• [OPERATOR] The Telegraf image used by Loki pod is built from scratch with static binary. (gardener/logging#158, @vlvasilev)

[gardener]

🐛 Bug Fixes

• [OPERATOR] An issue has been fixed for shoot clusters on multi-zonal seeds that prevented control-plane pods from being scheduled, e.g. after hibernation. With this version of Gardener, zone-pinning for shoot control-planes will be suspended until a new version of the feature will be rolled out in a future release. (gardener/gardener#6938, @timuthy)
• [OPERATOR] The KubeApiServerTooManyAuditlogFailures alert is now fixed to fire also when the audit plugins buffered and truncate are failing to process an audit event. (gardener/gardener#6886, @gardener-ci-robot)

🏃 Others

• [USER] The rotation procedure of the ServiceAccount token signing key has been improved. (gardener/gardener#6945, @gardener-ci-robot)
• [OPERATOR] Revert removal DNSProvider from supported extension kinds until v1.60.0 or later. (gardener/gardener#6952, @gardener-ci-robot)
• [OPERATOR] The zone-pinning feature for control-planes on multi-zonal seeds (introduced by https://github.com/gardener/gardener/pull/6579) has been removed. There will be a new version of the feature soon, that takes a different approach and fixes some bug and flaws along the way. (gardener/gardener#6938, @timuthy)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.58.1
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.58.1
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.58.1
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.58.1
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.58.1
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.58.1
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.58.1

[gardener]

🐛 Bug Fixes

• [OPERATOR] The KubeApiServerTooManyAuditlogFailures alert is now fixed to fire also when the audit plugins buffered and truncate are failing to process an audit event. (gardener/gardener#6887, @gardener-ci-robot)

🏃 Others

• [USER] The rotation procedure of the ServiceAccount token signing key has been improved. (gardener/gardener#6946, @gardener-ci-robot)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.57.2
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.57.2
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.57.2
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.57.2
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.57.2
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.57.2
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.57.2

[gardener]

🐛 Bug Fixes

• [OPERATOR] The KubeApiServerTooManyAuditlogFailures alert is now fixed to fire also when the audit plugins buffered and truncate are failing to process an audit event. (gardener/gardener#6888, @gardener-ci-robot)
• [OPERATOR] The broken preStop hook from Gardener API Server deployment have been removed. (gardener/gardener#6796, @gardener-ci-robot)

🏃 Others

• [USER] The rotation procedure of the ServiceAccount token signing key has been improved. (gardener/gardener#6947, @rfranzke)
• [OPERATOR] The following image is updated: (gardener/gardener#6803, @gardener-ci-robot)
  • grafana/grafana: 7.5.16 -> 7.5.17

[etcd-backup-restore]

🐛 Bug Fixes

• [OPERATOR] A bug has been fixed that caused the wrong minAvailable configuration being calculated for multi-node etcd PodDisruptionBudget. (gardener/etcd-backup-restore#441, @timuthy)

🏃 Others

• [OPERATOR] The BackupReady condition is not considered anymore when the PodDisruptionBudget configuration is calculated. This earlier blocked rolling out fixes that potentially solved problems with backup procedures. (gardener/etcd-backup-restore#441, @timuthy)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.56.2
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.56.2
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.56.2
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.56.2
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.56.2
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.56.2
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.56.2

[gardener]

⚠️ Breaking Changes

• [USER] Shoots with failure tolerance type node can be scheduled on seeds with .spec.highAvailability != nil only. (gardener/gardener#6833, @oliver-goetz)
• [OPERATOR] HAControlPlanes feature flag is removed from gardener-scheduler. (gardener/gardener#6833, @oliver-goetz)
• [OPERATOR] Remove DNSProvider from supported extension kinds. (gardener/gardener#6840, @MartinWeindel)
• [DEPENDENCY] Health checks performed by the healthcheck library no longer update the extensions resources' status.conditions[].LastUpdateTime on each reconciliation. Instead, a new heartbeat controller was added to the extensions library that will renew a dedicated Lease resource named gardener-extensions-heartbeat every 30 seconds by default. Extension controllers have to enable this controller as the gardener-extensions-heartbeat Lease will be used when gardenlet checks whether the extension resources' conditions are stale or not. gardenlet expects to find this Lease inside the namespace where the extension controller is installed by the corresponding ControllerInstallation. (gardener/gardener#6626, @plkokanov)

✨ New Features

• [USER] The kubelets running on shoot worker nodes are now requesting server certificates via the CertificateSigningRequest API. They have the default validity of 30d and are auto-rotated when 80% of their lifetime expires. (gardener/gardener#6784, @rfranzke)
• [USER] It is now possible to configure the seccompDefault field for the kubelet configuration in the Shoot API via .spec.{provider.workers[]}.kubernetes.kubelet.seccompDefault. This configuration is only available for k8s version >= 1.25 and it is not turned on by default. (gardener/gardener#6741, @AleksandarSavchev)
• [OPERATOR] Short names for machine (mc), machineclass (mcc), machinedeployment (mcd), and machineset (mcs) resources are now added. (gardener/gardener#6787, @rishabh-11)
• [OPERATOR] log-level, log-format and verbosity of gardener-apiserver can now be configured. (gardener/gardener#6817, @oliver-goetz)
• [OPERATOR] It is now possible to disable PodSecurityPolicy admission plugin, please make sure you have updated the extensions to a version which supports this change. (gardener/gardener#6700, @shafeeqes)
• [OPERATOR] log-level and log-format of gardener-resource-manager can now be configured. (gardener/gardener#6830, @oliver-goetz)
• [OPERATOR] log-level and log-format of gardener-seed-admission-controller can now be configured. (gardener/gardener#6831, @oliver-goetz)
• [OPERATOR] High availability for seed system components can be defined by specifying spec.highAvailability.failureTolerance.type (gardener/gardener#6723, @unmarshall)
  • Additional validation is added which checks for the value of seed label seed.gardener.cloud/multi-zonal which was not existing before. The allowed values will be:empty string or a valid boolean value true | false
• [OPERATOR] Gardenlet can now be deployed with multiple replicas and a failureToleranceType of either node or zone. This is supported by the gardenlet Helm chart as well as through deployment options in managedseed objects. The replica spread is implemented via TopologySpreadConstraints. (gardener/gardener#6750, @timuthy)
• [OPERATOR] The ManagedResource health status for objects on the seed cluster is now updated immediately on health status changes (switched from periodic checks to proper watching). (gardener/gardener#6770, @timebertt)
• [OPERATOR] Updated machine CRD, allowing the display of node name and providerID(using -owide flag) when listing machines in the control plane of the shoot (gardener/gardener#6779, @rishabh-11)
• [OPERATOR] Gardenlet will not start in case the seed configuration is incorrect, i.e. if the node, pod or service network specified in the Seed resource do not match to the cluster reality. (gardener/gardener#6782, @ScheererJ)
• [DEVELOPER] The local setup has been improved to support tests for HA scenarios (single-zone with node failure tolerance and multi-zone with zone failure tolerance). (gardener/gardener#6719, @seshachalam-yv)
• [DEVELOPER] ConditionBuilder interface is extended by a WithClock(...) function. (gardener/gardener#6729, @oliver-goetz)
  • ...WithClock(...) condition helper functions are introduced.
  • WithNowFunc(...) function is removed from ConditionBuilder interface.

🐛 Bug Fixes

• [USER] Shoot worker definitions are now validated using .spec.kubernetes.kubelet when .spec.provider.workers[].kubernetes.kubelet is not specified. (gardener/gardener#6741, @AleksandarSavchev)
• [OPERATOR] The broken preStop hook from Gardener API Server deployment has been removed. (gardener/gardener#6793, @vpnachev)
• [OPERATOR] An issue causing the gardener-shoot-controlplane PriorityClass to be deleted too early when there are still Deployments (vpn-seed-server) that reference it is now mitigated. (gardener/gardener#6799, @ialidzhikov)
• [OPERATOR] The gardenlet is no longer put under time pressure during its start-up procedure by preventing its liveness probe from falsely failing. (gardener/gardener#6808, @rfranzke)
• [OPERATOR] kube-scheduler and cluster-autoscaler Pods now run with the appropriate priority set according to the following document. Previously these Pods were running without a priority class set and were preempted in favour of less important Pods. (gardener/gardener#6838, @ialidzhikov)
• [OPERATOR] Remove /scale subresource from etcd CRD. (gardener/gardener#6850, @shreyas-s-rao)

📖 Documentation

• [OPERATOR] The documentation for triggering control-plane migration is updated with a slight change. (gardener/gardener#6843, @shafeeqes)

🏃 Others

• [OPERATOR] The following image is updated: (gardener/gardener#6790, @ialidzhikov)
  • grafana/grafana: 7.5.16 -> 7.5.17
• [OPERATOR] The following image is updated: (gardener/gardener#6820, @Kristian-ZH)
  • quay.io/brancz/kube-rbac-proxy: v0.13.0 -> v0.13.1
• [OPERATOR] The following image is updated: (gardener/gardener#6824, @rickardsjp)
  • quay.io/prometheus/prometheus: v2.38.0 -> v2.39.1
• [OPERATOR] kubernetes.io/arch label can now be used for scaling the worker pools from 0 based on CPU architecture. (gardener/gardener#6825, @acumino)
• [OPERATOR] Deploy network policies to namespace istio-system to only allow traffic to configured endpoints inside the cluster and the seed api-server. (gardener/gardener#6826, @axel7born)
• [OPERATOR] The gardener.cloud/purpose: kube-system label is now added to the kube-system namespace by the gardenlet's Seed controller. (gardener/gardener#6829, @bd3lage)
• [OPERATOR] The following image is updated: (gardener/gardener#6828, @ialidzhikov)
  • eu.gcr.io/gardener-project/gardener/apiserver-proxy-pod-webhook: v0.6.0 -> v0.7.0
• [OPERATOR] Latency metrics of the attach subresource are not considered for the KubeApiServerLatency alert and API Server / Request Latency dashboard panel. (gardener/gardener#6844, @istvanballok)
• [OPERATOR] The ShootBinding admission plugin is removed in favour of existing ShootValidator plugin. All the checks are moved to the latter. (gardener/gardener#6727, @shafeeqes)
• [OPERATOR] When gardenlet checks the conditions of extension resources as part of the shoot health check, it checks if the gardener-extensions-heartbeat Lease maintained by the extension controllers has been renewed within the ShootCare controller's staleExtensionHealthChecks.thresholds[] settings and sets the corresponding Shoot condition to Unknown if that is not the case. If the Lease is not found, the status.conditions[].LastUpdateTime of the extension resource is checked as well for backwards compatibility. (gardener/gardener#6626, @plkokanov)
• [OPERATOR] Deploy network policies to namespace istio-ingress to only allow egress traffic to configured endpoints inside the cluster. (gardener/gardener#6765, @axel7born)
• [OPERATOR] Replace vpa-exporter with kube-state-metrics. (gardener/gardener#6771, @istvanballok)
  • The vpa-exporter is no longer used in Gardener.
  • The kube-state-metrics component is exposing the VPA related metrics.
• [DEVELOPER] Go is updated to 1.19.2 (gardener/gardener#6789, @oliver-goetz)

[hvpa-controller]

🏃 Others

• [DEPENDENCY] The version of golang used by hvpa-controller was updated from 1.15 to 1.18 (gardener/hvpa-controller#109, @andrerun)

[logging]

🏃 Others

• [OPERATOR] Published docker images for Logging are now multi-arch ready. They support linux/amd64 and linux/arm64. (gardener/logging#156, @acumino)
• [OPERATOR] Upgrade the Telegraf version from 1.23.4 to 1.24.2 (gardener/logging#157, @vlvasilev)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.58.0
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.58.0
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.58.0
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.58.0
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.58.0
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.58.0
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.58.0

[gardener]

🐛 Bug Fixes

• [OPERATOR] An issue causing the gardener-shoot-controlplane PriorityClass to be deleted too early when there are still Deployments (vpn-seed-server) that reference it is now mitigated. (gardener/gardener#6800, @gardener-ci-robot)
• [OPERATOR] The broken preStop hook from Gardener API Server deployment have been removed. (gardener/gardener#6797, @gardener-ci-robot)
• [OPERATOR] The gardenlet is no longer put under time pressure during its start-up procedure by preventing its liveness probe from falsely failing. (gardener/gardener#6815, @gardener-ci-robot)

🏃 Others

• [OPERATOR] The following image is updated: (gardener/gardener#6802, @gardener-ci-robot)
  • grafana/grafana: 7.5.16 -> 7.5.17

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.57.1
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.57.1
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.57.1
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.57.1
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.57.1
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.57.1
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.57.1

[gardener]

⚠️ Breaking Changes

• [USER] The create/modify/delete permissions for ServiceAccounts assigned to Project members with the admin role are now removed. Read permissions are preserved. In order to fully manage ServiceAccounts in the project namespace, use the serviceaccountmanager role. Please find more information here. (gardener/gardener#6740, @dimityrmirchev)
• [OPERATOR] Deprecated functions DeprecatedDetermineError and DeprecatedDetermineErrorCodes will be dropped in the upcoming releases, extensions using these functions now need to use their own methods to get the error code from the errors. (gardener/gardener#6677, @acumino)
• [OPERATOR] gardenlets component configuration API has been changed in the following breaking ways: (gardener/gardener#6688, @rfranzke)
  • .server.http has been split into server.{healthProbes,metrics} (health endpoints and metrics are now served on different ports)
  • .server.https has been removed
• [OPERATOR] gardenlet serves health endpoints and metrics on different ports now. Adapt your scrape configs accordingly to port metrics. (gardener/gardener#6688, @rfranzke)
• [OPERATOR] The metrics port of the gardener-scheduler is no longer hard-coded to 9090 but now uses the same value as the container target port (configurable via the component configuration). (gardener/gardener#6690, @rfranzke)
• [OPERATOR] The server.https field of the gardener-admission-controller configuration has been renamed to server.webhooks. Likewise, the Gardener control plane Helm chart has been changed. Please adapt your values.yaml files. (gardener/gardener#6706, @rfranzke)
• [OPERATOR] The ShootCARotation and ShootSARotation feature gates have been promoted to beta and are now enabled by default. Make sure that all provider extensions registered to your system support these features before upgrading to this Gardener version. (gardener/gardener#6734, @rfranzke)

✨ New Features

• [DEVELOPER] There is a new variant for running the local setup remotely. This can be helpful if your workstation does only have limited resources available (CPUs, memory). Please refer to this or this document. (gardener/gardener#6730, @istvanballok)

🐛 Bug Fixes

• [USER] A bug has been fixed that caused custom containerd config from /etc/containerd/conf.d not to be loaded. (gardener/gardener#6754, @timebertt)
• [OPERATOR] Prevent potential nil pointer exception in gardener-apiserver if Shoot's .spec.worker.machine.architecture is set to nil. The issue could only occur if the version skew of Gardener is not respected and minor version is skipped during the Gardener update. (gardener/gardener#6716, @breuerfelix)
• [OPERATOR] Gardener will now keep any custom annotations that were put to Etcd resources in the seed cluster. This can help if operators need to manually restore an ETCD cluster in exceptional cases. (gardener/gardener#6757, @timuthy)

🏃 Others

• [OPERATOR] The following dependency is updated: (gardener/gardener#6737, @acumino)
  • sigs.k8s.io/controller-tools: v0.9.2 -> v0.10.0
• [OPERATOR] Updated vertical-pod-autoscaler to v0.12.0. (gardener/gardener#6739, @shafeeqes)
• [OPERATOR] Logs from pods managed by garden-resource-manager will be scraped and stored in the shoot's Loki. (gardener/gardener#6748, @vlvasilev)
• [DEVELOPER] Changes in static and embedded files, e.g. files in charts, are now considered when running gardener-up. This results in a new CRI image (typically gardenlet or provider-local) that is deployed to the local garden cluster. (gardener/gardener#6735, @timuthy)
• [DEVELOPER] Change filename of containerd config file in provider-local. (gardener/gardener#6753, @oliver-goetz)
• [DEVELOPER] The cluster-autoscaler's scale from/to zero integration test is no longer skipped on providers other than aws and azure. (gardener/gardener#6767, @ialidzhikov)

[etcd-backup-restore]

🐛 Bug Fixes

• [OPERATOR] A bug has been fixed that caused the wrong minAvailable configuration being calculated for multi-node etcd PodDisruptionBudget. (gardener/etcd-backup-restore#441, @timuthy)

🏃 Others

• [OPERATOR] Decreases the likelihood of potential race condition between the go-routines while closing the snapshotter. (gardener/etcd-backup-restore#537, @ishan16696)
• [OPERATOR] [bug-fix] backup-restore does not return error when it fails to update PeerURL of member. (gardener/etcd-backup-restore#540, @aaronfern)
• [OPERATOR] The BackupReady condition is not considered anymore when the PodDisruptionBudget configuration is calculated. This earlier blocked rolling out fixes that potentially solved problems with backup procedures. (gardener/etcd-backup-restore#441, @timuthy)

[logging]

🐛 Bug Fixes

• [OPERATOR] Fix sending on a closed channel in the fluent-bit's plugin SortedClient when closing it before the last batch is sent. (gardener/logging#153, @vlvasilev)

🏃 Others

• [DEVELOPER] Remove the alpine image used as a carrier for the fluent-bit-to-loki plugin. (gardener/logging#154, @vlvasilev)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.57.0
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.57.0
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.57.0
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.57.0
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.57.0
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.57.0
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.57.0

[etcd-backup-restore]

🏃 Others

• [OPERATOR] Decreases the likelihood of potential race condition between the go-routines while closing the snapshotter. (gardener/etcd-backup-restore#537, @ishan16696)
• [OPERATOR] [bug-fix] backup-restore does not return error when it fails to update PeerURL of member. (gardener/etcd-backup-restore#540, @aaronfern)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.56.1
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.56.1
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.56.1
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.56.1
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.56.1
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.56.1
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.56.1

[gardener]

🐛 Bug Fixes

• [OPERATOR] Added handling for v1alpha1 config of PodSecurity admission plugin for clusters v1.22.x. (gardener/gardener#6663, @gardener-ci-robot)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.55.1
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.55.1
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.55.1
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.55.1
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.55.1
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.55.1
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.55.1

[gardener]

⚠️ Breaking Changes

• [OPERATOR] Gardener has been being prepared for more shoot HA use-cases and thus some assumption about currently running landscapes are required: If you use a multi-zonal labelled seed and scheduled non-HA shoots onto it, this release of Gardener will potentially cause scheduling conflicts to the control-plane pods as it will try to locate all pods into a single zone only. Pods that can't be re-scheduled (mainly because of volume dependencies) will remain in Pending state. (gardener/gardener#6579, @timuthy)

✨ New Features

• [USER] Introduce shoot spec field spec.controlPlane to allow enabling HA control planes with failure tolerance type of node or zone. Please consult docs/usage/shoot_high_availability.md for more information. (gardener/gardener#6530, @shreyas-s-rao)
• [OPERATOR] Gardener is prepared to run non-HA and single-zonal shoots on multi-zonal seeds. In such a setup, control-plane pods of the mentioned shoots are scheduled into a single availability zone only to avoid any extra cross zonal traffic that would usually involve higher latency and cost. PLEASE NOTE: The StorageClass in seeds used for control-plane components must have volumeBindingMode: WaitForFirstConsumer to let the zone-pinning work properly. (gardener/gardener#6579, @timuthy)
• [OPERATOR] Gardener can now support shoot clusters with Kubernetes version 1.25. In order to allow creation/update of 1.25 clusters you will have to update the version of your provider extension(s) to a version that supports 1.25 as well. Please consult the respective releases and notes in the provider extension's repository. (gardener/gardener#6638, @shafeeqes)
• [OPERATOR] gardener-admission-controller's log level and log format can be now configured. (gardener/gardener#6652, @krgostev)
• [DEVELOPER] The e2e tests do now also tear down the Gardener environment, effectively verifying whether the Seed deletion works as expected. (gardener/gardener#6664, @plkokanov)
• [DEVELOPER] Gardener can now support shoot clusters with Kubernetes version 1.25. Extension developers have to prepare individual extensions as well to work with 1.25. (gardener/gardener#6638, @shafeeqes)

🐛 Bug Fixes

• [OPERATOR] gardener-apiserver now validates that the CloudProfile's .spec.seedSelector is matching Shoot's Seed when the .spec.seedName field of the Shoot is set or modified. (gardener/gardener#6680, @ialidzhikov)
• [OPERATOR] Added handling for v1alpha1 config of PodSecurity admission plugin for clusters v1.22.x. (gardener/gardener#6649, @shafeeqes)

📖 Documentation

• [USER] Architecture diagram was updated discouraging the use of the Kubernetes dashboard (among other hints and cosmetic updates). (gardener/gardener#6701, @vlerenc)

🏃 Others

• [USER] Add validations to disallow switching failure tolerance type for HA shoot control planes between node and zone. (gardener/gardener#6530, @shreyas-s-rao)
• [OPERATOR] The Gardener scheduler does now consider multi-zonal seeds as potential candidates for non-HA and single-zonal shoots. (gardener/gardener#6530, @shreyas-s-rao)
• [OPERATOR] The node-problem-detector image is updated from eu.gcr.io/gardener-project/3rd/node-problem-detector:v0.8.10-gardener.1 to registry.k8s.io/node-problem-detector/node-problem-detector:v0.8.12. (gardener/gardener#6660, @ialidzhikov)
• [OPERATOR] Adding an alpha HA annotation to the shoot spec where none existed is now allowed. (gardener/gardener#6533, @unmarshall)
• [OPERATOR] Fixed an issue that caused make gardener-down to fail when deleting the garden Project. (gardener/gardener#6664, @plkokanov)
• [OPERATOR] A Pod Topology Spread Constraints webhook has been added to the Gardener-Resource-Manager which mimics the matchLabelKeys feature on the pod-template-hash label. Gardener uses this webhook to circumvent imbalanced control plane deployments across nodes and zones. (gardener/gardener#6665, @timuthy)
• [OPERATOR] The following image is updated: (gardener/gardener#6670, @istvanballok)
  • ghcr.io/prometheus-operator/prometheus-config-reloader: v0.58.0 -> v0.59.1
• [OPERATOR] The kube-apiserver deployment was changed from pod anti-affinity to Topology Spread Constraints. Non-HA shoot clusters will still have the kube-apiserver pods being scheduled on different nodes on a best-effort basis. For HA clusters, the Topology Spread Constraints make sure that a distribution across nodes (single-zone) and zones (multi-zonal) is guaranteed, in order to tolerate failures in these domains. (gardener/gardener#6674, @timuthy)
• [OPERATOR] Add a panel "Response Size Rate" to the API Server dashboard (gardener/gardener#6675, @istvanballok)
• [OPERATOR] The gardener-resource-manager deployment was changed from pod anti-affinity to Topology Spread Constraints. Non-HA shoot clusters will still have the gardener-resource-manager pods being scheduled on different nodes on a best-effort basis. For HA clusters, the Topology Spread Constraints make sure that a distribution across nodes (single-zone) and zones (multi-zonal) is guaranteed, in order to tolerate failures in these domains. (gardener/gardener#6685, @timuthy)
• [OPERATOR] VPA components do now have a liveness probe defined. (gardener/gardener#6585, @andrerun)
• [OPERATOR] Kubernetes admission plugins that can be specified in shoot.kubernetes.apiServer.admissionPlugins are now validated aginst the kubernetes version of the shoot cluster. (gardener/gardener#6625, @plkokanov)
• [OPERATOR] Update vpa-exporter:0.1.5->0.3.0 (gardener/gardener#6640, @istvanballok)
  • Add targetName and targetKind labels
  • Added unit-tests and added a check for no targetRef.
  • Updated alpine image.
  • Added a new metric to export new VPA recommendations provided via an annotation.
  • Published docker images for VPA-Exporter are now multi-arch ready. They support linux/amd64 and linux/arm64.
  • The vpa-exporter container now uses distroless instead of alpine as a base image.
• [OPERATOR] gardenlet is now using gcr.io/distroless/static-debian11:nonroot instead of versions of alpine as a base image. (gardener/gardener#6641, @acumino)
• [OPERATOR] The istio ingress gateway prefers backends within the same availability zone to reduce cross-zonal traffic. (gardener/gardener#6653, @ScheererJ)
• [DEVELOPER] The number of e2e tests carried out in parallel is configurable now. (gardener/gardener#6682, @oliver-goetz)
• [DEVELOPER] Golang is updated to 1.19.1. (gardener/gardener#6650, @oliver-goetz)
• [DEPENDENCY] The following dependency is updated: (gardener/gardener#6668, @shafeeqes)
  • k8s.io/* : v0.24.4 -> v0.25.0
  • sigs.k8s.io/controller-runtime: v0.12.3 -> v0.13.0

[apiserver-proxy]

🏃 Others

• [OPERATOR] Updated base image of apiserver-proxy to alpine 3.16.2 (gardener/apiserver-proxy#21, @ScheererJ)

[etcd-backup-restore]

🐛 Bug Fixes

• [OPERATOR] To avoid potential race-condition between go-routines updated probeEtcd func() to use shorter timeout. (gardener/etcd-backup-restore#532, @ishan16696)

🏃 Others

• [OPERATOR] Always update member peer URL, changed the way scale-up of etcd cluster is identified. (gardener/etcd-backup-restore#534, @unmarshall)

📰 Noteworthy

• [OPERATOR] Updated golang version used to build images to 1.18.6 (gardener/etcd-backup-restore#535, @aaronfern)
• [OPERATOR] Updated golang version to 1.18 (gardener/etcd-backup-restore#535, @aaronfern)

[etcd-custom-image]

🏃 Others

• [OPERATOR] Etcd-custom-image will now retry fetching etcd configuration in case of any error (gardener/etcd-custom-image#26, @aaronfern)

[etcd-druid]

🏃 Others

• [OPERATOR] Enables etcd-druid to trigger restarts of the etcd pods when peer URL is TLS enabled. (gardener/etcd-druid#421, @unmarshall)
• [OPERATOR] Golang version used upgraded to 1.18.6 (gardener/etcd-druid#431, @aaronfern)
• [OPERATOR] etcd-custom-image upgraded to v3.4.13-bootstrap-8 (gardener/etcd-druid#432, @aaronfern)

[hvpa-controller]

🏃 Others

• [OPERATOR] The hvpa-controller container image now uses a non root user by default. (gardener/hvpa-controller#103, @dimityrmirchev)

[vpn2]

📰 Noteworthy

• [OPERATOR] Updated base image of vpn seed server and vpn shoot client to alpine 3.16.2 (gardener/vpn2#18, @ScheererJ)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.56.0
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.56.0
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.56.0
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.56.0
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.56.0
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.56.0
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.56.0