[gardener]

🏃 Others

• [OPERATOR] Error messages containing RequestLimitExceeded are now treated as ERR_INFRA_RATE_LIMITS_EXCEEDED (instead of ERR_INFRA_QUOTA_EXCEEDED before). (#4255, @vpnachev)
• [OPERATOR] Split envoyfilter resources from SNI and ReversedVPN into separate resources. (#4252, @DockToFuture)
• [OPERATOR] gardener-controller-manager's Seed controller now checks the seed namespace's ownerReferences before adopting it. (#4233, @timebertt)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.25.1
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.25.1
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.25.1
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.25.1
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.25.1
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.25.1
landscaper-gardenlet: eu.gcr.io/gardener-project/gardener/landscaper-gardenlet:v1.25.1

[gardener]

✨ New Features

• [USER] Allow changing container runtime on existing workers. This triggers a graceful recreation of the workers. (#4171, @voelzmo)
• [OPERATOR] Operators can now easily browse all available metrics and logs via Grafana's Explore view without logging into Grafana explicitly (basic auth via Ingress is still enabled). (#4140, @timebertt)
• [DEVELOPER] Enable runtime configuration of hook-me.sh inlets pod port (#4181, @deitch)

🐛 Bug Fixes

• [OPERATOR] Fixed incorrect usage of shoot dns server in network policy inside the control plane where the seed dns server should have been used. This is only relevant if node local dns is activated. (#4168, @ScheererJ)
• [DEVELOPER] Pull inlets image for hack/hook-me.sh from its proper location (#4146, @deitch)

🏃 Others

• [USER] If the gardener-resource-manager is unable to apply/reconcile its desired state due to a conflicting state of resources in the shoot then it will be marked as "configuration problem" error. (#4177, @rfranzke)
• [OPERATOR] Shoots are no longer automatically requeued when an extension controller is updated. (#4189, @rfranzke)
• [OPERATOR] The error code mapping has been extended such that more quota-related issues are properly detected. (#4178, @rfranzke)
• [OPERATOR] Terraform pods stuck in creation are now deleted after 5m instead of being waited for the configured deadline period (usually > 20m/30m). (#4176, @rfranzke)
• [OPERATOR] Upgrade grafana to 7.5.7 (#4170, @wyb1)
• [OPERATOR] When removing Kubernetes or image versions from a cloud profile, shoots currently being deleted are now ignored. (#4165, @stoyanr)
• [OPERATOR] Logs from Shoots in deletion phase are kept in the Loki instance in the garden namespace. (#4137, @vlvasilev)
• [OPERATOR] All time period based settings now include the measure unit (#4137, @vlvasilev)
• [OPERATOR] Prometheus scrape configuration and a dashboard for the kubernetes node-local-dns feature are added to gardener. (#4136, @DockToFuture)
• [OPERATOR] Updated core dns to 1.8.4. (#4133, @ScheererJ)
• [OPERATOR] Added alert to monitor for any snapshotter failure in etcd-backup-restore (#4094, @aaronfern)

📰 Noteworthy

• [USER] In case the shoot VPN tunnel connection does not work and the classic VPN solution is used, the latest 'warning' events for the vpn-shoot service are now read and returned to the user. (#4183, @rfranzke)
• [USER] The .spec.kubernetes.kubeAPIServer.oidc.signingAlgs[] list is now validated such that it may only contain algorithms listed in https://datatracker.ietf.org/doc/html/rfc7518#section-3.1. (#4175, @rfranzke)
• [OPERATOR] It's no longer allowed to delete a ManagedSeed if there are shoots scheduled on its seed. (#4166, @stoyanr)
• [OPERATOR] When creating or updating shoots, any Kubernetes feature gates mentioned are validated against the Kubernetes version. If any feature gates are unknown or not supported in the Kubernetes version, the validation fails. (#4149, @stoyanr)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.25.0
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.25.0
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.25.0
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.25.0
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.25.0
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.25.0
landscaper-gardenlet: eu.gcr.io/gardener-project/gardener/landscaper-gardenlet:v1.25.0

[gardener]

🏃 Others

• [OPERATOR] The error code mapping has been extended such that more quota-related issues are properly detected. (#4186, @rfranzke)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.24.1
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.24.1
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.24.1
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.24.1
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.24.1
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.24.1
landscaper-gardenlet: eu.gcr.io/gardener-project/gardener/landscaper-gardenlet:v1.24.1

[gardener]

🏃 Others

• [OPERATOR] The error code mapping has been extended such that more quota-related issues are properly detected. (#4185, @rfranzke)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.23.3
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.23.3
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.23.3
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.23.3
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.23.3
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.23.3
landscaper-gardenlet: eu.gcr.io/gardener-project/gardener/landscaper-gardenlet:v1.23.3

[gardener]

🐛 Bug Fixes

• [OPERATOR] The Gardener API server now allows Bastion to be specified for ControllerRegistration .spec.resources[].type. (#4092, @ialidzhikov)

🏃 Others

• [OPERATOR] The error code mapping has been extended such that more quota-related issues are properly detected. (#4184, @rfranzke)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.22.6
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.22.6
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.22.6
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.22.6
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.22.6
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.22.6
landscaper-gardenlet: eu.gcr.io/gardener-project/gardener/landscaper-gardenlet:v1.22.6

[gardener]

⚠️ Breaking Changes

• [OPERATOR] The admission plugin ShootStateDeletionValidator is removed. Explicitly enabling or disabling it via the gardener-api-server will cause the gardener-api-server to fail to start. This fixes an error caused by a not-in-time cleaned up ShootState resulting in Shoot creation to fail if a Shoot was deleted and created with the same name in quick succession. (#4100, @BeckerMax)

✨ New Features

• [USER] It is now possible to enable anonymous authentication on the kube-apiserver for shoots by setting .spec.kubernetes.kubeAPIServer.enableAnonymousAuthentication=true. Anonymous authentication will be disabled by default. (#4072, @dimityrmirchev)
• [USER] If enabled in the gardener-apiserver, AdminKubeConfigRequest can be used to issue a kubeconfig with cluster-admin privileges for shoot clusters. The default expiration for such request is one hour, but the expiration time can be configured by setting .spec.expirationSeconds (minimum 10 minutes or 600 seconds). (#3932, @mvladev)
• [OPERATOR] New AdminKubeConfigRequest alpha feature gate enables AdminKubeConfigRequest subresource on shoot resources. The feature gate is disabled by default in the gardener-apiserver and must be explicitly enabled. (#3932, @mvladev)
• [OPERATOR] New --shoot-admin-kubeconfig-max-expiration flag in gardener-apiserver allows to specify the maximum validity duration of a credential requested to a Shoot by an AdminKubeconfigRequest. If an otherwise valid AdminKubeconfigRequest with a validity duration larger than this value is requested, a credential will be issued with a validity duration of this value. This flag is only effective when AdminKubeConfigRequest feature gate is enabled. (#3932, @mvladev)

🐛 Bug Fixes

• [OPERATOR] The Gardener API server now allows Bastion to be specified for ControllerRegistration .spec.resources[].type. (#4090, @ialidzhikov)
• [DEPENDENCY] The hack/generate-controller-registration.sh script does now produce valid ControllerDeployment resources. (#4088, @rfranzke)

🏃 Others

• [OPERATOR] Update coredns to 1.8.3. (#4116, @DockToFuture)
• [OPERATOR] The following images are updated: (#4104, @ialidzhikov)
  • k8s.gcr.io/autoscaling/vpa-admission-controller: 0.9.0 -> 0.9.2
  • k8s.gcr.io/autoscaling/vpa-recommender: 0.9.0 -> 0.9.2
  • k8s.gcr.io/autoscaling/vpa-updater: 0.9.0 -> 0.9.2
• [OPERATOR] Istio, used by the ManagedIstio feature gate, is upgraded from 1.8.0 to 1.9.5 (#4101, @mvladev)
• [OPERATOR] SNI feature gate: Prevent throttling by increasing requests and limits for the istio-ingressgateway envoy proxies & limit the used worker threads. (#4080, @danielfoehrKn)
• [OPERATOR] Fixed a race in shoot cluster deletion, which could affect other clusters as the envoy filter (as part of the kube-apiserver-sni) was not deleted before the kube-apiserver-service. This is now explicitly ensured. (#4068, @ScheererJ)
• [OPERATOR] Gardener now supports using worker-controller generated bootstrap-tokens for machines, see new flow here. ⚠️ If you maintain an infrastructure extension make sure to use a worker controller that supports generating a bootstrap token, see here and if you maintain an os-extension make sure to support the transmitUnencoded flag similar to os-gardenlinx. Currently, the old flow is still supported but we plan to deprecate it in the future. (#3902, @BeckerMax)

📰 Noteworthy

• [DEVELOPER] If a milestone for the next minor version exists then PRs to the master branch are only mergeable if they are assigned to this milestone. (#4085, @rfranzke)

[etcd-druid]

🏃 Others

• [OPERATOR] Updated number of chunks while uploading to never exceed the cloud provider limits. (gardener/etcd-druid#183, @amshuman-kr)
• [OPERATOR] Removed synchronisation before updating ETCD status. (gardener/etcd-druid#176, @amshuman-kr)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.24.0
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.24.0
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.24.0
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.24.0
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.24.0
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.24.0
landscaper-gardenlet: eu.gcr.io/gardener-project/gardener/landscaper-gardenlet:v1.24.0

[etcd-druid]

🏃 Others

• [OPERATOR] Updated number of chunks while uploading to never exceed the cloud provider limits. (gardener/etcd-druid#183, @amshuman-kr)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.23.2
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.23.2
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.23.2
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.23.2
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.23.2
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.23.2
landscaper-gardenlet: eu.gcr.io/gardener-project/gardener/landscaper-gardenlet:v1.23.2

[gardener]

🐛 Bug Fixes

• [OPERATOR] The Gardener API server now allows Bastion to be specified for ControllerRegistration .spec.resources[].type. (#4091, @ialidzhikov)
• [DEPENDENCY] The hack/generate-controller-registration.sh script does now produce valid ControllerDeployment resources. (#4089, @ialidzhikov)

🏃 Others

• [OPERATOR] SNI feature gate: Prevent throttling by increasing requests and limits for the istio-ingressgateway envoy proxies & limit the used worker threads. (#4082, @danielfoehrKn)

[etcd-druid]

🏃 Others

• [OPERATOR] Removed synchronisation before updating ETCD status. (gardener/etcd-druid#176, @amshuman-kr)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.23.1
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.23.1
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.23.1
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.23.1
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.23.1
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.23.1
landscaper-gardenlet: eu.gcr.io/gardener-project/gardener/landscaper-gardenlet:v1.23.1

[gardener]

🏃 Others

• [OPERATOR] SNI feature gate: Prevent throttling by increasing requests and limits for the istio-ingressgateway envoy proxies & limit the used worker threads. (#4081, @danielfoehrKn)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.22.5
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.22.5
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.22.5
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.22.5
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.22.5
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.22.5
landscaper-gardenlet: eu.gcr.io/gardener-project/gardener/landscaper-gardenlet:v1.22.5

[gardener]

🏃 Others

• [OPERATOR] Fixed an issue that prevented the update and deletion of managed seeds on soil clusters. (#4075, @stoyanr)
• [OPERATOR] Adjust hvpa limitsRequestsGapScaleParams for prometheus (#4058, @wyb1)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.22.4
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.22.4
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.22.4
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.22.4
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.22.4
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.22.4
landscaper-gardenlet: eu.gcr.io/gardener-project/gardener/landscaper-gardenlet:v1.22.4

[gardener]

🏃 Others

• [OPERATOR] Fixed an issue that prevented the update and deletion of managed seeds on soil clusters. (#4074, @stoyanr)
• [OPERATOR] Increase limits for metrics-server. This is a temporary fix until we have non-circular auto-scaling for metrics-server. (#4035, @prashanth26)

[autoscaler]

🐛 Bug Fixes

• [OPERATOR] Allow scaling down of machine with already lowered priority (gardener/autoscaler#75, @prashanth26)
• [DEVELOPER] Avoids panics when VM type isn't found during scale from zero (gardener/autoscaler#78, @prashanth26)
• [DEVELOPER] Fetches the VM from the correct map for MCM provider Azure and hence doesn't panic anymore (gardener/autoscaler#78, @prashanth26)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.21.2
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.21.2
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.21.2
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.21.2
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.21.2
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.21.2
landscaper-gardenlet: eu.gcr.io/gardener-project/gardener/landscaper-gardenlet:v1.21.2

[gardener]

⚠️ Breaking Changes

• [OPERATOR] Gardener API server does no longer allow creating a Seed and ManagedSeed with . (dot) in the name. Before upgrading to this version of Gardener, make sure that you don't have Seed or ManagedSeed with . (dot) in the system. (#3929, @ialidzhikov)

🏃 Others

• [OPERATOR] Fixed an issue that prevented the update and deletion of managed seeds on soil clusters. (#4073, @stoyanr)

[autoscaler]

🐛 Bug Fixes

• [OPERATOR] Allow scaling down of machine with already lowered priority (gardener/autoscaler#75, @prashanth26)
• [DEVELOPER] Avoids panics when VM type isn't found during scale from zero (gardener/autoscaler#78, @prashanth26)
• [DEVELOPER] Fetches the VM from the correct map for MCM provider Azure and hence doesn't panic anymore (gardener/autoscaler#78, @prashanth26)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.20.5
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.20.5
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.20.5
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.20.5
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.20.5
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.20.5
landscaper-gardenlet: eu.gcr.io/gardener-project/gardener/landscaper-gardenlet:v1.20.5

[gardener]

⚠️ Breaking Changes

• [DEVELOPER] Deployment specific information from ControllerRegistrations was outsourced to a new resource type ControllerDeployment. If you use Gardener's /hack scripts and utilities, please make sure to run make generate after you have updated your Gardener dependencies to this version, as it updates the example registration in your extension. Otherwise, please make sure to adjust your example ControllerRegistration manifest to the new structure. (#3995, @timuthy)

✨ New Features

• [USER] The shoot resource now support optional columns that are shown only when kubectl get is used with the -o wide flag. (#4032, @vpnachev)
  • The optional columns are: SEED, PURPOSE, GARDENER VERSION, APISERVER, CONTROL, NODES and SYSTEM.
  • New mandatory columns REGION and STATUS are introduced.
• [USER] Add the resource ShootExtensionStatus to the resource group core.gardener.cloud which is intended to hold the information of the ProviderStatus field from multiple Gardener Extension resources from the Seed cluster. (#3878, @danielfoehrKn)
• [OPERATOR] A new resource ControllerDeployment was added to the core.gardener.cloud API group. A ControllerDeployment is supposed to contain specifications about how extension controllers are deployed to seed clusters. Specifying deployments via ControllerRegistrations directly is therefore deprecated. Instead, we recommend to use a dedicated ControllerDeployment resource and reference it in a ControllerRegistration via spec.deployment.DeploymentRefs[].Name. Please consult the documentation for more information. (#3995, @timuthy)
  • As ControllerDeployments may contain sensitive information, we recommend to add this type to the Gardener API server's EncryptionConfiguration. This happens automatically if you deploy Gardener via our pre-configured Helm charts (/charts/gardener).
• [OPERATOR] Introduce compressed etcd backups for faster upload and download of etcd snapshots. (#3953, @shreyas-s-rao)

🐛 Bug Fixes

• [OPERATOR] A bug generally preventing ManagedSeed deletion has been fixed. (#4045, @rfranzke)
• [OPERATOR] Fix an issue where the gardenlet overwrites changes from hvpa for the aggregate-prometheus. (#3998, @wyb1)

🏃 Others

• [OPERATOR] Fixed an issue that prevented the update and deletion of managed seeds on soil clusters. (#4070, @stoyanr)
• [OPERATOR] Gardener administrators are now allowed to access certificatesigningrequests. (#4059, @ialidzhikov)
• [OPERATOR] Some shoot clusters try to wrongly configure the kube-apiserver and it fails to start. Since some time, Gardener shows the last 10 log lines for such kube-apiservers in the `shoot..status.lastOperation.description', but error message with the useful information might not be in those lines. Therefore, for shoots clusters running on k8s <1.19 now also the first 1KiB of logs will be shown. (#4050, @vpnachev)
• [OPERATOR] Adjust hvpa limitsRequestsGapScaleParams for prometheus (#4049, @wyb1)
• [OPERATOR] Apply a mitigation that will prevent gardenlet to panic under certain circumstances. (#4046, @ialidzhikov)
• [OPERATOR] istio images are now pulled from the official mirrors to Google Container Registry to prevent any potential dockerhub rate limit issues in environments that use the default images (don't specify any imagevector overwrite). (#4022, @ialidzhikov)
• [OPERATOR] Increase limits for metrics-server. This is a temporary fix until we have non-circular auto-scaling for metrics-server. (#4017, @amshuman-kr)
• [OPERATOR] Disable etcd scale down for shoot clusters with purpose "production". This avoids multiple etcd restarts during the shoot's maintenance time window if VPA recommendation for scale down is inappropriate. (#4016, @amshuman-kr)
• [DEVELOPER] The unknown github label area/operations has been removed from the github issues templates. (#4013, @vpnachev)

📰 Noteworthy

• [OPERATOR] The .gardener.garden.identity value (deprecated with v1.11.0, removed with v1.22.0) is added again and will be passed to the Helm chart values of ControllerInstallations. It's still deprecated and planned to be removed in a future version, hence, Gardener operators have to make sure to update affected provider extensions accordingly. (#4021, @rfranzke)

[autoscaler]

🐛 Bug Fixes

• [DEVELOPER] Avoids panics when VM type isn't found during scale from zero (gardener/autoscaler#78, @prashanth26)
• [DEVELOPER] Fetches the VM from the correct map for MCM provider Azure and hence doesn't panic anymore (gardener/autoscaler#78, @prashanth26)

🏃 Others

• [OPERATOR] Bug fix: Allow scaling down of machine with already lowered priority (gardener/autoscaler#75, @prashanth26)

[etcd-backup-restore]

🏃 Others

• [USER] Added CLI flags --auto-compaction-mode and --auto-compaction-retention to configure auto-compaction for embedded etcd. Default values: auto-compaction-mode="periodic" and auto-compaction-retention="30m" (gardener/etcd-backup-restore#315, @ishan16696)
• [OPERATOR] Added a new metric etcdbr_snapshotter_failure used as a blanket metric to capture any snapshotter error. (gardener/etcd-backup-restore#326, @aaronfern)
• [OPERATOR] etcdbr-compression specification can be configured through helm-charts. (gardener/etcd-backup-restore#307, @ishan16696)
• [OPERATOR] Added CLI-flags (max-call-send-message-size, max-request-bytes and max-txn-ops) to enable restoration for delta snapshots with large amount of data (large number of events or events with large data). (gardener/etcd-backup-restore#282, @abdasgupta)

📰 Noteworthy

• [USER] Move bootstrap script out of the helm chart, as it will now be part of the etcd-custom-image. (gardener/etcd-backup-restore#327, @shreyas-s-rao)
• [USER] Add support for snapshot compression/decompression. Compression and compression policy can be configured through flags: --compress-snapshots and --compression-policy respectively. Supported compression policies currently are gzip (default), lzw and zlib. Snapshot compression is disabled by default. (gardener/etcd-backup-restore#293, @ishan16696)

[etcd-druid]

✨ New Features

• [USER] Configure auto-compaction policy for etcd and backup sidecar's embedded etcd via Etcd resource via .spec.sharedConfig.autoCompactionMode and .spec.sharedConfig.autoCompactionRetention. (gardener/etcd-druid#157, @ishan16696)
• [OPERATOR] Etcd bootstrap script now resides in the custom etcd image instead of being mounted as a configmap. (gardener/etcd-druid#162, @shreyas-s-rao)

🏃 Others

• [OPERATOR] If an etcd StatefulSet remains pending, warning events of unbound PVCs are now added to the .status.lastError of the etcd resource. This makes it easier for operators to spot potential issues. (gardener/etcd-druid#146, @timuthy)
• [OPERATOR] Snapshot compression specification can be configured through helm-charts as well as etcd resource spec configuration file. (gardener/etcd-druid#138, @ishan16696)

📰 Noteworthy

• [USER] Moved parts of reconciliation for ETCD status that deals with Statefulset to a separate controller. (gardener/etcd-druid#133, @abdasgupta)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.23.0
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.23.0
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.23.0
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.23.0
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.23.0
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.23.0
landscaper-gardenlet: eu.gcr.io/gardener-project/gardener/landscaper-gardenlet:v1.23.0

[gardener]

🏃 Others

• [OPERATOR] Apply a mitigation that will prevent gardenlet to panic under certain circumstances. (#4047, @ialidzhikov)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.22.3
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.22.3
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.22.3
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.22.3
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.22.3
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.22.3
landscaper-gardenlet: eu.gcr.io/gardener-project/gardener/landscaper-gardenlet:v1.22.3

[gardener]

🏃 Others

• [OPERATOR] Increase limits for metrics-server. This is a temporary fix until we have non-circular auto-scaling for metrics-server. (#4033, @prashanth26)

📰 Noteworthy

• [OPERATOR] The .gardener.garden.identity value (deprecated with v1.11.0, removed with v1.22.0) is added again and will be passed to the Helm chart values of ControllerInstallations. It's still deprecated and planned to be removed in a future version, hence, Gardener operators have to make sure to update affected provider extensions accordingly. (#4024, @rfranzke)

[autoscaler]

🐛 Bug Fixes

• [DEVELOPER] Avoids panics when VM type isn't found during scale from zero (gardener/autoscaler#78, @prashanth26)
• [DEVELOPER] Fetches the VM from the correct map for MCM provider Azure and hence doesn't panic anymore (gardener/autoscaler#78, @prashanth26)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.22.2
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.22.2
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.22.2
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.22.2
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.22.2
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.22.2
landscaper-gardenlet: eu.gcr.io/gardener-project/gardener/landscaper-gardenlet:v1.22.2

[gardener]

⚠️ Breaking Changes

• [OPERATOR] Gardener API server does no longer allow creating a Seed and ManagedSeed with . (dot) in the name. Before upgrading to this version of Gardener, make sure that you don't have Seed or ManagedSeed with . (dot) in the system. (#3928, @ialidzhikov)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.21.1
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.21.1
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.21.1
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.21.1
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.21.1
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.21.1
landscaper-gardenlet: eu.gcr.io/gardener-project/gardener/landscaper-gardenlet:v1.21.1

[gardener]

🐛 Bug Fixes

• [OPERATOR] Fix an issue where the gardenlet overwrites changes from hvpa for the aggregate-prometheus. (#4009, @ialidzhikov)

[autoscaler]

🐛 Bug Fixes

• [OPERATOR] Bug fix: Allow scaling down of machine with the already lowered priority and nodes with ToBeDeletedByClusterAutoscaler taints. This solves issues where pods are pending due to nodes with such taints. (gardener/autoscaler#75, @prashanth26)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.22.1
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.22.1
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.22.1
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.22.1
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.22.1
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.22.1
landscaper-gardenlet: eu.gcr.io/gardener-project/gardener/landscaper-gardenlet:v1.22.1

[gardener]

⚠️ Breaking Changes

• [OPERATOR] There are now dedicated ClusterRoles for the gardener-apiserver and gardener-controller-manager. As the roleRef fields of the binding ClusterRoleBindings are immutable, also the names of the ClusterRoleBindings have been changed. When you apply this version of Gardener then you have to make sure to cleanup the old resources that will be still left in your system be running the following command: (#3975, @rfranzke)
  • kubectl delete clusterrolebinding/gardener.cloud:apiserver:admin clusterrolebinding/gardener.cloud:controller-manager:admin clusterrole/gardener.cloud:system:gardener-scheduler clusterrolebinding/gardener.cloud:system:gardener-scheduler --ignore-not-found --wait=false
• [OPERATOR] The .gardener.garden.identity value (deprecated with v1.11.0) is removed and no longer passed to the Helm chart values of ControllerInstallations. Gardener operators have to make sure to update affected provider extensions accordingly. (#3941, @rfranzke)
• [OPERATOR] Gardener API server does no longer allow creating a Seed and ManagedSeed with . (dot) in the name. Before upgrading to this version of Gardener, make sure that you don't have Seed or ManagedSeed with . (dot) in the system. (#3927, @ialidzhikov)
• [OPERATOR] The legacy garden/gardener-controller-manager-internal-config ConfigMap is now deleted on start-up of gardener-controller-manager. Please ensure that you run at least v1.20 of your gardenlets before upgrading to this version. (#3888, @rfranzke)
• [DEVELOPER] Because of the new optional field spec.machineTypes[].storage.minSize, spec.machineTypes[].storage.size is now optional as well. Please perform a nil check before accessing this field. (#3976, @timuthy)

✨ New Features

• [OPERATOR] A new field minSize has been added to spec.volumeTypes[] and spec.machineTypes[].storage of the CloudProfile. It allows to configure the minimum allowed size of a volume configured for shoots (shoot.spec.workers[].volume.size). (#3976, @timuthy)
• [OPERATOR] It is now possible to set custom values for kube-controller-manager --node-monitor-grace-period via .spec.kubernetes.kubeControllerManager.nodeMonitorGracePeriod (should not be less than 2m). (#3947, @mwennrich)
• [OPERATOR] The Gardenlet supports a new ReversedVPN feature gate (disabled by default). If enabled, the network connection between the shoot control plane in the seed and the shoot worker nodes will be established from shoot to seed instead of seed to shoot like earlier. Furthermore, in this case the additional "vpn-shoot" load balancer in the shoot cluster will no longer be required. Please note that the feature is in alpha state and might be promoted in future Gardener releases. (#3812, @DockToFuture)
  • This feature allows seed and shoot clusters to operate in different network domains. Only the shoot clusters need to be able to establish connections to the seed clusters. The other direction is not required.
  • ReversedVPN only works if APIServerSNI is enabled.
  • Apart from the feature gate, which enables/disables the feature per gardenlet for all managed shoot clusters, it is also possible to enable/disable the functionality on a per shoot basis. The shoot cluster annotation alpha.featuregates.shoot.gardener.cloud/reversed-vpn can be used for this purpose.
  • Please note that this feature is only compatible with Kubernetes >= 1.18. Clusters with older Kubernetes releases will continue to use the previous approach, i.e. the standard VPN-based tunnel.

🐛 Bug Fixes

• [OPERATOR] An issue has been fixed which could cause Projects not to be reconciled immediately when corresponding RoleBindings were changed. (#3985, @timebertt)
• [OPERATOR] The restoration flow for the Worker resource no longer enters a “rolling update” loop which was causing the restoration flow to take too much time. (#3970, @kris94)
• [OPERATOR] The ManagedSeed controller no longer watches secrets in the garden cluster. (#3939, @stoyanr)
• [OPERATOR] Migration and restoration of extensions.gardener.cloud.BackupEntry resources is now handled by the BackupEntry controller in the gardenlet. (#3880, @plkokanov)
• [OPERATOR] The core.gardener.cloud.BackupEntry resource is no longer reconciled multiple times in a row. (#3880, @plkokanov)
• [OPERATOR] Fixes a possible caching issue by directly returning an error when updating the Shoot.Status to reflect the start of a reconcile, restore or migrate operation, instead of retrying the update on conflict. (#3845, @plkokanov)

🏃 Others

• [USER] The following image is updated: (#3944, @ialidzhikov)
  • k8s.gcr.io/metrics-server/metrics-server: v0.4.2 -> v0.4.3 (see CHANGELOG)
• [USER] Grafana is upgraded to version 7.5.4 (#3891, @Kristian-ZH)
• [OPERATOR] istio-ingressgateway memory limit is increased to 2560Mi (#3984, @dguendisch)
• [OPERATOR] Error code detection has been improved and is now enabled for more steps in the shoot reconciliation, deletion and migration. (#3969, @timuthy)
• [OPERATOR] The kube-scheduler VPA does now specify minAllowed values to prevent too low resource recommendations from VPA that lead to OOM. (#3966, @ialidzhikov)
• [OPERATOR] gardener-resource-manager now uses the default leader election settings again (retries leader election every 2s). (#3964, @timebertt)
• [OPERATOR] New alert FluentBitIdleInputPlugins for idle fluent-bit pods. (#3943, @vlvasilev)
• [OPERATOR] Ensure gardener-resource-manager is present during hibernation. (#3926, @timebertt)
• [OPERATOR] It is now possible to specify the shoot purpose as infrastructure and to leave the machine image version empty in a ManagedSeedSet's shootTemplate. (#3924, @stoyanr)
• [OPERATOR] It is now possible to trigger an immediate reconciliation of a ManagedSeedSet by adding the annotation gardener.cloud/operation=reconcile. (#3922, @stoyanr)
• [OPERATOR] Gardener now finalizes all VolumeAttachments on hibernation to unblock hibernation of clusters with custom CSI drivers. (#3916, @timebertt)
• [OPERATOR] ManagedSeedSets can now be scaled via the scale command, e.g. kubectl scale mss/my-seeds --replicas 3 (#3911, @stoyanr)
• [OPERATOR] Add HVPA for all prometheus instances managed by seed-bootstrap (#3903, @wyb1)
• [OPERATOR] Loki is upgraded to version 2.2.1 and Fluent-bit to 1.7.3 (#3891, @Kristian-ZH)
• [DEPENDENCY] The Terraformer library now conducts the Pod's termination message for improved readability of error messages. (#3950, @timebertt)

📰 Noteworthy

• [USER] Shoot operations that error due to cloud provider rate limit exceeded errors are now classified with the new ERR_INFRA_REQUEST_THROTTLING error code. Previously these errors were classified as ERR_INFRA_QUOTA_EXCEEDED and they were no longer retried. There is now a new control loop in GCM that is responsible for retrying such failed Shoots due to rate limit exceeded errors. (#3925, @ialidzhikov)
• [DEVELOPER] When using the local garden development environment, the Gardener components do now use dedicated kubeconfigs constrained by RBAC rules (earlier, they were always using the admin kubeconfig). (#3901, @rfranzke)

[gardener-resource-manager]

⚠️ Breaking Changes

• [OPERATOR] The default leader election resource lock of gardener-resource-manager has been changed from configmapsleases to leases. (gardener/gardener-resource-manager#119, @timebertt)
  • Please make sure, that you had at least [email protected] running before upgrading to v0.24, so that it has successfully required leadership with the hybrid resource lock (configmapsleases) at least once.

[vpn2]

📰 Noteworthy

• [OPERATOR] This PR prevents any direct traffic from the shoot cluster towards the vpn-seed-server pod via the OpenVPN connection. Connectivity from the shoot cluster to the seed cluster via the VPN connection is not needed and blocked with this PR for security reasons. (gardener/vpn2#2, @marwinski)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.22.0
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.22.0
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.22.0
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.22.0
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.22.0
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.22.0
landscaper-gardenlet: eu.gcr.io/gardener-project/gardener/landscaper-gardenlet:v1.22.0

[gardener]

🏃 Others

• [OPERATOR] Increase limits for etcd to avoid potential long down time during maintenance window. (#3921, @timuthy)
• [OPERATOR] The update procedure of Gardener's Care Controller has been improved so that the Status sub-resource of a shoot always reflects the latest results of health and constraint checks. (#3920, @timuthy)
• [OPERATOR] Configure VPA minAllowed for dependency-watchdog. (#3919, @timuthy)
• [OPERATOR] A bug has been fixed which prevented shoot from being scheduled, hibernated or maintained when they hold references to external resources via shoot.spec.resources. (#3915, @timuthy)

[autoscaler]

✨ New Features

• [USER] Enable configuraiton of flags such as control-apiserver-burst, control-apiserver-qps, target-apiserver-burst, target-apiserver-qps and min-resync-period for kubernetes client configurations while fetching objects for MCM cloud provider. (gardener/autoscaler#73, @prashanth26)

🐛 Bug Fixes

• [OPERATOR] Switch to using cached informers to fetch cloud provider details more optimally. (gardener/autoscaler#73, @prashanth26)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.20.4
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.20.4
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.20.4
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.20.4
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.20.4
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.20.4
landscaper-gardenlet: eu.gcr.io/gardener-project/gardener/landscaper-gardenlet:v1.20.4

[gardener]

⚠️ Breaking Changes

• [OPERATOR] The temporary workaround in the ProblematicWebhooks check that was skipping Shoot webhooks is now removed. Before updating to this version of Gardener, please make sure that the provider extensions in the system vendor at least github.com/gardener/[email protected]. (#3867, @ialidzhikov)
• [OPERATOR] ⚠️ Gardener does no longer support shoot clusters with Kubernetes versions < 1.15. With this change, the .spec.kubernetes.kubeControllerManager.horizontalPodAutoscaler.{up,down}scaleDelay fields have been dropped because they are no longer meaningful. Make sure to upgrade all existing clusters before upgrading to this Gardener version. (#3862, @rfranzke)
• [OPERATOR] ⚠️ The minimum Kubernetes version for seed clusters has been raised from v1.11 to v1.15. Make sure that all your registered seed clusters meet this requirement before upgrading to this Gardener version. (#3862, @rfranzke)
• [OPERATOR] Invalid image vectors and component image vector overwrites will cause validation errors upon reading. If you encounter such errors, make sure image vectors specified in ConfigMap or ComponentRegistration resources are valid. (#3853, @stoyanr)
• [DEPENDENCY] ⚠️ The utility functions for working with ManagedResources have been mostly moved from pkg/operation/common to pkg/utils/managedresources. Please note that the signature of the functions might have changed. Especially, the order of the name, namespace string parameters is now namespace, name string. (#3780, @rfranzke)

✨ New Features

• [USER] New .status.advertisedAddresses field in the Shoot resource now provides a list of advertised URLs of the Kubernetes API Server. (#3883, @mvladev)
• [DEVELOPER] Gardener can now support shoot clusters with Kubernetes version 1.21. In order to allow creation/update of 1.21 clusters you will have to update the version of your provider extension(s) to a version that supports 1.21 as well. Please consult the respective releases and notes in the provider extension's repository. (#3860, @rfranzke)

🐛 Bug Fixes

• [USER] An issue has been fixed which prevented DNS entries being created correctly. Only requests coming from shoot clusters were affected. (#3863, @MartinWeindel)
• [USER] Several regressions related to the AuditPolicy validation are fixed. (#3855, @timebertt)
• [OPERATOR] Gardener care operations now only consider conditions of relevant BackupEntries. Earlier, the controller retrieved all entries instead of only checking the one that is associated to the processed shoot. (#3854, @timuthy)
• [OPERATOR] An issue has been fixed which led to Shoots not being reconciled immediately after changing the referenced AuditPolicy ConfigMap. (#3848, @timebertt)
• [DEVELOPER] A bug that prevented gardenlet to start-up when there is no seed in the garden cluster is now fixed. (#3840, @vpnachev)

📖 Documentation

• [DEVELOPER] This pull request contains the GEP for an updated cluster VPN implementation. (#3771, @marwinski)

🏃 Others

• [USER] The following image is updated: (#3825, @ialidzhikov)
  • k8s.gcr.io/node-problem-detector/node-problem-detector: v0.8.5 -> v0.8.7
• [OPERATOR] A bug has been fixed which prevented shoot from being scheduled, hibernated or maintained when they hold references to external resources via shoot.spec.resources. (#3906, @timuthy)
• [OPERATOR] Increase limits for etcd to avoid potential long down time during maintenance window. (#3876, @amshuman-kr)
• [OPERATOR] VPA minAllowed configuration for node-exporter. (#3868, @amshuman-kr)
• [OPERATOR] The update procedure of Gardener's Care Controller has been improved so that the Status sub-resource of a shoot always reflects the latest results of health and constraint checks. (#3861, @timuthy)
• [OPERATOR] Grafana is now deleted separately from the monitoring stack during the deletion flow. This prevents phantom alerts from being fired during shoot deletion. (#3852, @wyb1)
• [OPERATOR] It is now possible to trigger an immediate reconciliation of a ManagedSeed (and therefore a rollout of its gardenlet) by adding the annotation gardener.cloud/operation=reconcile. (#3847, @stoyanr)
• [OPERATOR] Fix a bug which could block seed deletion due to an already deleted etcd crd (#3832, @BeckerMax)
• [OPERATOR] Configure VPA minAllowed for dependency-watchdog. (#3831, @amshuman-kr)

[autoscaler]

📰 Noteworthy

• [USER] Enable configuraiton of flags such as control-apiserver-burst, control-apiserver-qps, target-apiserver-burst, target-apiserver-qps and min-resync-period for kubernetes client configurations while fetching objects for MCM cloud provider. (gardener/autoscaler#73, @prashanth26)
• [OPERATOR] Switch to using cached informers to fetch cloud provider details more optimally. (gardener/autoscaler#73, @prashanth26)

[gardener-resource-manager]

✨ New Features

• [OPERATOR] It is now possible to specify the leader election resource lock via the command line flag --leader-election-resource-lock (defaults to configmapsleases) and the chart value leaderElection.resourceLock. Please be careful when changing the resource lock and always migrate via multilocks in order to prevent situations where multiple instances of the controller are running with leader election and thus acting on the same resources. (gardener/gardener-resource-manager#117, @timebertt)

🏃 Others

• [DEVELOPER] gardener-resource-manager now supports a Ignore mode for resources managed by a ManagedResource. The primary use case for this mode is a migration of resource from one ManagedResource to another one. (gardener/gardener-resource-manager#118, @ialidzhikov)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.21.0
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.21.0
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.21.0
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.21.0
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.21.0
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.21.0
landscaper-gardenlet: eu.gcr.io/gardener-project/gardener/landscaper-gardenlet:v1.21.0