Kubernetes-native system managing the full lifecycle of conformant Kubernetes clusters as a service on Alicloud, AWS, Azure, GCP, OpenStack, vSphere, KubeVirt, Hetzner, EquinixMetal, MetalStack, and OnMetal with minimal TCO.
OTHER License
Published by gardener-robot-ci-3 about 5 years ago
kubernetes-dashboard
addon is token
instead of basic
. You can still enable it manually, but you should consider migrating away from it. We will drop basic authentication support in a future release when Kubernetes doesn't support it anymore. (#1443, @rfranzke)g-users
to gu
. (#1417, @wyb1)0.13.0
of the provider extension controllers. (#1443, @rfranzke)garden.sapcloud.io/v1beta1.BackupInfrastructure
resources are deprecated and will be removed in the next release. The core.gardener.cloud/v1alpha1.BackupBucket
and core.gardener.cloud/v1alpha1
are the replacement. (#1427, @swapnilgm)g-operators
to go
. (#1417, @wyb1)KubeEtcdFullBackupFailed
alert has been fixed. This requires at least version 0.12.0 of the gardener-extensions
(containing 0.7.3 of etcd-backup-restore
). (#1381, @wyb1)1.4.0
to 1.6.3
. (#1443, @rfranzke).spec.backup
section in the garden.sapcloud.io/v1beta1.Shoot
resource has been removed. (#1430, @rfranzke)Shoot
resource which is part of the core.gardener.cloud/v1alpha1
API group. It is fully forwards and backwards compatible to the old Shoot
resource in the garden.sapcloud.io/v1beta1
API group. It will be the new default, and the old garden.sapcloud.io/v1beta1.Shoot
resource are deprecated now. It will be removed in a future version. Consider switching to using the new core.gardener.cloud/v1alpha1.Shoot
resource. The example
directory contains proper example manifests. (#1430, @rfranzke)CloudProfile
resource which is part of the core.gardener.cloud/v1alpha1
API group. It is fully forwards and backwards compatible to the old CloudProfile
resource in the garden.sapcloud.io/v1beta1
API group. It will be the new default, and the old garden.sapcloud.io/v1beta1.CloudProfile
resource are deprecated now. It will be removed in a future version. Consider switching to using the new core.gardener.cloud/v1alpha1.CloudProfile
resource. The example
directory contains proper example manifests. (#1403, @rfranzke)OpenIDConnectPreset
resource allows for specifying OpenID Connect configurations which are applied to Shoot
namespace-wide. (#1394, @mvladev)Shoot
specification has been added spec.kubernetes.kubeAPIServer.oidcConfig.clientAuthentication
. It can specify OpenID Connect settings used for kubeconfig
generation. (#1394, @mvladev).spec.dns.includeDomains
, .spec.dns.excludeDomains
. (#1430, @rfranzke)BackupInfrastructure
resources post reconciliation of shoots using Gardener 0.29+ are now getting deleted. (#1427, @swapnilgm)
ClusterOpenIDConnectPreset
resource allows for specifying OpenID Connect configurations which are applied to Projects
and Shoot
cluster-wide. (#1394, @mvladev)ClusterOpenIDConnectPreset
and OpenIDConnectPreset
. Those controllers can are enabled by default and can be disabled with --disable-admission-plugins
flag on gardener-apiserver. (#1394, @mvladev)gardener-controller-manager
's componentconfig. (ddfdc74622a37893040a83718c361615790af37e)gardener-scheduler
is now working only with core.gardener.cloud/v1alpha1
instead of garden.sapcloud.io/v1beta1
resources. (#1435, @rfranzke)garden.sapcloud.io.CIDR
and garden.sapcloud.io/v1beta1.CIDR
type has been replaced with string
type. (#1430, @rfranzke).spec.kubernetes.kubeAPIServer.admissionPlugins[].config
type has been changed from *string
to *ProviderConfig
(which effectively is a *runtime.RawExtension
). (#1430, @rfranzke)./hack/dev-setup-register-gardener
must be ran to register the new settings.gardener.cloud
API group. (#1394, @mvladev)Alertmanager
if they configured their shoot to receive alerts. (#1417, @wyb1)gardener-apiserver
, gardener-controller-manager
and gardener-scheduler
. To enable VPA for each component global.apiserver.vpa
, global.controller.vpa
and global.scheduler.vpa
must be set to true respectively. (#1440, @wyb1)spec.maintenance.autoUpdate.kubernetesVersion: false
in the Shoot when the Kubernetes version does not exist in the CloudProfile. (#1423, @danielfoehrKn)2.12.0
(#1410, @wyb1)garden/seed-monitoring-ingress-credentials
. (#1405, @wyb1)gardener-apiserver
now also react for resources migrated to the new core.gardener.cloud/v1alpha1
API. (9b73a905e3136d840a1ee6ffe5c47a0494d118a1)Seed
object for shooted seeds does no longer have an owner reference to the respective Shoot
because Seed
is cluster-scoped while Shoot
is namespaced, and according to https://kubernetes.io/docs/concepts/workloads/controllers/garbage-collection/ this is not supported. (0dc122cc110759b255fa3f59ec7e8c7a345bed8d)Kinds
belonging to a recently deployed CRD. (gardener/gardener-resource-manager#14, @timuthy)ResourcesApplied
and ResourcesHealthy
. (gardener/gardener-resource-manager#11, @ialidzhikov)target
cluster even though they were supposed to be deleted through a change or removal of the ManagedResource
. (gardener/gardener-resource-manager#8, @timuthy)1.60.0
-> 2.26.0
1.20.0
-> 2.14.0
1.22.1
-> 1.33.1
1.16.0
-> 1.21.1
1.31.0
-> 1.55.2
1.7.2
-> 2.3.0
1.0.0
-> 2.1.2
1.0.0
-> 2.1.2
tzdata
package is now used instead of assets/zoneinfo.zip
to make all timezones available. (gardener/terraformer#24, @ialidzhikov)gardener-apiserver: eu.gcr.io/gardener-project/gardener/apiserver:0.30.0
gardener-controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:0.30.0
gardener-scheduler: eu.gcr.io/gardener-project/gardener/scheduler:0.30.0
Published by gardener-robot-ci-1 about 5 years ago
KubeEtcdFullBackupFailed
alert has been fixed. This requires at least version 0.12.0 of the gardener-extensions
(containing 0.7.3 of etcd-backup-restore
). (9f1bb8d510ca6c40680d1b0406583a63004597af)gardener-controller-manager
's componentconfig. (b7b3a14bbbcf799d7af6665c8f569273bdeb8999)gardener-apiserver
now also react for resources migrated to the new core.gardener.cloud/v1alpha1
API. (31d150efca5441c29fb4b5424619fa1c5f05f9f7)gardener-apiserver: eu.gcr.io/gardener-project/gardener/apiserver:0.29.2
gardener-controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:0.29.2
gardener-scheduler: eu.gcr.io/gardener-project/gardener/scheduler:0.29.2
Published by gardener-robot-ci-1 about 5 years ago
Seed
object for shooted seeds does no longer have an owner reference to the respective Shoot
because Seed
is cluster-scoped while Shoot
is namespaced, and according to https://kubernetes.io/docs/concepts/workloads/controllers/garbage-collection/ this is not supported. (3fc63b22ecb04d35fcff6babc560ad6f4e52ba20)gardener-apiserver: eu.gcr.io/gardener-project/gardener/apiserver:0.29.1
gardener-controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:0.29.1
gardener-scheduler: eu.gcr.io/gardener-project/gardener/scheduler:0.29.1
Published by gardener-robot-ci-1 about 5 years ago
spec.cloud.openstack.networks.workers[]
. (#1370, @mvladev)
Shoot
s which are failing by removing the additional CIDRs (if they exist). Please update your Shoot
manifests for future compatibility.ControllerRegistration
resources to make these extension controllers known to Gardener (otherwise none of your shoots will be able to get reconciled anymore). After updating Gardener you need to update your existing ControllerRegistration
resources to make these extension controllers known to Gardener (otherwise none of your shoots will be able to get reconciled anymore). You should already have ControllerRegistration
resources for various providers. Add .spec.resources[].kind="BackupBucket"
and .spec.resources[].type="<provider-name>"
, and also .spec.resources[].kind="BackupEntry"
and .spec.resources[].type="<provider-name>"
to it. Please find example ControllerRegistration
resources here (Alicloud), here (AWS), here (Azure), here (GCP), and here (OpenStack). To get information about more details please walk through these documents. (#1128, @swapnilgm)
Seed
resources have to be updated with backup configuration, i.e. you have to specify spec.backup
with backup provider details before updating gardener-controller-manager
version to this release.gardener-controller-manager
deployment: kubectl -n garden scale deployment/gardener-controller-manager --replicas=0
Seed
s to contain the correct backup configuration.ControllerRegistration
resources for the provider extensions you want to use.gardener-controller-manager
deployment: kubectl -n garden scale deployment/gardener-controller-manager --replicas=1
authURL
with the keystone URL. This is due to https://github.com/gardener/gardener/pull/1128. (1a91fed670be1969f2281b268bfa214ffa3885ea)Project
resource which is part of the core.gardener.cloud/v1alpha1
API group. It is fully forwards and backwards compatible to the old Project
resource in the garden.sapcloud.io/v1beta1
API group. It will be the new default, and the old garden.sapcloud.io/v1beta1.Project
resource are deprecated now. It will be removed in a future version. Consider switching to using the new core.gardener.cloud/v1alpha1.Project
resource. The example
directory contains proper example manifests. (#1382, @rfranzke)SecretBinding
and Quota
resources which are part of the core.gardener.cloud/v1alpha1
API group. They are fully forwards and backwards compatible to the old SecretBinding
and Quota
resources in the garden.sapcloud.io/v1beta1
API group. They will be the new defaults, and the old garden.sapcloud.io/v1beta1.SecretBinding
and garden.sapcloud.io/v1beta1.Quota
resources are deprecated now. They will be removed in a future version. Consider switching to using the new core.gardener.cloud/v1alpha1.SecretBinding
and core.gardener.cloud/v1alpha1.Quota
resources. The example
directory contains proper example manifests. (#1377, @rfranzke).spec.dns.hostedZoneID
has been removed from the Shoot
API. (#1372, @rfranzke)CloudProfile
in the .spec.<cloud>.constraints.kubernetes[].offeredVersions[].expirationDate
fields. Example: The CloudProfile
specifies (1.14.2, 2019-08-08T14:00:00Z)
and (1.14.5)
as possible versions for 1.14
. A shoot that is opted-out of automatic updates uses (1.14.2)
. In its maintenance time window it won't be updated to 1.14.5
before 2019-08-08T14:00:00Z
. (#1363, @danielfoehrKn)Seed
resource which is part of the core.gardener.cloud/v1alpha1
API group. It is fully forwards and backwards compatible to the old Seed
resource in the garden.sapcloud.io/v1beta1
API group. It will be the new default, and the old garden.sapcloud.io/v1beta1.Seed
resource is deprecated now. It will be removed in a future version. Consider switching to using the new core.gardener.cloud/v1alpha1.Seed
resource. The example
directory contains proper example manifests. (#1308, @rfranzke)spec.networks.shootDefaults
on your Seed
resources if you want to use this. (#1386, @rfranzke)Extension
resources are now only deleted after the shoot Kubernetes resources have been cleaned up during the shoot deletion flow. (#1374, @rfranzke)dns.gardener.cloud/include-zones
and dns.gardener.cloud/exclude-zones
annotations on the secrets (see example resource). If you are using a custom domain for your shoot then you can use .spec.dns.includeZones
, .spec.dns.excludeZones
. (#1372, @rfranzke)spec.cloud.gcp.networks.workers[]
could be created (which is not a valid behavior). To ensure that those clusters are going to be safely reconciled with this update, on the next Shoot
UPDATE
request only the first spec.cloud.openstack.networks.workers[0]
is going to be persisted. (#1370, @mvladev)
.spec.<cloud>.constraints.kubernetes[].versions
field in the CloudProfile
is deprecated and will be removed in the future. It is now replaced with a list of offered versions (.spec.<cloud>.constraints.kubernetes[].offeredVersions
). Each version can have an expirationDate
which specifies the time after all shoots that are using this version and opted-out of automatic Kubernetes patch version updates will get forcefully updated to the latest Kubernetes patch version for the used <major>.<minor>
version. Example: The CloudProfile
specifies (1.14.2, 2019-08-08T14:00:00Z)
and (1.14.5)
as possible versions for 1.14
. A shoot that is opted-out of automatic updates uses (1.14.2)
. In its maintenance time window it won't be updated to 1.14.5
before 2019-08-08T14:00:00Z
. (#1363, @danielfoehrKn)BackupBucket
and BackupEntry
in its core.gardener.cloud/v1alpha1
API group. The backup infrastructure for new and existing shoots will be provisioned as per GEP-2. (#1128, @swapnilgm)Seed
manifest to see how this can be configured. (#1128, @swapnilgm)flow
package now has a LimitSubmitter
which can be used to restrict the amount of operations being executed in parallel. (#1391, @timuthy)ConfigMap
s labelled with extensions.gardener.cloud/configuration=logging
and injects their data into the fluent-bit
configuration. This allows extension controllers to define their provider-specific logging configuration for the components they deploy. (#1371, @svetlinas)Shoot
s that were created with a wrong .spec.dns.domain
field. (#1378, @rfranzke)Shoot
's .status
fields). (#1364, @rfranzke)gardener-controller-manager
are now being propagated to every Kubernetes client used by Gardener. This improves the overall performance of the system, especially for ControllerInstallation
, Seed
, and Shoot
reconciliations. (#1379, @timuthy)DNSProvider
objects for the external cluster domain(s) with dns.gardener.cloud/realms="<shoot-namespace>,"
. (#1368, @rfranzke)DNSProvider
s for a shoot do now only include the shoot's base domain (without api.
prefix). (#1368, @rfranzke)shoot.garden.sapcloud.io/use-as-seed
annotation. Please set backup.provider
, backup.region
, backup.secretRef.name
, backup.SecretRef.namespace
if you want to configure it. If no such configuration is found the default behaviour is that the same provider will be used for backup. In order to explicitly disable backup for a seed configure backup.provider=none
in the annotation. (1a91fed670be1969f2281b268bfa214ffa3885ea)shoot.garden.sapcloud.io/use-as-seed
annotation. Please set shootDefaults.pods=<cidr>
and shootDefaults.services=<cidr>
if you want to configure it. (018214b04bdb2336089515ff4a80d25daeb28268)gardener-apiserver: eu.gcr.io/gardener-project/gardener/apiserver:0.29.0
gardener-controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:0.29.0
gardener-scheduler: eu.gcr.io/gardener-project/gardener/scheduler:0.29.0
Published by gardener-robot-ci-1 about 5 years ago
spec.cloud.gcp.networks.workers[]
. (#1346, @mvladev)
Shoot
s which are failing by removing the additional CIDRs (if they exist). Please update your Shoot
manifests for future compatibility.shoot.garden.sapcloud.io/operation=rotate-kubeconfig-credentials
this token will be rotated. (#1339, @rfranzke)ControllerRegistration
resources to make these extension controllers known to Gardener (otherwise none of your shoots will be able to get reconciled anymore). Please find example ControllerRegistration
resources here (Alicloud), here (AWS), here (Azure), here (GCP), here (OpenStack), and here (Packet). To get information about more details please walk through these and these documents. (#1313, @svetlinas)
ControllerRegistration
resource to make this extension controller known to Gardener (otherwise none of your shoots will be able to get reconciled anymore). Please find example ControllerRegistration
resources here (Calico). To get information about more details please walk through these documents. (#1293, @zanetworker)Shoot
resources by only specifying the <major>.<minor>
parts of the desired Kubernetes version. Gardener will automatically try to pick the latest patch version offered by the referenced CloudProfile
. (#1350, @danielfoehrKn).spec.kubernetes.kubeAPIServer.enableBasicAuthentication=false
. By default, basic authentication is still enabled for all shoots. (#1341, @rfranzke)Shoot
resource with shoot.garden.sapcloud.io/operation=rotate-kubeconfig-credentials
. ⚠️ Please be aware that if this cluster was created before Gardener version 0.28.0
then your new kubeconfig will no longer contain a client certificate (however, it the previously issued one will remain valid). (#1339, @rfranzke)CustomResourceDefinition
s and APIService
s now have 1h
time to cleanup before getting forcefully finalized. (#1326, @adracus)kubelet
settings for shoots in .spec.kubernetes.kubelet
per worker pool in .spec.cloud.<name>.workers[].kubelet
. If no worker-specific kubelet settings are provided then the defaults in .spec.kubernetes.kubelet
apply. (#1299, @danielfoehrKn)110
(like in Kubernetes), and you can override it based on your node mask CIDR. Set .spec.kubernetes.kubelet.maxPods
and .spec.kubernetes.kubeControllerManager.nodeMaskCIDR
. Please be aware that the node mask CIDR is immutable, so you cannot increase or shrink it without recreating your shoot. The maximum number of pods can be changed as long as it is in the boundaries of the node mask CIDR. (#1299, @danielfoehrKn).spec.kubernetes.kubeControllerManager.nodeCIDRMaskSize
flag in the Shoot
resource. (#1280, @afritzler)spec.cloud.gcp.networks.workers[]
could be created (which is not a valid behavior). To ensure that those clusters are going to be safely reconciled with this update, on the next Shoot
UPDATE
request only the first spec.cloud.gcp.networks.workers[0]
is going to be persisted. (#1346, @mvladev)
gardener-resource-manager
deployment in its garden
namespace. This instance manages ManagedResources
with .spec.class=seed
for which contained manifests are applied to or removed from the very same seed cluster. Extension controllers might be interested in using this feature to let the gardener-resource-manager
handle resources that need to be deployed to the seed. (#1331, @timuthy)1.12.8
(fixing CVE-2019-9512
and CVE-2019-9514
). (#1328, @mvladev)ControlPlane
extension CRD does now have an optional .spec.purpose
field. The default purpose will be normal
. Another purpose is exposure
which can be used to trigger the deployment of components that are required for exposing a shoot control plane. As the shoot control plane is running in the seed cluster this is specific to the seed provider. (#1303, @svetlinas)ConfigMap
that contains an audit policy which is referenced by a Shoot
is changed then the Shoot
gets reconciled immediately again (even if the Shoot
is not within its maintenance time window (in case reconciliation should only happen in the maintenance time window)). (#1320, @rfranzke)spec.kubernetes.kubeAPIServer.serviceAccountConfig
and spec.kubernetes.kubeAPIServer.apiAudiences
as described in https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/#service-account-token-volume-projection. (#1288, @adracus)Extensions
that are removed from the Shoot
or through the ControllerRegistration
are now correctly deleted. (#1342, @timuthy)ControllerRegistration
for Extension
resources is deleted then the ControllerInstallation
controller now only waits for the relevant Extension
objects to be deleted. Previously it was waiting for all Extension
objects which resulted in an end-less wait loop. (#1329, @rfranzke)Secret
that is synced to the project namespace in the garden cluster does now have a owner reference to the Shoot
(allows proper garbage collection after shoot deletion). (#1319, @rfranzke)nginx-ingress
controller on shooted seeds have been increased to 1
CPU, 2Gi
memory. (#1318, @rfranzke)prometheus
stateful set does now have the requirement environment variables for the latest VPN version. (#1307, @zanetworker)ControlPlane
extension CRD is now deployed before the sleeping components are woken up. This is to ensure that possible provider specific configuration exists before the wake-up. (#1305, @rfranzke)ControlPlane
CRD if the kube-apiserver
deployment does still exist. The reason for this is that the kube-apiserver
depends on some resources that are managed via the ControlPlane
CRD for some providers. (#1304, @rfranzke)ControlPlane
and Worker
extension resource only if they still exist. (#1301, @rfranzke)kube-apiserver
deployment does now have the requirement environment variables for the latest VPN version. (#1287, @DockToFuture)garden
namespace can now be monitored using the seed-prometheus
. (#1179, @wyb1)seed-prometheus
. (#1179, @wyb1)fluent-bit
, fluentd
, and elasticsearch
in the garden
namespace. (#1006, @KristianZH)cloud-config-downloader
running on every shoot worker nodes is now enabling all the systemd units before starting them. (#1334, @vlvasilev).spec
field. (#1292, @timuthy)ManagedResource
should be kept in the system although the ManagedResource
is deleted. The .spec.keepObjects
field defaults to false
. (gardener/gardener-resource-manager@6b2029430332753e158c1611b8cf4b1dcd68f0c0).spec.forceOverwriteLabels=true
or .spec.forceOverwriteAnnotations=true
. (gardener/gardener-resource-manager@3eff3a859192584a07ded5f174b403de4aeea1db)ManagedResource
is specified in .spec.class
. (gardener/gardener-resource-manager#5, @mandelsoft)dep
is replaced by go mod
(gardener/gardener-resource-manager#4, @ialidzhikov)vpn-seed
container now allows authentication against the kube-apiserver with a client certificate. The new environment variables APISERVER_AUTH_MODE
can be either basic-auth
or client-cert
. (gardener/vpn@02eb33cad28ad214d1631219021569a5126d14c0)
basic-auth
the APISERVER_AUTH_MODE_BASIC_AUTH_CSV
environment variable tells the path to the basic auth CSV file, and APISERVER_AUTH_MODE_BASIC_AUTH_USERNAME
tells the user name (only the password will be read out of the CSV).client-cert
the APISERVER_AUTH_MODE_CLIENT_CERT_CA
, APISERVER_AUTH_MODE_CLIENT_CERT_CRT
, APISERVER_AUTH_MODE_CLIENT_CERT_KEY
variables tell the paths to the CA, client cert, and client key.gardener-apiserver: eu.gcr.io/gardener-project/gardener/apiserver:0.28.0
gardener-controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:0.28.0
gardener-scheduler: eu.gcr.io/gardener-project/gardener/scheduler:0.28.0
Published by gardener-robot-ci-1 about 5 years ago
CustomResourceDefinition
s and APIService
s now have 1h
time to cleanup before getting forcefully finalized. (c50eb1fb8c8cd41453e5651da5fef83d2cbf109d)1.12.8
(fixing CVE-2019-9512
and CVE-2019-9514
). (a9904b5d84f999612913c4f7eb5bba0e4da8bf64)ControllerRegistration
for Extension
resources is deleted then the ControllerInstallation
controller now only waits for the relevant Extension
objects to be deleted. Previously it was waiting for all Extension
objects which resulted in an end-less wait loop. (a0d815430fc219c3f4649d9eaace6a09ed25ec94)gardener-apiserver: eu.gcr.io/gardener-project/gardener/apiserver:0.27.6
gardener-controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:0.27.6
gardener-scheduler: eu.gcr.io/gardener-project/gardener/scheduler:0.27.6
Published by gardener-robot-ci-1 about 5 years ago
ConfigMap
that contains an audit policy which is referenced by a Shoot
is changed then the Shoot
gets reconciled immediately again (even if the Shoot
is not within its maintenance time window (in case reconciliation should only happen in the maintenance time window)). (2657b7c1283c6ac94c395ee2a0f08bae4d0eabbd)nginx-ingress
controller on shooted seeds have been increased to 1
CPU, 2Gi
memory. (8cdcf4469c4a79be616b0edc025bd81f63c12fc7)Secret
that is synced to the project namespace in the garden cluster does now have a owner reference to the Shoot
(allows proper garbage collection after shoot deletion). (35260e440a3931eee4af5d4b5afa935348bf97ee)gardener-apiserver: eu.gcr.io/gardener-project/gardener/apiserver:0.27.5
gardener-controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:0.27.5
gardener-scheduler: eu.gcr.io/gardener-project/gardener/scheduler:0.27.5
Published by gardener-robot-ci-1 about 5 years ago
prometheus
stateful set does now have the requirement environment variables for the latest VPN version. (c04d33ffc7e3eb2f483fec14e8eabc5a19ea57d4)ControlPlane
extension CRD is now deployed before the sleeping components are woken up. This is to ensure that possible provider specific configuration exists before the wake-up. (8048af0a06d10163e61f41bb9d958810a1d656d3)kube-apiserver
deployment does now have the requirement environment variables for the latest VPN version. (8967d71271fa8399de5333a502a54a09ab6500ed)gardener-apiserver: eu.gcr.io/gardener-project/gardener/apiserver:0.27.4
gardener-controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:0.27.4
gardener-scheduler: eu.gcr.io/gardener-project/gardener/scheduler:0.27.4
Published by gardener-robot-ci-1 about 5 years ago
ControlPlane
CRD if the kube-apiserver
deployment does still exist. The reason for this is that the kube-apiserver
depends on some resources that are managed via the ControlPlane
CRD for some providers. (f4a28f592fbd55a79dc9f7820ecbbbeeb1b4ee2e)gardener-apiserver: eu.gcr.io/gardener-project/gardener/apiserver:0.27.3
gardener-controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:0.27.3
gardener-scheduler: eu.gcr.io/gardener-project/gardener/scheduler:0.27.3
Published by gardener-robot-ci-1 about 5 years ago
ControlPlane
and Worker
extension resource only if they still exist. (8bc6083554a518e4724a6d7923eaf395593415c8).spec
field. (a883e63cf345b9d1e0480996dfe06e4304c7a50a)gardener-apiserver: eu.gcr.io/gardener-project/gardener/apiserver:0.27.2
gardener-controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:0.27.2
gardener-scheduler: eu.gcr.io/gardener-project/gardener/scheduler:0.27.2
Published by gardener-robot-ci-1 about 5 years ago
gardener-apiserver: eu.gcr.io/gardener-project/gardener/apiserver:0.27.1
gardener-controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:0.27.1
gardener-scheduler: eu.gcr.io/gardener-project/gardener/scheduler:0.27.1
Published by gardener-robot-ci-1 about 5 years ago
VPA
feature gate has been entirely removed. VPA
components will now always be deployed. You have to remove the VPA
feature gate from your gardener-controller-manager configuration after upgrading Gardener. (#1173, @wyb1)ControlPlane
, Extension
, Infrastructure
, OperatingSystemConfig
and Worker
extension resources with gardener.cloud/operation=reconcile
. Extension controllers must only react/reconcile their resources if this annotation is set. After they have picked up the event they should update the status to Reconcile Processing
and remove the annotation. This only applies for extension resources that have to be reconciled. Those that are newly created or marked for deletion shall be operated on independent of the annotation set by Gardener. Also, you might want to take a look at gardener/gardener-extensions#178 or this document. (#1165, @timuthy)pkg/client/kubernetes
are now removed. Consider to switch your invocations to kubernetes-sigs/controller-runtime client (.Client()
) or kubernetes/client-go (.Kubernetes()
). (#1140, @ialidzhikov).spec.hibernated.enabled
field in the Shoot
resource is now optional. (#1277, @danielfoehrKn)machineImage
section in the Shoot
resource does now have a providerConfig
field that can be filled with operating system specific configuration. Please consult the documentation of the respective OS-controller you are using. (#1261, @pablochacin)Shoot
. It is now possible to configure different machine images per worker pool. The .spec.cloud.<provider-name>.machineImage
section contains the default image that should be used for worker pools which don't explicitly configure a machine image (which is possible via the .spec.cloud.<provider-name>.workers[*].machineImage
field). (#1250, @KristianZH)CloudProfile
in the .spec.<cloud>.constraints.machineImages[].versions.expirationDate
fields. If you want to opt-out set .spec.maintenance.machineImageVersion=false
in your Shoot
resource (default: true
). Example: The CloudProfile
specifies (coreos, 1967.5.0, 2019-08-08T14:00:00Z)
and (coreos, 2023.4.0)
as possible versions for coreos
. A shoot that is opted-out of automatic updates uses (coreos, 1967.5.0)
. In its maintenance time window it won't be updated to 2023.4.0
before 2019-08-08T14:00:00Z
. (#1177, @danielfoehrKn)gardener-apiserver
, gardener-controller-manager
, and gardener-scheduler
replicas are now deployed with anti-affinity such that they run on different worker nodes. Additionally, for each of these deployments, a pod disruption budget configuration will be created if replicas > 1
. It will allow max. replicas-1
unavailable pods. (#1231, @rfranzke)deletionGracePeriodHoursByPurpose
is now introduced for the backupInfrastructure controller to specify different deletion grace period values per shoot purpose. (#1204, @shreyas-s-rao).spec.<cloud>.constraints.machineImages[].version
field in the CloudProfile
is deprecated and will be removed in the future. It is now replaced with a list of versions (.spec.<cloud>.constraints.machineImages[].versions
). Each version can have an expirationDate
which specifies the time after all shoots that are using this version and opted-out of automatic machine image version updates will get forcefully updated to the latest machine image version for the used machine image name. Example: The CloudProfile
specifies (coreos, 1967.5.0, 2019-08-08T14:00:00Z)
and (coreos, 2023.4.0)
as possible versions for coreos
. A shoot that is opted-out of automatic updates uses (coreos, 1967.5.0)
. In its maintenance time window it won't be updated to 2023.4.0
before 2019-08-08T14:00:00Z
. (#1177, @danielfoehrKn)ControlPlane
, Extension
, Infrastructure
, OperatingSystemConfig
and Worker
resources by annotating them with gardener.cloud/operation=reconcile
. (#1165, @timuthy)v1.13
and higher are now encrypted before stored in etcd. It is possible to force Gardener to not encrypt secrets by annotating the etcd encryption secret in the shoot namespace in the seed with shoot.gardener.cloud/etcd-encryption-force-plaintext-secrets=true
. (#1066, @michael-engler)dep
is replaced by go mod
. github.com/gardener/gardener
now publishes go module files containing dependency version information. (#1185, @ialidzhikov)Shoot
status does now contain a new .status.hibernated
field which indicates whether the shoot is hibernated or not. The .spec.hibernated.enabled
field just indicates the desired state of the user. (#1277, @danielfoehrKn)kube-apiserver
during maintenance activities such as a rolling update or scaling (say, by HPA) of the kube-apiserver
Deployment
. (#1275, @amshuman-kr)Seed
object and seed secret. (#1242, @rfranzke)gardener-resource-manager
component is now always deployed (even if the shoot is hibernated (with replicas=0
)). This allows waking it up again when a hibernated shoot shall be deleted. (#1239, @rfranzke)garden.sapcloud.io:admin
cluster role now includes the dashboard.gardener.cloud
API Group (#1197, @petersutter)ControllerInstallation
controller does now inject the volume provider name into the Helm charts for the respective seed if it was annotated with persistentvolume.garden.sapcloud.io/provider=<name>
. The path of the value is .gardener.seed.volumeProvider
. (#1162, @jia-jerry)NetworkPolicy
migrations are now removed. (#1144, @mvladev)gardener-apiserver: eu.gcr.io/gardener-project/gardener/apiserver:0.27.0
gardener-controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:0.27.0
gardener-scheduler: eu.gcr.io/gardener-project/gardener/scheduler:0.27.0
Published by gardener-robot-ci-1 over 5 years ago
gardener-apiserver: eu.gcr.io/gardener-project/gardener/apiserver:0.26.4
gardener-controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:0.26.4
gardener-scheduler: eu.gcr.io/gardener-project/gardener/scheduler:0.26.4
Published by gardener-robot-ci-1 over 5 years ago
deletionGracePeriodDays
for the backupInfrastructure controller under controller manager config is now modified deletionGracePeriodHours
to provide more fine-grained control over backup infrastructure retention period post shoot deletion. ⚠️ Please update your component-config.yaml to reflect the same. (#1204, @shreyas-s-rao)deletionGracePeriodHoursByPurpose
is now introduced for the backupInfrastructure controller to specify different deletion grace period values per shoot purpose. (#1204, @shreyas-s-rao)gardener-apiserver: eu.gcr.io/gardener-project/gardener/apiserver:0.26.3
gardener-controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:0.26.3
gardener-scheduler: eu.gcr.io/gardener-project/gardener/scheduler:0.26.3
Published by gardener-robot-ci-1 over 5 years ago
ControllerInstallation
controller does now inject the volume provider name into the Helm charts for the respective seed if it was annotated with persistentvolume.garden.sapcloud.io/provider=<name>
. The path of the value is .gardener.seed.volumeProvider
. (#1163, @jia-jerry)gardener-apiserver: eu.gcr.io/gardener-project/gardener/apiserver:0.26.2
gardener-controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:0.26.2
gardener-scheduler: eu.gcr.io/gardener-project/gardener/scheduler:0.26.2
Published by gardener-robot-ci-1 over 5 years ago
gardener-apiserver: eu.gcr.io/gardener-project/gardener/apiserver:0.26.1
gardener-controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:0.26.1
gardener-scheduler: eu.gcr.io/gardener-project/gardener/scheduler:0.26.1
Published by gardener-robot-ci-1 over 5 years ago
0.25.0
.0.7.1
of the provider extension controllers. (#1127, @rfranzke)ControllerRegistration
resources to make these extension controllers known to Gardener (otherwise none of your shoots will be able to get reconciled anymore). You should already have ControllerRegistration
resources for various providers. Add .spec.resources[].kind="ControlPlane"
and .spec.resources[].type="<provider-name>"
to it. Please find example ControllerRegistration
resources here (Alicloud), here (AWS), here (Azure), here (GCP), here (OpenStack), and here (Packet). It is recommended to use more than one replica as the control plane extensions use webhooks to inject configuration into the standard control plane, and you want the webhook to be highly available. To get information about more details please walk through these documents. (#1076, @stoyanr)shoot.gardener.cloud/no-cleanup="true"
. This might be useful for extension controllers that are deploying resources into the shoot. (#1114, @rfranzke).controllers.shoot.reconcileInMaintenanceOnly=true
in the controller-manager's component configuration. If the shoot's spec
changes outside of their maintenance time windows then they will be reconciled immediately. Also, they will be reconciled immediately if they didn't complete their last operation successfully. With this in place users are now able to have a HA setup of shoots if their maintenance time windows do not overlap. (#1094, @adracus)etcd
s, kube-apiserver
s, cloud-controller-manager
s, and csi-{*}
controllers of existing shoots will be restarted. The reason for this is that the extension controllers use another mechanism to compute the checksum of mounted secrets and configmaps. (#1076, @stoyanr).status.lastError
section in the shoot status does now contain a new field lastUpdateTime
that indicates when the section has been updated the last time. (#1125, @vpnachev)PodSecurityPolicy
for calico-kube-controller
. (#1120, @mvladev)garden
namespace is protected. (#1139, @schrodit)ConfigMap
s or Secret
s and mounting them to the kube-addon-manager pod during deployment time.CustomResourceDefinition
s which allows to dynamically add, change, and remove resources with immediate action and without the need to reconfigure the volume mounts/restarting the pod.gardener-apiserver: eu.gcr.io/gardener-project/gardener/apiserver:0.26.0
gardener-controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:0.26.0
gardener-scheduler: eu.gcr.io/gardener-project/gardener/scheduler:0.26.0
Published by gardener-robot-ci-1 over 5 years ago
Seed
resource does now contain a new .spec.blockCIDRs
field. Every CIDR that is added here will be blocked for communcation of any control plane component of shoots in this seed. Typically, you should add the (cloud) provider's metadata service CIDR(s) here. For shooted seed clusters you might want to extend your shoot.garden.sapcloud.io/use-as-seed
annotation with blockCIDRs
, e.g. shoot.garden.sapcloud.io/use-as-seed="true,blockCIDRs=1.2.3.4/5;6.7.8.9/10
. (#1081, @rfranzke)0.22.0+
. Use the helping script to perform migration of etcd PVC for hibernated cluster. This will reduce the time for etcd migration. (#1038, @swapnilgm)ShootSeedManager
that was responsible for finding an adequate seed cluster when creating a shoot. Now, this admission plugin has been removed and its logic has been moved to the gardener-scheduler. Similar to how the kube-scheduler finds an adequate node when for scheduling a pod, the gardener-scheduler finds an adequate seed when scheduling a shoot. The Gardener Helm chart does now contain configuration for the scheduler that you can set when deploying this and further versions. (#981, @danielfoehrKn)NetworkPolicies
must be created, allowing Egress
or/and Ingress
traffic if custom components are deployed next to the Shoot control plane in the Seed cluster. (#904, @mvladev)
networking.gardener.cloud/to-
labels on these control plane components to enable Egress
traffic to various other components or endpoints.NetworkPolicy
controller (such as Calico) to all Seed clusters, so that NetworkPolicies
are enforced. (#904, @mvladev)spec.cloud.{PROVIDER}.networks.nodes
is now defaulted to the first worker CIDR, only if there is one worker in spec.cloud.{PROVIDER}.networks.workers
. (#1082, @mvladev)<shootName>.monitoring
in the project namespace in the garden cluster. (#1000, @wyb1)<shootName>.logging
in the project namespace in the garden cluster. (This requires that the Logging
feature gate is enabled in the gardener-controller-manager configuration.) (#851, @KristianZH)Extension
resources then, before Gardener deletes the controller, first all Extension
resources in the respective seed cluster(s) are deleted. Only after that the extension controller is deleted. This is to allow proper clean up. (#1095, @timuthy)etcd-main
now emits extensive (histogram) metrics. The cardinality of these metrics could be similar to the kube-apiserver's latency metrics. (#1069, @amshuman-kr)etcd-main-etcd-main-0
which was previously backing etcd-main pod before gardener version 0.22.0
. This will result in a restart of all etcd pods. (#1038, @swapnilgm)Egress
and Ingress
traffic is disabled. (#904, @mvladev)Waiting for Infrastructure
on failed Shoots that never had an infrastructure. (#1080, @adracus)2000m
CPU and 7Gi
memory, based on consistent recommendation from VPA. The requests are left untouched. (#1078, @amshuman-kr)kubelet.service
now reads from /etc/hostname
for hostname override. (#1077, @mvladev)Extension
resources hung in deletion. (#1071, @timuthy)kube-proxy
Init container crash, when IPVS
is enabled on systems without loaded IPv6
kernel module. (#1067, @mvladev)0.11.14
. (gardener/terraformer@157bce76c33d1d2ab95b28822be61f0edf300e63)gardener-apiserver: eu.gcr.io/gardener-project/gardener/apiserver:0.25.0
gardener-controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:0.25.0
gardener-scheduler: eu.gcr.io/gardener-project/gardener/scheduler:0.25.0
Published by gardener-robot-ci-1 over 5 years ago
extensions.extensions.gardener.cloud
of type: certificate-service
. (see GEP-1 on extensibility). This change also implies the removal of CertificateManagement
feature gate from gardener-controller-manager configuration (please remove when deploying this Gardener version, otherwise the gardener-controller-manager won't start). Also, if you previously have enabled the CertificateManagement
feature gate you have to create an adequate ControllerRegistration
resource now (e.g., see this). You also have to set .spec.resources.globallyEnabled=true
so that every shoot gets the certificate service extension. Previously, the configuration had been provided as a Secret
, but now it's part of the extension itself. See this for information how to configure ACME and the providers (note that clusterIssuerName
has been renamed to issuerName
). (#1029, @timuthy)ControllerRegistration
resources to make these extension controllers known to Gardener (otherwise none of your shoots will be able to get reconciled anymore). Please find example ControllerRegistration
resources here (Alicloud), here (AWS), here (Azure), here (GCP), and here (OpenStack). To get information about more details please walk through these documents. (#930, @rfranzke)ControllerRegistration
resources to make these extension controllers known to Gardener (otherwise none of your shoots will be able to get reconciled anymore). Please find example ControllerRegistration
resources here (Alicloud), here (AWS), here (Azure), here (GCP), and here (OpenStack). To get information about more details please walk through these documents. (#930, @rfranzke)CloudProfile
s, and Shoot
s. Now, the machine image for every cloud provider is just defined by a (name,version)
-tuple, e.g. {machineImage: {name: coreos, version: 1234.0}}
(see example CloudProfile
resources, e.g. this). The respective extension controller has to map this information to the cloud specific data, e.g., the AWS controller would map this to an AMI, the Azure controller to SKU, Offer, Publisher, etc. The extension controllers have a configuration (e.g., see this) that contains this mapping (basically, everything that previously was part of the CloudProfile
is now part of this configuration). You can configure this also via their Helm charts, see e.g. this. Please make sure, that you use exactly the same versions of the machine images that you used previously in the CloudProfile
s, otherwise, Gardener will trigger a rolling update of the worker machines of the shoot clusters. (#930, @rfranzke)
gardener-controller-manager
deployment: kubectl -n garden scale deployment/gardener-controller-manager --replicas=0
CloudProfile
s to correctly reflect the machine images (name/version) tuples instead of provider-specific fields/information.ControllerRegistration
resources for the provider extensions you want to use (with correct machine image mapping configuration).gardener-controller-manager
deployment: kubectl -n garden scale deployment/gardener-controller-manager --replicas=1
StatefulSet
s for the shoot clusters have been updated to use a fast storage volume. However, the migration of etcd data directory from the old volume to the new volume happens only when the etcd pod runs post migration. For hibernated clusters, this is not the case. Hence, migration does not happen for these clusters. To force migration of etcd for these clusters, we should use this script to scale up the etcd StatefulSet
s from hibernation and subsequently scale it back down post migration. This script is meant to be run once before the next release of Gardener which removes the slow volumes of etcd. (#1043, @georgekuruvillak)PodSecurityPolicy
gardener.unprivileged
now uses the default docker seccomp policy (https://docs.docker.com/engine/security/seccomp/). (#1048, @mvladev)
.spec.kubernetes.allowPrivilegedContainers=false
and blocked system calls are used by containers, then a new PodSecurityPolicy
needs to be created, allowing unconfined
seccomp access. See https://kubernetes.io/docs/concepts/policy/pod-security-policy/#seccomp..spec.kubernetes.kubelet
configuration for shoot clusters. (#1064, @afritzler)scaleDownUnneededTime
, scaleDownDelayAfterAdd
, scaleDownDelayAfterFailure
, scaleDownDelayAfterDelete
, scanInterval
) for the cluster-autoscaler in the shoot. Please consult the example Shoot
manifests for details. (#1049, @afritzler).spec.kubernetes.kubelet
configuration for shoot clusters. (#1040, @afritzler)Project
resource is extended with the optional field viewers
which is a list of subjects with read-only access (except Secret
s) to the Gardener API. (#1004, @vpnachev)BackupInfrastructure
resources prior to globally configured deletionGracePeriodDays
by annotating the resource with backupinfrastructure.garden.sapcloud.io/force-deletion=true
. (#1058, @swapnilgm)CloudProfile
and a Shoot
is created without specifying a machine image then the first one in the CloudProfile
is always chosen and considered to be the default. Previous versions of Gardener prohibited creating Shoot
resources without specifying a machine image when the referenced CloudProfile
did contain more than one machine image. (8969ff71e9549a5b2fb4255f408afaad097ab488)BackupInfrastructure
controller does now delete pending buckets faster (based on the configured grace period and the actual deletion and independent of the controller's sync period). (#1053, @swapnilgm)PodSecurityPolicy
gardener.unprivileged
now has a default seccomp policy runtime/default
for clusters >= 1.11
and docker/default
for clusters < 1.11
. (#1048, @mvladev).spec.kubernetes.clusterAutoscaler
configuration for shoot clusters. (#1033, @afritzler)etcd-backup
secret in the shoot namespaces does now also contain the bucket name. This is only a temporary solution until the backup infrastructure extension controllers have been implemented. By then, this information can be read out of the backup infrastructure extension CRD. (8fe1ce87d7ec89f45f7d71472c77272c7b545050)gardener.cloud/role=extension
as well as controllerregistration.core.gardener.cloud/name=<name>
. (2c0cb6f88fb612730fca7d05e6500540e6d8bd56)gardener.cloud/role=shoot
. The shoot.garden.sapcloud.io/hibernated
label still exists but is deprecated now. To get the information whether the shoot is hibernated or not the Cluster
extension resource should be consulted. (2c0cb6f88fb612730fca7d05e6500540e6d8bd56)gardener-apiserver: eu.gcr.io/gardener-project/gardener/apiserver:0.24.0
gardener-controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:0.24.0
Published by gardener-robot-ci-1 over 5 years ago
v3.7.2
. (#1018, @rfranzke)Flexvolumes
for selected Seeds on Alicloud. (#1025, @jia-jerry)gardener-apiserver: eu.gcr.io/gardener-project/gardener/apiserver:0.23.0
gardener-controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:0.23.0