Bot releases are hidden (Show)

ceph-csi - Ceph-CSI v3.10.0 Release Latest Release

Published by Rakshith-R 11 months ago

3.10.0 release notes

We are excited to announce another feature-packed release of Ceph CSI, v3.10.0. This is another significant step towards using enhanced Container Storage Interface (CSI) features with Ceph Cluster in the backend. With this release, we are introducing many brand-new features and enhancements to the Ceph CSI driver. Here are the changelog/release highlights.

Changelog and Highlights:

Features

RBD

deploy: support for read affinity options per cluster by @iPraveenParihar in #4165

CephFS

cephfs: Add support to create RWX PVC from ROX PVC by @Madhu-1 in #4094
cephfs: enable read affinity by @iPraveenParihar in #4153
deploy: support for read affinity options per cluster by @iPraveenParihar in #4165
cephfs: kernel and fuse mount options per cluster by @iPraveenParihar in #4245

NFS

nfs: add support for clients in the StorageClass by @spuiuk in #3895

Breaking Changes

Removed the deprecated grpc metrics flag's in #4225
Support for pre-creation of cephFS subvolumegroup before creating subvolume
is removed in #4195. Users will need to create the specified(or default csi) subvolumegroup before provisioning CephFS PVC on a new ceph. Refer to deploy-cephfs.md for more details.

Note:

Support is limited to only the active Ceph releases. Support for EOLed Ceph
releases are removed in #4262

Enhancements

rbd: do not execute rbd sparsify when volume is in use by @Rakshith-R in #3985
rbd: Don't depend on image state to issue resync by @Madhu-1 in #4076
rebase: all the dependencies have been updated to latest available release.
cephfs: handle cephfs clone limit error by @karthik-us in #4276
cephfs: remove snapshot protect/unprotect by @iPraveenParihar in #4202
cephfs: Update fetchIP() to add support for IPv6 address by @riya-singhal31 in #4230
helm: add option to enable read affinity for rbd by @iPraveenParihar in #4111
helm: Allow templating of RBD striping parameters by @KingJ in #4229
helm: add annotations for ceph-csi-rbd secret by @mustdiechik in #4248
util: Remove deprecated grpc metrics code by @Madhu-1 in #4225
util: include request-IDs in all gRPC calls for the Controller by @nixpanic in #4263
deploy: allow mkfsOptions by @Sea-you in #4233
deploy: update CSI sidecars to latest versions available by @iPraveenParihar in #4132
deploy: enable featuregate for volume expansion recovery by @iPraveenParihar in #4279
deploy: API for CSI Config Struct by @iPraveenParihar in #4278

Bug Fixes

cephfs: Fix cephfs PVC sizing by @karthik-us in #4180
cephfs: prevent hanging NodeGetVmolumeStats on stat() syscall when an MDS is slow by @nixpanic in #4200
cephfs: remove subvolume during clone by @Madhu-1 in #4223
cephfs: set Pool parameter to empty for Snapshot-backed volumes by @Rakshith-R in #4047
cephfs: safeguard localClusterState struct from race conditions by @Rakshith-R in #4163
rbd: do not try to run resizefs on an encrypted BlockMode volume by @nixpanic in #3958
rbd: discard not found error from GetMetadata by @Madhu-1 in #4097
rbd: update snap RbdImageName by @iPraveenParihar in #4152
rbd: update snap RbdImageName in createSnapshot by @iPraveenParihar in #4156

E2E

e2e: add test for validation of fuseMountOptions and kernelMountOptions by @riya-singhal31 in #3970
e2e: Fixing the TODO in createCephfsStorageClass by @karthik-us in #3995
e2e: add timeout for pvc deletion in ephemeral e2e by @riya-singhal31 in #4058
e2e: add option to enable read affinity for rbd by @iPraveenParihar in #4111
e2e: add multiple labels to node by @Madhu-1 in #4224

CI

ci: run tickgit after merging a PR in the devel branch by @nixpanic in #4241
ci: skip ./api/vendor in codespell runs by @nixpanic in #4201
ci: group golang.org/x/ under golang dependencies by @nixpanic in #4178
ci: disable addons after cluster creation by @Madhu-1 in #4170
ci: add CSI_UPGRADE_VERSION var to build.env by @Rakshith-R in #4008
ci: use podman for simple GitHub workflows by @nixpanic in #4035
mergify: add support for 3.9 backports by @Rakshith-R in #3936
ci: run versioned k8s jobs only on selected branches by @nixpanic in #4061
ci: exclude branches from the testing matrix for ok-to-test comments by @nixpanic in #4068
ci: only add /test .. comment if the branch for the PR matches by @nixpanic in #4070
ci: enable debug logs in kubelet by @Madhu-1 in #4074
ci: use dependabot group feature by @Madhu-1 in #4087
ci: add snyk for security scanning by @Madhu-1 in #4259
ci: add snyk for container image by @Madhu-1 in #4261
ci: add ci bot for auto assigning issue by @riya-singhal31 in #4275
ci: update minikube to v1.32.0 by @nixpanic in #4284
build: disable ceph-iscsi repository by @nixpanic in #3959
build: disable ceph-iscsi repository for test-container builds too by @nixpanic in #3965
build: make sure nfs-utils is installed by @nixpanic in #4243

Documentation

doc: remove /retest all command for Jenkins jobs by @nixpanic in #3957
doc: update documentation for v3.8.1 release by @Rakshith-R in #4006
doc: adding empty storageClassName in static pvc by @subhamkrai in #4010
doc: fix helm doc of ceph-csi deployment by @astraw99 in #4036
doc: Update cephcsi.go by @runzhliu in #4096
doc: adding empty storageClassName in static pvc by @Rakshith-R in #4110
doc: add design doc for RBD QoS by @Madhu-1 in #4089
doc: Remove unwanted steps/details from upgrade doc by @Madhu-1 in #4123
doc: remove storageclass details for shallow volume by @Madhu-1 in #4124
doc:add reference to CSI snapshot deployment guide by @iPraveenParihar in #4119
doc: remove unwanted steps from upgrade by @Madhu-1 in #4133
doc: Clarify default values for mountOptions and mkfsOptions in storageclass example by @MaGaroo in #4197
doc: add pending release notes by @Madhu-1 in #4222
doc: remove use of XXX in example descriptions by @nixpanic in #4228
doc: update release notes for grpc metrics by @Madhu-1 in #4237
doc: modify README and upgrade docs for release v3.10.0 by @Rakshith-R in #4286

New Contributors ( Thanks !! 👍 )

@spuiuk made their first contribution in #3895
@subhamkrai made their first contribution in #4010
@astraw99 made their first contribution in #4036
@runzhliu made their first contribution in #4096
@crazytaxii made their first contribution in #4099
@MaGaroo made their first contribution in #4197
@mustdiechik made their first contribution in #4248

Full Changelog: https://github.com/ceph/ceph-csi/compare/v3.9.0...v3.10.0

Thanks to the fantastic Ceph CSI community for this great release 👍 🎉

ceph-csi - Ceph-CSI v3.8.1 Release

Published by Rakshith-R about 1 year ago

Changelog and Highlights:

Bug Fixes

Do not execute rbd sparsify when volume is in use #3985
Fix invalid "invalid encryption kms configuration" error #3854
Limit cryptsetup PBKDF memory usage #3781

CI

Install Helm with script located on GitHub #3843
Disable ceph-iscsi repository #3959
Disable ceph-iscsi repository for test-container builds too #3965
Fix codespell and shell check failures #3762
Update github actions for k8s 1.27 #3745

Vendor Updates

Update golang to 1.19.8 #3768
Update minikube to 1.30 #3734
Bump github.com/hashicorp/vault from 1.4.2 to 1.9.9 #3712

Documentation

Use the Ceph Slack instance and not our silo'ed own one #3782

Full Changelog: https://github.com/ceph/ceph-csi/compare/v3.8.0...v3.8.1

ceph-csi - Ceph-CSI v3.9.0 Release

Published by Rakshith-R over 1 year ago

We are excited to announce another feature-packed release of Ceph CSI, v3.9.0. This is another significant step towards using enhanced Container Storage Interface (CSI) features with Ceph Cluster in the backend. With this release, we are introducing many brand-new features and enhancements to the Ceph CSI driver. Here are the changelog/release highlights.

Changelog and Highlights:

Features

RBD

Add mkfsOptions to the StorageClass and pass them to mkfs by @nixpanic in #3692
Get lastsyncbytes and lastsycduration for volume replication by @yati1998 in #3894
Complete removal and migration of replication server into csiaddons by @riya-singhal31 in #3924 #3884 #3608
Add support for efficient selinux relabelling support by @Rakshith-R in #3902

CephFS

Honor MountOptions during NodeStageVolume request and add support for efficient selinux relabelling support by @Rakshith-R in #3902
Add cephFS/CSIDriver and cephFS/csi-config-map to API by @riya-singhal31 in #3837

NFS

Add support for secTypes parameters in StorageClass by @nixpanic in #3434
Add support for efficient selinux relabelling support by @Rakshith-R in #3902

Breaking Changes

#3902 introduced a breaking change in CephFS driver. Please follow the steps mentioned in the upgrade guide to ensure a smooth upgrade.
Volumereplication service running on the controller server is now completely removed and replaced by CSI-Addons. See #3314 for more details

Enhancements

Exit early if image-meta.json does not exist by @microyahoo in #3788
Update golang to 1.20 #3879
Eebase: bump k8s.io/kubernetes from 1.26.2 to 1.27.2 by @dependabot in #3848
Update various dependencies #3896 #3850 #3917
Add support & e2e for mountOptions & efficient selinux relabelling support by @Rakshith-R in #3902
Helm: add imagePullSecrets for helm charts by @fungaren in #3906
Deploy: update CSI sidecars to the latest versions available by @iPraveenParihar in #3871
Cleanup: Move common files to the deploy folder by @karthik-us in #3860

Bug Fixes

Helm chart rendered duplicate affinities in rbd and cephFS by @dashjay in #3751
Invalid "invalid encryption KMS configuration" error by @riya-singhal31 in #3854
Configuring cephFS snapshots and clones by @riya-singhal31 in #3742
Limit cryptsetup PBKDF memory usage by @BenoitKnecht in #3781
Set pid limit only for nodeserver by @Madhu-1 in #3776

E2E

Remove extra check for snapshot count by @riya-singhal31 in #3735
Fix codespell and shell check failures by @riya-singhal31 in #3762
Add test cases for pv.Spec.MountOptions by @rakshith in #3902

CI

Install Helm with a script located on GitHub by @nixpanic in #3843
Many Mergify enhancements for better CI resource utilization #3672 #3797
Added gha-mergify-merge-queue-labels-copier action to better handle merge queue prs #3809
Use the "ceph-csi-bot" account for commenting on PRs by @nixpanic in #3877
Prevent Retest Workflow from running on forked repos by @nixpanic in #3883
Do not add ok-to-test if CentOS jobs were successful by @nixpanic in #3688
Run ci tests on latest k8s versions v1.25, v1.26 and v1.27

New Contributors ( Thanks !! 👍 )

@karthik-us made their first contribution in #3761
@microyahoo made their first contribution in #3788
@dashjay made their first contribution in #3751
@iPraveenParihar made their first contribution in #3871
@fungaren made their first contribution in #3906

Full Changelog: https://github.com/ceph/ceph-csi/compare/v3.8.0...v3.9.0

Thanks to the fantastic Ceph CSI community for this great release 👍 🎉

ceph-csi - Ceph-CSI v3.8.0 Release

Published by Madhu-1 over 1 year ago

We are excited to announce another feature-packed release of Ceph CSI, v3.8.0. This is another significant step towards using enhanced Container Storage Interface ( CSI) features with Ceph Cluster in the backend. With this release, we are introducing many brand-new features and enhancements to the Ceph CSI driver. Also, this release enabled smooth integration into various projects. Here are the changelog/release highlights.

Changelog and Highlights:

Features

RBD

fscrypt support #3310
- Add fscrypt integration with the Ceph CSI KMS. Supports ext4 on RBD. Snapshots are supported as well.
- Brief docs for fscrypt support #3571
Provide new command line configuration to enable read affinity #3639

CephFS

Shallow volumes for the ROX accessModes by default #3651
- Shallow volumes as default for cephfs ROX clones/restore for better performance.
Add fscrypt support for volumes, snapshots, and clones #3460
- There are dependencies with kernel and ceph

Enhancements

Update kubernetes dependencies to 1.26.1 #3638
Update go-ceph to 0.20.0 #3678
Update packages in release image #3635
Add basic upgrade documentation for Helm Charts #3655
Update rook installation to default latest version #3610
Add extraArgs for sidecars #3560
csidriver added to helper scripts #3573
Lift the minimum supported version of ceph to v15.0.0 #3513
Update csi spec to v1.7.0 #3503
Add commonLabels value to helm charts #3438

Bug Fixes

Make inode metrics optional in FilesystemNodeGetVolumeStats for CephFS #3407
Discover if StagingTargetPath in NodeExpandVolume exists #3624
Set disableInUseChecks on rbd volume #3605
Skip expanding for BackingSnapshot volume #3586
Fix CVEs in image #3526
Ignore stderr for ceph osd blocklist when there is no error #3524
Check volume details from original volumeID #2931
Setup encryption if rbdVol exits during CreateVol #3422
Return error if last sync time is not present #3489
Return abnormal if the mount is corrupted #3462
Fix namespace name update in metadata and rados object #3477
Remove dummy image workaround #3413
Get description from remote status #3392
Fix mdl configuration #3447
ParseAcceptLanguage takes a long time to parse complex tags #3439

E2E

Run E2E tests with kubernetes v1.26 release
Many tests are added to make sure we stay with backward compatibility for existing features of v3.7
New tests are added for features introduced in this release
Lots of cleanup and deprecated API removals were done on the test framework

CI

Update golang to 1.19.5 #3640
Many Mergify enhancements for better CI resource utilization #3672 #3671 #3684 #3681
Add GitHub action to trigger E2E #3468

Breaking Changes

Removal of option to run cephcsi as both controller and node server.

New Contributors ( Thanks !! 👍 )

@saiprashanth173 made their first contribution in https://github.com/ceph/ceph-csi/pull/3377
@bastienbosser made their first contribution in https://github.com/ceph/ceph-csi/pull/3438
@riya-singhal31 made their first contribution in https://github.com/ceph/ceph-csi/pull/3510
@Sea-you made their first contribution in https://github.com/ceph/ceph-csi/pull/3560
@lentzi90 made their first contribution in https://github.com/ceph/ceph-csi/pull/3595
@mohag made their first contribution in https://github.com/ceph/ceph-csi/pull/3635
@Syphdias made their first contribution in https://github.com/ceph/ceph-csi/pull/3655

Full Changelog: https://github.com/ceph/ceph-csi/compare/v3.7.2...v3.8.0

Thanks to the awesome Ceph CSI community for this great release 👍 🎉

ceph-csi - Ceph-CSI v3.7.2 Release

Published by Madhu-1 almost 2 years ago

Changelog or Highlights:

Bug Fixes:

CephFS

Delete subvolume if SetAllMetadata fails #3435
Allow subvolume creation if ceph cluster doesnt support metadata API #3423

RBD

Fix volume leak if metadata operation fails #3436

Vendor Update

Rebase: golang.org/x/text/language to v0.3.8 to fix a vulnerability #3439

CI improvements

Create kubernetes cluster with podman driver #3420

Breaking Changes

None.

ceph-csi - Ceph-CSI v3.7.1 Release

Published by humblec about 2 years ago

Bug Fixes:

rbd: fix bug in kmip kms Decrypt function & improve error msg https://github.com/ceph/ceph-csi/pull/3341
rbd: modify stripSecret mechanism in logGRPC() https://github.com/ceph/ceph-csi/pull/3350
cephfs: return success if metadata operation not supported https://github.com/ceph/ceph-csi/pull/3352
rbd: change default FsGroupPolicy to "File" for RBD CSI driver https://github.com/ceph/ceph-csi/pull/3364
rbd: map only primary image https://github.com/ceph/ceph-csi/pull/3373
ci: use resync to sync helm charts https://github.com/ceph/ceph-csi/pull/3374
cephfs: Fix subvolumegroup creation https://github.com/ceph/ceph-csi/pull/3376
rbd: create token and use it for vault SA everytime possible https://github.com/ceph/ceph-csi/pull/3378
rbd: use blocklist range cmd, fallback if it fails https://github.com/ceph/ceph-csi/pull/3386

NOTE

Helm upgrade may fail with message:

UPGRADE FAILED: cannot patch "rbd.csi.ceph.com" with kind CSIDriver: CSIDriver.storage.k8s.io "rbd.csi.ceph.com" is invalid: spec.fsGroupPolicy: Invalid value: "File": field is immutable"
 FAILED! => {"changed": false, "command": "/usr/sbin/helm --version=v3.7.1 upgrade -i --reset-values --create-namespace -f=/tmp/tmp2sr2me9a.yml ceph-csi ceph-csi/ceph-csi-rbd", "msg": "Failure when executing Helm command. Exited 1.\nstdout: \nstderr: Error: UPGRADE FAILED: cannot patch \"rbd.csi.ceph.com\" with kind CSIDriver: CSIDriver.storage.k8s.io \"rbd.csi.ceph.com\" is invalid: spec.fsGroupPolicy: Invalid value: \"File\": field is immutable\n", "stderr": "Error: UPGRADE FAILED: cannot patch \"rbd.csi.ceph.com\" with kind CSIDriver: CSIDriver.storage.k8s.io \"rbd.csi.ceph.com\" is invalid: spec.fsGroupPolicy: Invalid value: \"File\": field is immutable\n", "stderr_lines": ["Error: UPGRADE FAILED: cannot patch \"rbd.csi.ceph.com\" with kind CSIDriver: CSIDriver.storage.k8s.io \"rbd.csi.ceph.com\" is invalid: spec.fsGroupPolicy: Invalid value: \"File\": field is immutable"], "stdout": "", "stdout_lines": []}

If so, delete the csidriver object

kubectl delete csidriver rbd.csi.ceph.com

Then do helm upgrade

ceph-csi - Ceph-CSI v3.7.0 Release

Published by humblec about 2 years ago

We are excited to announce another feature packed release of Ceph CSI , v3.7.0. This is another great step towards making it possible to use enhanced features of Container Storage Interface ( CSI) with Ceph Cluster in the backend. With this release, we are introducing many brand new features and enhancements to Ceph CSI driver. Also this release enabled a smooth integration to various projects. Here are the changelog / release highlights..

Changelog and Highlights:

Features

KMIP integration for RBD PVC encryption
- The Key Management Interoperability Protocol (KMIP)
  is an extensible communication protocol
  that defines message formats for the manipulation
  of cryptographic keys on a key management server.
  Ceph-CSI can now be configured to connect to
  various KMS using KMIP for encrypting RBD volumes.
NFS
- Added support for volume expansion, snapshot, restore and clone.
- Added NFS nodeserver within CephCSI with support for pod networking with nsenter.
Support enabling PV and snapshot metadata on the RBD images and CephFS subvolumes
- For persistent volumes, clones and volume restores we support adding PVName/PVCName/PVCNamespace and ClusterName details
- For snapshot volumes we support adding snapshot-name/snapshot-namespace/snapshotcontent-name and ClusterName details
Shallow Read Only support for Ceph CSI driver:
- cephfs-csi expose CephFS snapshots as shallow, read-only volumes, without needing to clone the underlying snapshot data (https://github.com/ceph/ceph-csi/blob/devel/docs/design/proposals/cephfs-snapshot-shallow-ro-vol.md ) which enables users to Restore snapshots selectively - users may want to traverse snapshots, restoring data to a writable volume more selectively instead of restoring the whole snapshot and this feature also help to perform more efficient Volume backup.

Enhancements

All kubernetes sidecars ( external provisioner,snapshotter, resizer..etc) are rebased to latest available versions. Along with other dependency module updates this release consume go-ceph v0.17.0 and kubernetes 1.24.4 version.
snapshot API support has been lifted to GA version in this release.
From this release onwards, the CSI driver make use of File fsgroup policy for its fsgroup based operations.
New feature gates are enabled ( HonorPVReclaimPolicy..etc) in the sidecar deployments.

Bug Fixes

While mounting the volume, CSI drivers no longer open world wide permission on mount path ( See ).
Support linux kernels <=4.11.0, /sys/bus/rbd/supported_features is part of Linux kernel v4.11.0, prepare the supported feature attributes and use them in case if supported_features file is missing (See #2678)
Fix volume healer for StagingTargetPath issue for Kubernetes 1.24 (See #3176)
RBACs are restricted to a great extend in this release version compared to previous. The CSI driver operate on least required RBAC in a cluster from now on.

E2E

many tests are added for making sure we stay with backward compatibility for existing features of v3.6.
new tests are added for features introduced in this release
lots of cleanup and deprecated API removals done on the test framework
Dropped support for kubernetes v<=1.22 tests in the framework

Deprecation

Volumereplication service running on controller server is deprecated and replaced by CSI-Addons, see https://github.com/ceph/ceph-csi/issues/3314 for more details
cephfs provisioner will not make use of attacher sidecar from this release onwards. See https://github.com/ceph/ceph-csi/pull/3149 for more details

Breaking Changes

NFS daemonset is renamed from csi-nfs-node to csi-nfsplugin, refer to upgrade steps for more details.

NOTE

Helm upgrade may fail with message:

UPGRADE FAILED: cannot patch "rbd.csi.ceph.com" with kind CSIDriver: CSIDriver.storage.k8s.io "rbd.csi.ceph.com" is invalid: spec.fsGroupPolicy: Invalid value: "File": field is immutable"
 FAILED! => {"changed": false, "command": "/usr/sbin/helm --version=v3.7.0 upgrade -i --reset-values --create-namespace -f=/tmp/tmp2sr2me9a.yml ceph-csi ceph-csi/ceph-csi-rbd", "msg": "Failure when executing Helm command. Exited 1.\nstdout: \nstderr: Error: UPGRADE FAILED: cannot patch \"rbd.csi.ceph.com\" with kind CSIDriver: CSIDriver.storage.k8s.io \"rbd.csi.ceph.com\" is invalid: spec.fsGroupPolicy: Invalid value: \"File\": field is immutable\n", "stderr": "Error: UPGRADE FAILED: cannot patch \"rbd.csi.ceph.com\" with kind CSIDriver: CSIDriver.storage.k8s.io \"rbd.csi.ceph.com\" is invalid: spec.fsGroupPolicy: Invalid value: \"File\": field is immutable\n", "stderr_lines": ["Error: UPGRADE FAILED: cannot patch \"rbd.csi.ceph.com\" with kind CSIDriver: CSIDriver.storage.k8s.io \"rbd.csi.ceph.com\" is invalid: spec.fsGroupPolicy: Invalid value: \"File\": field is immutable"], "stdout": "", "stdout_lines": []}

If so, delete the csidriver object

kubectl delete csidriver rbd.csi.ceph.com

Then do helm upgrade

Release Image : docker pull quay.io/cephcsi/cephcsi:v3.7.0

New Contributors ( Thanks !! 👍 )

@losil made their first contribution in https://github.com/ceph/ceph-csi/pull/2993
@Cytrian made their first contribution in https://github.com/ceph/ceph-csi/pull/3091
@naveensrinivasan made their first contribution in https://github.com/ceph/ceph-csi/pull/3127
@irq0 made their first contribution in https://github.com/ceph/ceph-csi/pull/2912
@iceman91176 made their first contribution in https://github.com/ceph/ceph-csi/pull/3177
@BenoitKnecht made their first contribution in https://github.com/ceph/ceph-csi/pull/3232
@takmatsu made their first contribution in https://github.com/ceph/ceph-csi/pull/3233
@anthonyeleven made their first contribution in https://github.com/ceph/ceph-csi/pull/3274
@palvarez89 made their first contribution in https://github.com/ceph/ceph-csi/pull/3273

Full Changelog: https://github.com/ceph/ceph-csi/compare/v3.6.2...v3.7.0

Thanks to awesome Ceph CSI community for this great release 👍 🎉

ceph-csi - Ceph-CSI v3.6.2 Release

Published by Madhu-1 over 2 years ago

Changelog or Highlights:

Bug Fixes:

Add allowPrivilegeEscalation: true to containerSecurityContext to nodeplugin daemonset

NFS

Delete the CephFS volume when the export is already removed

RBD

Use vaultAuthPath variable name in error msg
Support pvc-pvc clone with different sc & encryption
Consider rbd as default mounter if not set
Fix bug with missing supported_features

CephFS

Skip NetNamespaceFilePath if the volume is pre-provisioned

CI improvements

Improve logging for kubectl_retry helper
Fix commitlint problem
Prevent ERR trap inheritance for kubectl_retry

Breaking Changes

None.

ceph-csi - Ceph-CSI v3.6.1 Release

Published by Madhu-1 over 2 years ago

Changelog or Highlights:

Feature:

Add network namespace to support pod networking for CephFS and RBD plugins.

Bug Fixes/Enhancements:

NFS

Add NFS provisioner & plugin sa to scc.yaml
Use go-ceph API for creating/deleting exports
Return gRPC status from CephFS CreateVolume failure

RBD

Fix logging in ExecuteCommandWithNSEnter
Check nbd tool features only for RBD driver
Use leases for leader election in RBD omap controller
Consider remote image health state for PromoteVolume

Breaking Changes

None.

ceph-csi - Ceph-CSI v3.6.0 Release

Published by humblec over 2 years ago

We are excited to announce another feature packed release of Ceph CSI , v3.6.0. This is another great step towards making it possible to use enhanced features of Container Storage Interface ( CSI) with Ceph Cluster in the backend. With this release, we are introducing many brand new features and enhancements to Ceph CSI driver. Also this release enabled a smooth integration to various projects. Here are the changelog / release highlights..

Changelog and Highlights:

New Features

NFS based dynamic provisioner:

Ceph-CSI already creates CephFS volumes, that can be mounted over the native CephFS protocol. A new provisioner in Ceph-CSI can create CephFS volumes, and include the required NFS CSI parameters so that the NFS CSI driver can mount the CephFS volume over NFS. The CephFS volumes would be internally managed by the NFS provisioner, and only be exposed as NFS CSI volumes towards the consumers.

Fuse Mount recovery

Mounts managed by ceph-fuse may get corrupted by e.g. the ceph-fuse process exiting abruptly, or its parent container being terminated, taking down its child processes with it. This was an issue for FUSE based CephFS mounts performed by the Ceph CSI driver, however from this release onwards CSI driver is capable of detecting the corrupted ceph fuse mounts and it will try to remount automatically.

AWS KMS Encryption

Ceph-CSI can be configured to use Amazon STS, when kubernetes cluster is configured with OIDC identity provider to fetch credentials to access Amazon KMS. With Amazon STS and kubernetes cluster is configured with OIDC identity provider, credentials to access Amazon KMS can be fetched using oidc-token(serviceaccount token).

Quincy Support

Ceph CSI driver has been built on top of Quincy release of Ceph.

Enhancements

Improved RBD image flattening support: from this release onwards, only temporary intermediate clones and snapshot will be flattened. See #2190 for more details.
Topology aware provisioning has been revisited with this release and enhancements have been made to make it more production ready.
image features as optional parameter in Storage Class make the rbd images features in the storageclass parameter list as optional so that default image features of librbd can be used.
Added support for deep-flatten image feature: as deep-flatten is long supported in ceph and its enabled by default in the librbd, via this enhancement we are providing an option to enable it in cephcsi for the rbd images we are creating.
Added selinuxMount flag to enable/disable /etc/selinux host mount: selinuxMount flag has been added to enable/disable /etc/selinux host mount inside pods to support selinux-enabled filesystems
A new reference tracker has been introduced with this release which is a key-based implementation of a reference counter. This allows accounting in situations where idempotency must be preserved.

Bug Fixes:

BlockMode recalimspace request has been adjusted to avoid data loss on the reclaim space operation
RBD and CephFS driver has fixed an issue at node mount operation, to take care explicit permission set done by the CSI driver previous to this release which was causing unwanted pod delay.
RBD force promote timeout has been increased to 2 minutes to give enough time for rollback to complete.
Storage class map options has been corrected to ensure it works in various combinations of the input setting from the storage class and also made it flexible to work with different mounters like kernel,nbd..etc.
Previously, restoring a snapshot with a new PVC results with a wrong dataPoolName in case of initial volume linked
to a storageClass with topology constraints and erasure coding. This has been fixed in this release.
omap deletion in DeleteSnapshot operation has been fixed with this release which helps to cleanup the omap properly once the subvolume snapshot is deleted.

Rebase

The dependencies of Ceph CSI driver are updated to latest version to consume various fixes and enhancements in the same.

E2E

Documentation

Breaking Changes

RBD Thick provisioning support is removed see #2795 for more details.

Release Image : docker pull quay.io/cephcsi/cephcsi:v3.6.0

Thanks to awesome Ceph CSI community for this great release 👍 🎉

ceph-csi - Ceph-CSI v3.5.1 Release

Published by Madhu-1 over 2 years ago

Changelog or Highlights:

Bug Fix:

Log cephfs clone failure message in CreateVolumeRequest
Use ceph 16.2.7 as the base image
Fix RBD parallel PVC creation hang issue

Breaking Changes

None.

ceph-csi - Ceph-CSI v3.5.0 Release

Published by humblec almost 3 years ago

We are excited to announce another feature packed release of Ceph CSI , v3.5.0. This is another great step towards making it possible to use enhanced features of Container Storage Interface ( CSI) with Ceph Cluster in the backend. With this release, we are introducing many brand new features and enhancements to Ceph CSI driver. Also this release enabled a smooth integration to various projects. Here are the changelog / release highlights..

Ceph CSI 3.5.0 Release Changelog/Highlights

New features

IBM HPCS/Key Protect KMS Support

Ceph CSI added support for IBM HPCS/Key protect KMS services. This enables admins to enable PV encryption by making use of IBM key protect services in a kubernetes or openshift cluster. ( https://github.com/ceph/ceph-csi/pull/2723)

Network Fencing

Ceph CSI now supports Network Fencing; which allows admins to blocklist any malicious clients. (https://github.com/ceph/ceph-csi/pull/2738)

Kubernetes in-tree RBD volume migration

Ceph CSI support in-tree kubernetes volume migration to CSI driver ( kubernetes.io/rbd to rbd.ceph.csi.com) which is available with kube 1.23 release. All requests to the kubernetes in-tree provisioner will be redirected to the Ceph CSI RBD driver for its operations. Refer here for more details.

Support for Reclaimspace operation

The Ceph CSI driver has added support for csi addon's nodeReclaimSpace and controllerReclaimSpace operation while csi addons sidecar request these services from the CSI driver. (https://github.com/ceph/ceph-csi/pull/2724 )

Ephemeral Volume

Ephermeral Volume Support have been validated with this release, With ephemeral volume support a user can specify ephemeral volumes in its pod spec and tie the lifecycle of the PVC with the POD.

RWOP PVC access mode

By advertising proper capabilities introduced in latest CSI spec 1.5, the Ceph CSI driver have been validated against RWOP PVC access mode which is introduced recently in kubernetes release.

Enhancements

Go-Ceph

Ceph CSI now uses go-ceph API for adding task to flatten image and remove image from trash instead of cmdline. This is expected to improve performance.

RBD krbd mounter

This release added RBD feature support for object-map, fast-diff ..etc with krbd mounter.

RBD nbd mounter

rbd-nbd can now support expansion of volumes, encrypted volumes and journal based mirroring. rbd-nbd log strategies can be tuned to, preserve, compress, remove on detach, read more about it here. nbd mounter utilize rbd-nbd cookie support at ceph-csi, to avoid any misconfiguration issues on nodeplugin restart, this adds to more reliable functionality of volume healer.

StorageClass Enhancements

The fixed security context can be enabled for PVs by mount options in the SC. This make it possible to specify selinux-related mount options like context.
Ceph CSI now provides a way to supply multiple mounters mapOption from storageclass, like mapOption: "kbrd:v1,v2,v3;nbd:v1,v2,v3"

Expansion of Volumes

The user can create the bigger PVC from an existing PVC and restore a snapshot to a bigger size PVC

Rebase

Along with many other dependency update of go packages which Ceph CSI uses, Ceph CSI have been rebased to make use of latest code release of kubernetes (v1.23) and also to make use of latest available sidecars.

e2e

rwop validation for cephfs and rbd volumes
added tests for bigger size rbd and cephfs Volumes
ephemeral validation have been enabled for rbd and cephfs in the e2e
test is added to validate encrypted image mount inside the nodeplugin
validation added for thick encrypted PVC restore
added tests to validate PVC restore from vaultKMS to vaulttenantSAKMS
intree migration tests are part of the e2e
ceph.conf deployment model has been accommodated in the tests
test cases added for pvc-pvcclone chain with depth 2
added tests for volume expansion, encrypted volumes with rbd-nbd mounter
covered tests for different accessModes and volumeModes with rbd-nbd mounter
added cases for snapshot restore chain with depth 2
...etc.

Documentation

design doc added for, CephFS snapshots as shallow RO volumes, in-tree migration, hpcs/key protect integration, clusterid poolid mapping,..etc
updated support matrix for deprecated ceph csi releases
updated development guide for new rules
updated rbd-nbd documentation with volume expansion, encryption volume support, various rbd-nbd log strategies..etc
support matrix update to readme
....etc

Breaking Changes

None

Release Image : docker pull quay.io/cephcsi/cephcsi:v3.5.0

Thanks to awesome Ceph CSI community for this great release 👍 🎉

ceph-csi - Ceph-CSI v3.4.0 Release

Published by humblec about 3 years ago

We are excited to announce another feature packed release of Ceph CSI , v3.4.0. This is another great step towards making it possible to use enhanced features of Container Storage Interface ( CSI) with Ceph Cluster in the backend. With this release, we have lifted many highly usable production features ( Snapshot, Clone, Metrics..etc) to its higher level of support. Also enhancements have been done on features like Encryption, Disaster Recovery, NBD mounter, Thick Provisioning..etc. Code improvements which increase performance on various CSI operations are also part of this release. With this release Ceph CSI make use of latest versions of kubernetes , sidecar containers, go ceph library which include many bug fixes and enhancements its own.

Changelog or Highlights:

Features:

Beta:

Below features have been lifted from its Alpha support to Beta

Snapshot creation and deletion
Volume restore from snapshot
Volume clone support
Volume/PV Metrics of File Mode Volume
Volume/PV Metrics of Block Mode Volume

Alpha:

rbd-nbd volume mounter

Enhancement:

Restore RBD snapshot to a different Pool
Snapshot schedule support for RBD mirrored PVC
Mirroring support for thick PVC
Multi-Tenant support for vault encryption
AmazonMetadata KMS provider support
rbd-nbd volume healer support
Locking enhancement for improving POD deletion performance
Improvements in lock handling for snap and clone operations
Better thick provisioning support
Create CephFS subvolume with VolumeNamePrefix
CephFS Subvolume path addition in PV object
Consumption of go-ceph APIs for various CephFS controller and node operations.
Resize of the RBD encrypted volume
Better error handling for GRPC
Golang profiling support for debugging
Updated Kubernetes sidecar versions to the latest release
Kubernetes dependency update to v1.21.2
Create storageclass and secrets using helm charts

CI/E2E

Expansion of RBD encrypted volumes
Update and addition of new static golang tools
Kubernetes v1.21 support
Unit tests for SecretsKMS
Test for Vault with ServiceAccount per Tenant
E2E for user secret based metadata encryption
Update rook.sh and Ceph cluster version in E2E
Added RBD test for testing sc, secret via helm
Update feature gates setting from minikube.sh
Add CephFS test for sc, secret via helm
Add e2e for static PVC without imageFeature parameter
Make use of snapshot v1 API and client sets in e2e tests
Validate thick-provisioned PVC-PVC cloning
Adding retry support for various e2e failure scenarios
Refactor KMS configuration and usage

Documentation

Hashicorp Vault with a ServiceAccount per Tenant
Added documentation for Disaster Recovery
rbd-nbd mounter
Updated helm chart doc
Contribution guide update

Breaking Changes

None

Thanks to awesome Ceph CSI community for this great release 👍 🎉

ceph-csi - Ceph CSI v3.2.2 Release

Published by Madhu-1 over 3 years ago

Changelog or Highlights:

Bug Fixes

Build

Update ceph to 15.2.11 to fix CVE-2021-20288

Breaking Changes

None.

ceph-csi - Ceph CSI v3.3.1 Release

Published by Madhu-1 over 3 years ago

Changelog or Highlights:

Bug Fixes

Build

Update ceph to 15.2.11 #1995
Fix helm chart push issue #2007

RBD

Modified logic to check image watchers to avoid already in use issue for mirroring image #1993
Return crypt error for the rpc return #2005

Breaking Changes

None.

ceph-csi - Ceph CSI v3.3.0 Release

Published by humblec over 3 years ago

Changelog or Highlights:

Features:

Async DR

A new volume replication protobuf and specification to achieve the Volume replication has been added with Ceph CSI driver.
Ceph CSI has implemented the required GRPC services ( EnableVolumeReplication, DisableVolumeReplication, PromoteVolume, DemoteVolume, ResyncVolume..etc) for volume replication. A new sidecar controller will be deployed as part of the RBD provisioner pod which will expose the CRD to a user to interact with the Ceph cluster for DR operations. When a User creates a CR with the PVC name, the new operator will get the required pvc and PV information and send a request to the ceph csi to perform the rbd async operation.

Encryption

Users will be able to configure AWS KMS for Ceph-CSI volume encryption. This makes it possible to have in-flight encrypted data, and securely stored volume contents on Ceph clusters outside of the control/responsibility of the Ceph-CSI deployer. With this addition
- users can enable volume encryption in a StorageClass
- the CMK configured in Amazon KMS will be used for encrypting/decrypting the DEKs
- the encrypted DEK for a volume will be stored in the volumes metadata
Snapshot and cloning on encrypted RBD PVCs are enabled.

Multus Support

Added support for network namespaces (Multus CNI)

Enhancement:

Update Kubernetes sidecars to latest releases
Update go-ceph to the latest release
The external snapshotter APIs are updated from v1beta1 to V1
Proper reuse of go ceph cluster Connections are established with this release.
Fixed many warnings/errors reported by static code analyzers
CSI driver creates a CSIDriver object, Kubernetes users can easily discover the CSI Drivers installed on their cluster (simply by issuing kubectl get CSIDriver)
E2E tests are added/updated with this release to make sure the stability of the code achieved on various use cases and also for new features.
Build utilities and dependencies are updated to latest versions.
CSI driver deployment yamls are updated and various helm chart fixes for snapshot controller deployment, RBAC permissions...etc are part of this release.

CI

Make use of ceph users created in e2e
Enhanced e2e logging for failure debugging
Track deletion of PVC and PV more closely
Error out in case deploying Hashicorp Vault fails
Added e2e for snapshot retention case/scenario
Updated feature gate settings from minikube
Verify (non)existence of keys for VaultTokensKMS
Pass namespace once in deletePodWithLabel()
Use secret with "encryptionPassphrase" for RBD tests

Documentation

Updated snapshot and clone documentation
Updated Encryption documentation for new KMS provider support and for other enhancements
Corrected various reference link issues on doc
Upgrade documentation is updated for release 3.3
Updated release matrix and compatibility docs
Various cleanups and corrections in general.

Breaking Changes

None

NOTE:

Ceph CSI repo Master branch has been renamed to Devel

ceph-csi - Ceph CSI v3.2.1 Release

Published by nixpanic almost 4 years ago

Changelog or Highlights:

Bug Fixes

Deployment

Fix snapshot controller deployment (#1823)

RBD

Fix namespace json parser (#1822)

Breaking Changes

None.

ceph-csi - Ceph CSI v3.2.0 Release

Published by Madhu-1 almost 4 years ago

Changelog or Highlights:

Features:

Add new controller sidecar for omap regeneration
- In case of Disaster Recovery, the admin can restore the Kubernetes objects on the second cluster. if the RBD images
  are available in the second cluster and admin as created a static binding between PVC and PV, the controller will
  regenerate the omap data.
Add support for map options in RBD Storageclass
Add support for unmap options in RBD Storageclass