Improvements

Rook v1.9.6 is a patch release limited in scope and focusing on feature additions and bug fixes to the Ceph operator.

• core: Delete v1beta1 cronjob only when v1 cronjob not found (#10480, @subhamkrai)
• helm: Increase memory of osd prepare pods (#10496, @parth-gr)
• file: Finalizer blocks deletion on existence of subvolume groups (#9915, @BlaineEXE)
• core: Increase liveness probe timeout to 2s (#10460, @subhamkrai)
• csi: Add RBAC to create token for RBD CSI ClusterRole (#10442, @Rakshith-R)
• doc: Update ceph external cluster flags (#10234, @parth-gr)
• osd: Add lvm flag for activation of legacy lvm OSDs (#10445, @leseb)
• ceph: update CSI-Addons side-car to latest (v0.4.0) release (#10454, @nixpanic)
• csi: Update storage class for csi-users in external cluster (#10278, @parth-gr)
• mon: Update node labels as per new key value (#10405, @humblec)
• test: Add tests for NFS CSI pvc (#10354, @Rakshith-R)

Improvements

Rook v1.9.5 is a patch release limited in scope and focusing on feature additions and bug fixes to the Ceph operator.

• csi: Update Ceph CSI from 3.6.1 to 3.6.2 (#10424, @Madhu-1)
• osd: Fix encrypted OSDs on latest Ceph releases and add test for encrypted osd (#10333, @satoru-takeuchi)
• core: Report basic cluster and version for Ceph v17.2.1 or newer if telemetry is enabled (#10341, @travisn)
• csi: Update sidecar to latest release (#10306, @Madhu-1)
• ceph: Enable holder pod if csi host networking is disabled (#10342, @Madhu-1)
• osd: Ignore the region label when it is the same as zone label in topology (#10383, @xu-cheng)
• csi: Change sidecar images repo to registry.k8s.io (#10116, @Madhu-1)
• osd: Improved errors for encrypted OSD provisioning failures (#10373, @leseb)
• csi: Do not block the reconcile if all clusters are not ready (#10380, @leseb)
• rgw: Convert FQDN RGW to an IP in external script (#10309, @parth-gr)
• pool: Delete undlerying ceph pool when blockpool CR is deleted (#10362, @travisn)
• docs: Fix broken doc links since the docs redesign (#10349, @subhamkrai)
• docs: Updated links to CRD objects in helm chart (#10348, @denis256)
• csi: Add --extra-create-metadata arg to csi sidecars (#10346, @pkalever)
• external: Format python files using tool black (#10422, @subhamkrai)
• external-script: Remove signature import (#10421, @subhamkrai)

Improvements

Rook v1.8.10 is a patch release limited in scope and focusing on small feature additions and bug fixes to the Ceph operator.

• core: Improve detection of filesystem properties for disk in use (#10230, @leseb)
• osd: Remove broken argument for upgraded OSDs on PVCs in legacy lvm mode (#10298, @leseb)
• osd: Allow the osd to take two hours to start in case of ceph maintenance (#10250, @travisn)
• operator: Report telemetry 'rook/version' in mon store (#10161, @BlaineEXE)

Improvements

Rook v1.9.4 is a patch release limited in scope and focusing on feature additions and bug fixes to the Ceph operator.

• build: Update examples and base operator image to use ceph v16.2.9 (#10275, @parth-gr)
• test: Add canary integration test for osd with metadata device (#10286, @satoru-takeuchi)
• csi: Update CephFS OSD caps for external clusters (#10325, @parth-gr)
• docs: New doc framework with mkdocs and related build changes (#10321, @galexrt @travisn)
• docs: Add ceph telemetry design doc (#10254, @BlaineEXE)
• osd: Remove broken argument for upgraded lvm-based OSDs on PVCs (#10298, @leseb)
• operator: Rename telemetry rook-version to rook/version (#10279, @BlaineEXE)
• helm: Only copy relevant monitoring settings (#10274, @travisn)
• rgw: Validate RGW endpoint IP for external clusters (#10006, @parth-gr)

Improvements

Rook v1.9.3 is a patch release limited in scope and focusing on feature additions and bug fixes to the Ceph operator.

• osd: Add additional checks so OSDs are not created on existing filesystems (#10230, @leseb)
• osd: Allow the osd to take a long time to start (#10250, @travisn)
• osd: Exit immediately if pvc block has not changed (#10246, @leseb)
• osd: Close the encrypted disk after cleanup is done (#10191, @leseb)
• operator: Increase memory limit of operator (#10195 #10199, @travisn)
• operator: Set rook-version in mon k/v store (#10161, @BlaineEXE)
• csi: Add the option to disable hostNetwork in the helm chart (#10253, @yuvalman)
• operator: Update tolerations name from controlplane to control-plane (#10242, @ttyS0)
• mon: quorumStatus.MonMap.Mons changed to quorumStatus.Quorum (#10172, @fengjiankui121)
• build: Add psp for missing roles (#10244, @y1r)
• webhook: Update the check-in cluster spec validation (#10226, @subhamkrai)
• osd: Fix disk uuid management (#10212, @satoru-takeuchi)
• test: Add canary integration tests for osd on device (#9931, @satoru-takeuchi)
• rgw: Support FQDN for RGW endpoint (#10232, @parth-gr)

Improvements

Rook v1.9.2 is a patch release limited in scope and focusing on feature additions and bug fixes to the Ceph operator.

• core: Enable multus network configuration with the csi driver (#9925, @leseb)
• build: Update minimum supported version to k8s 1.17 (#10123, @travisn)
• csi: Update ceph-csi to v3.6.1 release (#10150, @leseb)
• rgw: Fix dashboard admin creation for multiple object stores (#10137, @galexrt)
• helm: Add ingress annotations for ssl (#9936, @slavonicsniper)
• build: Update go-ceph version to v0.15 and remove ceph_preview (#10125, @subhamkrai)
• ci: use stringData instead of data for secret (#10156, @leseb)
• core: RGW realm pull spec is optional (#10139, @travisn)
• build: Add support for go 1.18 (#10127, @subhamkrai)
• csi: Refactor csi initialization code for setting parameters (#10124, @Madhu-1)
• helm: ingressClassName was not available for networking.k8s.io/v1 (#10132, @casperbiering)
• csi: Add/remove mon IP from csi config (#10135, @Madhu-1)
• build: Add a placeholder rule to the RGW RBAC (#10142, @travisn)
• build: Add placeholder NFS RBAC for csv generation (#10183, @Rakshith-R)

Improvements

Rook v1.9.1 is a patch release limited in scope and focusing on feature additions and bug fixes to the Ceph operator.

• monitoring: Disable new alerts that are not applicable to rook (#10109, @travisn)
• core: Examples set default priority class names (#10096, @travisn)
• nfs: use k8s.gcr.io for csi nfsplugin image (#10059, @Madhu-1)
• csi: Add CSI encryption support (#9940, @Rakshith-R)
• csi: Make GRPC timeout configurable (#10074, @Madhu-1)
• helm: Add missing default resources in rook components (#10089, @yuvalman)
• osd: Enable Quincy osd release version (#10088, @travisn)
• core: Update controller-runtime version to v0.11.2 (#10023, @subhamkrai)
• block: Validate name updates to the BlockPool CR (#10099, @Madhu-1)
• helm: Add ingressClassName field (#10093, @log1cb0mb)
• csi: Enable csi nfs to generate nfs rbac for csv (#10071, @Rakshith-R)
• docs: Update NFS docs including latest CSI work (#10014, @BlaineEXE)
• build: Generate a separate NFS CSI RBAC manifest (#10058, @BlaineEXE)
• docs: Add info about mounting NFS exports externally (#10078, @BlaineEXE)
• csi: Add nfs nodeplugin & provisioner (#9927, @Rakshith-R)
• build: update golang version to v1.17 (#10094, @subhamkrai)
• build: Add a placeholder rgw role for csv generation (#10108, @travisn)
• subvolumegroup: Refactor subvolumegroup controller (#10111, @Madhu-1)

Improvements

Rook v1.8.9 is a patch release limited in scope and focusing on small feature additions and bug fixes to the Ceph operator.

• helm: Add ingressClassName field (#10093, @log1cb0mb)
• monitoring: Only set prometheus rules ownerref in same namespace (#10028, @travisn)
• osd: only set kek to env var on encryption scenario (#10035, @leseb)
• docs: Update the s3 client example for accessing RGW (#9968, @thotz)
• osd: Add NixOS specific PATHs to check for lvm2 (#9967, @nazarewk)

Upgrade Guide

To upgrade from previous versions of Rook, see the Ceph upgrade guide.

Breaking Changes

• The MDS liveness and startup probes are now configured by the CephFilesystem CR instead of the CephCluster CR. To apply the MDS probes, they need to be specified in the CephFilesystem CR. See the CephFilesystem doc for more details.
• In the Helm charts, all Ceph components now have default values for the pod resources. The values can be modified or removed in values.yaml depending on cluster requirements.
• Prometheus rules are installed by the cluster Helm chart. If you were relying on the CephCluster CR setting monitoring.enabled to create the prometheus rules, they now need to be enabled by setting monitoring.createPrometheusRules in the cluster Helm chart values.
• Removed the obsolete cross build container, now unused by the CI

Features

• The number of mgr daemons for example clusters is increased to 2, resulting in a standby mgr daemon. If the active mgr goes down, Ceph will update the passive mgr to be active, and rook will update all the services with the label app=rook-ceph-mgr to direct traffic to the new active mgr.
• Network encryption is configurable with settings in the CephCluster CR. Requires the 5.11 kernel or newer.
• Network compression is configurable with settings in the CephCluster CR. Requires Ceph Quincy (v17) and a newer kernel as also required for encryption.
• Support for custom ceph.conf for csi pods.
• Ceph prometheus rules added and updated as recommended from the main Ceph project. The rules can be created with the cluster helm chart by setting the monitoring.createPrometheusRules, or see the monitoring documentation if not using helm.
• The service account rook-ceph-rgw is added to the RGW pods.
• A CephBlockPoolRadosNamespace CRD is provided to create RADOS namespaces in a pool.

Improvements

Rook v1.8.8 is a patch release limited in scope and focusing on small feature additions and bug fixes to the Ceph operator.

• core: Cluster CR status was not being refreshed after updating the cluster CR (#9962, @leseb)
• core: GetLogCollectorResources to get the right resources (#9898, @yuvalman)
• object: Remove unnecessary region option from the OBC StorageClass (#9906, @thotz)
• core: Add Phase in additionalPrinterColumns for all CRs (#9910, @subhamkrai)
• test: Avoid potential data inconsistency on zapping disk (#9930, @satoru-takeuchi)
• ci: Add pylint in ci (#9879, @subhamkrai)
• core: Incorrect join command in external cluster script (#9862, @vavuthu)
• core: Rework usage of ReportReconcileResult (#9873, @BlaineEXE)
• csi: Populate mon endpoints even if csi driver not enabled (#9878, @travisn)

Improvements

Rook v1.8.7 is a patch release limited in scope and focusing on small feature additions and bug fixes to the Ceph operator.

• build: Update ceph base image to v16.2.7-20220216 (#9814, @travisn)
• csi: default to ReadWriteOnceWithFSType for cephfs (#9729, @humblec)
• mon: Disable startup probe on canary pods (#9888, @travisn)
• core: Add Ceph FSID on the cephcluster CR status (#9847, @parth-gr)
• csi: Properly apply CSI resource requests and limits (#9868, @TomHellier)
• helm: Add resource requests and limits to the toolbox pod (#9856, @TomHellier)
• helm: Remove obsolete .Values.image.prefix (#9863, @kahirokunn)
• osd: Clarify vault auth error message (#9884, @leseb)
• nfs: Remove secret and configmap when downscaling NFS daemons (#9859, @BlaineEXE)
• helm: Handle empty StorageClass parameters for object, rbd, and cephfs in the helm chart (#9854, @Zempashi)
• helm: Remove obsolete setting for enabling multiple filesystems (#9841, @travisn)
• osd: Use lvm mode to create multiple OSDs per device (#9842, @BlaineEXE)
• helm: Add filesystem pool name to the storage class (#9838, @mtt0)
• docs: Document that the rook-ceph-operator-config ConfigMap is required (#9821, @matthiasr)
• core: Suppress verbose disruption controller log messages (#9834, @travisn)
• osd: Purge job will remove all pvcs for the osd, not just the data pvc (#9804, @travisn)
• osd: Remove osd with purge instead of destroy (#9807, @travisn)

Improvements

Rook v1.8.6 is a patch release limited in scope and focusing on small feature additions and bug fixes to the Ceph operator.

• osd: Enable raw mode for OSDs for Ceph v16.2.7 and newer (#9701, @satoru-takeuchi)
• file: Allow filesystem removal even with failures (#9799, @travisn)
• helm: Allow setting priority class on operator and toolbox (#9790, @travisn)
• docs: Warn in quickstart guide about using a vm (#9777, @travisn)
• helm: Fix the enabled flag for csiAddons (#9645, @peng225)
• osd: Return error if fail to list osds in prepare job (#9746, @travisn)
• core: various external mode enhancements (#9760, @leseb)
• ceph: remove rbd_default_features setting (#9758, @Madhu-1)
• docs: clarify teardown disk zap section (#9765, @BlaineEXE)
• docs: Add multus known limitations (#9511, @leseb)
• csi: Modify upgrade flag in external cluster (#9609, @parth-gr)
• object: Fix backend path for transit engine for rgw kms (#9728, @thotz)
• osd: Write log message when node is being skipped (#9718, @travisn)
• csi: Cleanup csi driver resources when zero cephclusters exist (#9713, @Rakshith-R)
• object: do not check for upgrade on external mode (#9724, @leseb)

Also try out the new v0.1.0 release of the Rook Krew Plugin!

Improvements

Rook v1.8.5 is a patch release limited in scope and focusing on small feature additions and bug fixes to the Ceph operator.

• core: Operator was crashing when the mon endpoints were updated (#9715, @leseb)
• csi: Bump registar image to v2.5.0 (#9702, @sathieu)
• osd: Add namespace in vault client (#9720, @leseb)
• doc: Add documentation for deploying the csi-addons (#9624, @yati1998)
• object: Update object pool properties during reconcile (#9696, @travisn)
• csi: Create separate csi users in external cluster configuration (#9703, @parth-gr)

Improvements

Rook v1.8.4 is a patch release limited in scope and focusing on small feature additions and bug fixes to the Ceph operator.

• csi: Skip validation when installing snapshotter (#9672, @Madhu-1)
• subvolumegroup: Handle external mode (#9694, @leseb)
• rgw: Gateway deployment strategy RollingUpdate on Pacific (#9578, @micalgenus)
• core: Dereference pointer before trying to compare with deepequal (#9663, @leseb)
• osd: Merge all osd resources including custom (#9658, @travisn)
• csi: bump csi snapshotter image to v5 (#9665, @Madhu-1)
• helm: add annotations and labels for VolumeSnapshotClasses (#9661, @Madhu-1)
• csi: Bump csi resizer and volume replication images (#9651, @sathieu)
• osd: Allow injecting extra OSD environment variables via ConfigMap (#9647, @rzarzynski)
• pool: Check for pool application already being set (#9649, @travisn)
• helm: Update CI to the latest helm and fix the linter (#9642, @travisn)
• rgw: Inject tls certs for bucket notification and topic operations (#9565, @thotz)
• rgw: Handle the non-existence of the bucket correctly when removing bucket (#9603, @cupnes)

Improvements

Rook v1.8.3 is a patch release limited in scope and focusing on small feature additions and bug fixes to the Ceph operator.

• helm: Apply operator settings in configmap instead of deployment (#9074, @travisn)
• csi: Update cephcsi to 3.5.1 release (#9627, @Madhu-1)
• csi: Update to the ceph csi release 3.5 image and sidecars (#9581, @humblec)
• csi: Change rook-ceph-csi-config to expose clusterID for subvolume (#9484, @leseb)
• core: Reconcile csi driver even if the operator configmap does not exist (#9613, @leseb)
• core: Support priority class for the crashcollector daemon (#9502, @satoru-takeuchi)
• csi: Add support to create a VolumeSnapshotClass (#9586, @Madhu-1)
• mon: Support annotations for critical metadata secrets and configmaps for backup (#9547, @subhamkrai)
• csi: Add osd blocklist capabilities to the external cephcluster (#9571, @parth-gr)
• csi: Add support for the csi-addons sidecar, though disabled by default (#9534, @yati1998)
• build: Set LC_ALL on the make target (#9594, @leseb)
• osd: Correct bluestore compression min blob size for ssd (#9582, @subhamkrai)
• osd: Add KMS support for IBM KeyProtect (#9545, @leseb)
• mgr: Update services with the label app=rook-ceph-mgr when the active Ceph mgr changes (#9467, @travisn)
• helm: Use operatorNamespace if exists for monitoring RBAC (#9554, @henryzhangsta)
• osd: Use cluster name when fetching the cephcluster (#9560, @leseb)
• security: Apdate auth permissions for external cluster (#9411, @parth-gr)
• core: Limit use of wildcards in cluster-scoped rbac (#9485, @BlaineEXE)

Improvements

Rook v1.7.11 is a patch release limited in scope and focusing on small feature additions and bug fixes to the Ceph operator.

• mgr: Update services with the app=rook-ceph-mgr label when the active Ceph mgr changes (#9467, @travisn)
• osd: Correct bluestore compression min blob size for ssd (#9582, @subhamkrai)
• build: Update to go v1.16.12 (#9478, @BlaineEXE)

Improvements

Rook v1.8.2 is a patch release limited in scope and focusing on small feature additions and bug fixes to the Ceph operator.

• subvolumegroup: Add CephFilesystemSubVolumeGroup CRD (#9384, @leseb)
• core: Run the discover daemonset as root uid (#9512, @leseb)
• core: Allow specifying daemon startup probes (#9468, @BlaineEXE)
• rgw: Startup probe uses same as the readiness probe instead of liveness probe (#9533, @satoru-takeuchi)
• rgw: Add events to the CephBucketNotification CR (#9400, @yuvalif)
• object: Update to the latest lib-bucket-provisioner (#9529, @BlaineEXE)
• build: Use the same csv name for all platforms (#9515, @leseb)
• object: Fix handling for notifications for OBC (#9365, @thotz)
• osd: Handle removal of encrypted osd deployment (#9434, @leseb)
• core: Re-enable check to disallow for multiple clusters in the same namespace (#9457, @leseb)
• operator: Report events more consistently for cluster status (#9427, @BlaineEXE)
• rgw: Add permissions to update finalizers on OBCs (#9441, @crd1985)
• docs: Recommend primary affinity over hybrid pools (#9465, @travisn)
• docs: Add jumbo frames note to common issues (#9453, @BlaineEXE)
• security: Add dry run mode for external cluster script (#9189, @subhamkrai)

Improvements

Rook v1.8.1 is a patch release limited in scope and focusing on small feature additions and bug fixes to the Ceph operator.

• file: Allow CephFS data pools to be named. NOTE For existing filesystems, do not change the names of existing pools or new pools will also be added to the filesystem (#9296, @degorenko)
• pool: Allow configuration of the built-in pools: device_health_metrics and .nfs (#9363, @travisn)
• osd: Add CAP_MKNOD privilege to the PSP for OSDs daemons (#9439, @y1r)
• mds: Skip sanity check during upgrades to 16.2.7 to avoid mon crash (#9418, @travisn)
• mgr: Start the watch-active container with correct privs when multiple mgr daemons are running (#9386, @leseb)
• helm: Add optional monitoring RBAC to operator chart (#9388, @BlaineEXE)
• pool: Add rbd ec pool support in external cluster (#9276, @subhamkrai)
• build: Remove tini command from the direct mount toolbox (#9423, @parth-gr)
• pool: Clean up health checkers for filesystem and object store forced deletion (#9417, @BlaineEXE)
• security: Use correct osd tags for restricted caps with external cluster script (#9410, @CO-lhagman)
• core: Add context parameter to various k8sutil helpers (#8700, @y1r)
• osd: Run the cleanup job using the root UID (#9380, @leseb)

Full Changelog: https://github.com/rook/rook/compare/v1.8.0...v1.8.1

Improvements

Rook v1.7.10 is a patch release limited in scope and focusing on small feature additions and bug fixes to the Ceph operator.

• mds: Skip MDS sanity check during upgrades to Ceph 16.2.7 to avoid mon crash (#9418, @travisn)
• pool: Clean up health checkers during filesystem and object store deletion (#9094, @BlaineEXE)
• core: On failure initializing the cluster, return the error instead of continuing (#9347, @leseb)
• osd: Truncate name of the osd prepare job to avoid pod generation failure on K8s 1.22 (#9312, @travisn)
• mon: Set stretch tiebreaker reliably during mon failover (#9282, @travisn)

K8s Version Support

K8s 1.16 or newer

The minimum supported version of Kubernetes is now 1.16. If running on an older version it is recommended to update to a newer version before updating to Rook v1.8.

Upgrade Guide

To upgrade from previous versions of Rook, see the Ceph upgrade guide.

Breaking Changes

• Support for Ceph Nautilus is removed. Clusters with Nautilus must upgrade to Octopus or Pacific before upgrading to v1.8.
• The Rook Flex driver is fully deprecated. If you are still using flex volumes, it is required to convert them to csi volumes before upgrading to v1.8. To convert your flex volumes, see the flex conversion tool.
• The directory structure of the example manifests has changed. The manifests have moved to deploy/examples. (The directory in previous releases was cluster/examples/kubernetes/ceph.)

Features

• Support added for Kubernetes Authentication when using HashiCorp Vault Key Management Service.
• The failure domain of a pool can be applied on the CephBlockPool instead of requiring manual commands run via the toolbox.
• The Rook Operator and the toolbox run under a rook user rather than root.
• The Rook image includes the s5cmd binary to simplify testing with S3 gateways from the toolbox.
• Rook adds a finalizer to a secret and configmap critical to the Ceph cluster
  so that the resources will not be accidentally deleted.
• The Rook Operator does not use "tini" as an init process. Instead, the rook binary handles
  signals on its own.
• Common K8s labels are added to the Rook and Ceph pods.
• The helm chart is the source of truth for generating crds.yaml, common.yaml, and the csv for integration with operatorhub.

Experimental Features

• Ceph RGW Bucket notification configuration is now supported via CRDs