This version supports stream computing engines Kafka and Pulsar and supports MySQL read-write splitting, introduces an interactive parameter configuration method.

We would like to extend our appreciation to all contributors who helped make this release happen.

Highlights

KubeBlocks supports Kafka v3.3
- Apache Kafka is an open-source distributed event storage and streaming computing system that provides high reliability, throughput, and low latency for data pipelines, streaming analysis, and data integration. It is widely used in log collection and metric monitoring scenarios. KubeBlocks supports Kafka v3.3, which announced that KRaft meets production requirements, providing better partition scalability and resilience, and saving the additional costs of ZooKeeper. In addition, KubeBlocks also supports pushing data changes from MySQL and PostgreSQL to Kafka, making it easier for users to further process and handle the data.
KubeBlocks supports Pulsar v2.11
- Apache Pulsar is an open-source distributed messaging and stream processing platform. It aims to provide scalability, high performance, and reliability to meet the needs of modern data processing and real-time messaging. KubeBlocks supports Apache Pulsar v2.11, and compared to traditional deployment methods, KubeBlocks can automate fault tolerance, scaling, and other day2 operations.
KubeBlocks supports MySQL read-write splitting
- Read-write splitting is designed to improve the read-only processing capability of a MySQL database cluster. All write queries are sent to the master node, while read-only queries that do not modify data are distributed to multiple slave nodes. Read-write splitting is used together with the MySQL Raft Group cluster, which automatically detects changes in the master node and uses the current master node of the cluster to achieve failover. Set read_write_splitting_policy as read-write splitting at the global or session level, and the default policy is LEAST_CURRENT_OPERATIONS, which routes read-only queries to the slave node with the least active operations. The MySQL Raft Group cluster supports up to 5 nodes.

Breaking changes

For KubeBlocks v0.6, newly created Redis and PG clusters need password to connect, while v0.5 used to allow password-free login
For KubeBlocks v0.6, the backup and restore function is updated greatly. You need to manually upgrade the 0.5 version cluster to 0.6 and update backupPolicy to use the new function.
For KubeBlocks v0.6, since password-free login function is not supported for Postgres cluster, when restoring cluster upgraded from v0.5, if the new restored cluster is in creating status for a long time, you can check the pod logs to see whether there is password authentication failed error, you can update password to fix this.

Acknowledgements

Thanks to everyone who made this release possible!

What's New

Pulsar

Cluster lifecycle management and operation management, supporting the creation of Pulsar clusters, cluster deletion, cluster restarting, horizontal scaling, vertical scaling, volume expanding, and parameter modifying.
For monitoring, support performance monitoring of CPU, memory, network read/write traffic, and more for ZooKeeper, BookKeeper, and Broker.

Kafka

Cluster lifecycle management and operation management, supporting cluster creation, deletion, horizontal scaling, vertical scaling, volume expanding, and modifying parameters.
- Horizontal scaling: In combined mode, broker replicas support 1, 3, or 5 copies. In separated mode, brokers support 1 to 100 copies and Kafka controllers support 1, 3, or 5 copies.
For monitoring, supports performance monitoring of Broker's CPU, memory, network read/write traffic, etc.

MySQL

MySQL RaftGroup Cluster supports Switchover
MySQL RaftGroup Cluster supports MySQL read-write splitting.
Data Recovery, Non-Destructive Recovery by Point-in-time（PITR）. It's a beta feature.
Support proxy enable when creating cluster. (Beta)
- The default specification of Vitess proxy can meet the vast majority of user scenarios. Vitess proxy automatically triggers resource increase or decrease based on changes in database nodes and node specifications, without manual operation. The CPU of Vitess proxy is 1/6 of the total number of CPU cores in the cluster nodes (three nodes), rounded up to 0.5c granularity, with a minimum of 0.5c and a maximum of 64c. The default number of replicas is 1, and currently, modifying the number of replicas is not supported
- Connection Address: The proxy has a default connection address that supports read-write splitting. The expose command can generate VPC addresses and public network addresses for the proxy connection address.
- Supports setting Vitess proxy read-write splitting policies.

PostgreSQL

Support switchover for PG Replication Cluster
Built-in pgBouncer

MongoDB

MongoDB replicaset supports switchover
Data recovery, non-destructive PITR (Beta)

Data migration

Add kbcli migration command, including creating migration tasks, viewing migration task list, viewing migration task details, terminating migration tasks, viewing logs, viewing migration templates, and other functions. Supports full migration and incremental synchronization.
Support MySQL data migration from MySQL 8.0 to MySQL 8.0.
Support PostgreSQL data migration from PostgreSQL 14 to PostgreSQL 14.
Support MongoDB data migration from MongoDB5.X/6.0 to MongoDB 5.X/6.0.

Compatibility

Support remote write to Prometheus server through Prometheus v2.41 - 2.45 compatibility testing
Adapt kbcli to package managers of Ubuntu and Amazon Linux 2
Adapt kbcli to Windows PowerShell and package managers
Support local environment running on Ubuntu and Amazon Linux 2 for kbcli playground
Support local environment running on Windows for kbcli playground

Usability

kbcli supports users to modify KubeBlocks parameters with local editing tools on the operating system
kbcli supports the "fault-inject" extension for fault injection
kbcli supports the "report" command to package cluster context information into a compressed file for problem troubleshooting assistance
kbcli supports interactive editing of configuration information for DB clusters. For cluster create, support --edit to interactively edit the yaml, and show the cluster summary before create.
Support canceling Hscale/Vscale OpsRequest that are running (Beta)
Add kbcli playground Grafana overview page
Kbcli alert can set email server
Support initializing the creation of databases and users
Specify a configuration file for initialization settings when installing KubeBlocks
Disk full lock for MySQL, PostgreSQL, MongoDB: When the database storage space is about to be full (disk usage is more than 90%), set the database to a read-only state.

Backup and Restore

The backup storage repository (backupRepo) refers to the target path where backup files are stored. KubeBlocks supports public cloud object storage and S3-compatible object storage.
Support modifying cluster configuration when restoring a new cluster
Add "describe-backup" command to view backup details

Observability

Support integration with external monitoring systems include Prometheus, VictoriaMetrics, and AMP. Cluster performance monitoring metrics are output to the target monitoring system through remote write.
Real-time dumping of operation logs for K8s/KubeBlocks clusters to object storage, divided by time and size, and providing read-only addresses
Real-time dumping of error logs for K8s/KubeBlocks clusters to object storage, divided by time and size, and providing read-only addresses

Bug Fixes

There is a quota problem, no default resource limits are set for containers such as metrics. (#2738, @nashtsai)
cli ops command output lost ops name (#2904, @ldming)
probe event lost (#3172, @xuriwuyun)
update probe internal module (#3414, @xuriwuyun)
cli playground init cluster without sa, role and rolebinding (#3428, @ldming)
cli created cluster with wrong sa name (#3500, @ldming)
The cluster status and pod status are inconsistent. clusters can terminate faster than pods. (#3558, @lynnleelhl)
enable consensus_set controller by feature gate (#3616, @free6om)
licensing description (#3664, @free6om)
change preflight checks rule for low spec machine in test case (#3722, @yangmeilly)
redis prohibits passwordless access by default (#3726,@Y-Rookie)
mongodb post start script (#3956, @xuriwuyun)
mongodb pod terminating takes too long time (#3971, @xuriwuyun)
mongodb restore problem, reset password (#3987, @xuriwuyun)
MongoDB does not receive signals when terminating pods. (#3990, @xuriwuyun)
add more judgements for mongodb replset initialization (#4034, @xuriwuyun)
parallel_transformer concurrent map writes (#4079, @free6om)
probe mysql binding test fail (#4082, @xuriwuyun)
kbcli set componentserviceaccontName (#4276, @xuriwuyun)
create a default sa if probes is required and serviceaccountname not set (#4280, @xuriwuyun)
The processing logic of WeSQL scaling is not stable enough, and downsizing can easily lead to inconsistent kernel and cluster states. (#4372, #4293, #3202, @leon-inf)
replicationSet cluster phase is abnormal during h/v-scale operations. (#4377, @Y-Rookie)
PVCs are occasionally not deleted after scale-in. (#4378, @lynnleelhl)
kbcli cluster connect show example error 4403 (#4404, @fengluodb)
if cluster backup field is nil, do nothing (#4435, @ldming)
failed to enable kafka addon (#4474, @ldming)
unstable data protection test case (#4497, @ldming)
shutdown redis-server when create replication relationship failed and update postgresql start scripts (#4568, @Y-Rookie)
sa creation problem for template (#4626, @xuriwuyun)
mongodb restore problem (#4690, @xuriwuyun)

API changes

New APIs:
- backuprepos.dataprotection.kubeblocks.io
- replicatedstatemachines.workloads.kubeblocks.io
- storageproviders.storage.kubeblocks.io
New API attributes:
- clusters.apps.kubeblocks.io API
  - spec.availabilityPolicy
  - spec.backup
  - spec.tenancy
  - spec.replicas
  - spec.resources
  - spec.storage
  - spec.mode
  - spec.parameters
  - spec.monitor
  - spec.network
  - spec.componentSpecs.noCreatePDB
- clusterdefinitions.apps.kubeblocks.io API
  - spec.componentDefs.componentDefRef
  - spec.componentDefs.configSpecs.asEnvfrom
  - spec.componentDefs.configSpecs.lazyRenderedConfigSpec
  - spec.componentDefs.statelessSpec
  - spec.componentDefs.statefulSpec
  - spec.componentDefs.switchoverSpec
  - spec.componentDefs.volumeProtectionSpec
  - spec.componentDefs.systemAccounts.accounts.provisionPolicy.statements.update
  - spec.componentDefs.consensusSpec.LLPodManagementPolicy
  - spec.componentDefs.consensusSpec.LLUpdateStrategy
- clusterversions.apps.kubeblocks.io API
  - spec.componentVersions.configSpecs.asEnvFrom
  - spec.componentVersions.configSpecs.lazyRenderedConfigSpec
  - spec.componentVersions.systemAccountSpec
  - spec.componentVersions.switchoverSpec
- configconstraint.apps.kubeblocks.io API
  - spec.toolsImageSpec
  - spec.scriptConfigs
  - spec.downwardAPIOptions
- opsrequests.apps.kubeblocks.io API
  - spec.cancel
  - spec.switchover
  - spec.ttlSecondsBeforeAbort
  - spec.scriptSpec
  - status.components.reason
  - status.components.message
  - spec.verticalScaling.classDefRef
  - status.lastConfiguration.components.classDefRef
- backuppolicytemplates.apps.kubeblocks.io API
  - spec.backupPolicies.schedule.StartingDeadlineMinutes
  - spec.backupPolicies.[snapshot|datafile|logfile].backupStatusUpdates.useTargetPodServiceAccount
- backups.dataprotection.kubeblocks.io API
  - status.sourceCluster
  - status.availableReplicas
  - status.manifests.backupTool.VolumeName
- backuppolicies.dataprotection.kubeblocks.io API
  - spec.[logfile|datafile].backupRepoName
  - spec.schedule.StartingDeadlineMinutes
  - spec.[snapshot|datafile|logfile].backupStatusUpdates.useTargetPodServiceAccount
- backuptools.dataprotection.kubeblocks.io API
  - spec.logical.podScope
- addons.extensions.kubeblocks.io API
  - spec.cliPlugins
  - spec.helm.chartsImage
  - spec.helm.chartsPathInImage
Modified API attributes:
- clusters.apps.kubeblocks.io API
  - spec.clusterDefinitionRef add validation MaxLength=63
  - spec.clusterVersionRef add validation MaxLength=63
  - spec.componentSpecs.name update validation MaxLength: 15 -> 22
  - spec.componentSpecs.name update validation Pattern: ^[a-z0-9]([a-z0-9\.\-]*[a-z0-9])?$ -> ^[a-z]([a-z0-9\-]*[a-z0-9])?$
  - spec.componentSpecs.componentDefRef update validation MaxLength: 63 -> 22
  - spec.componentSpecs.componentDefRef update validation Pattern: ^[a-z0-9]([a-z0-9\.\-]*[a-z0-9])?$ -> ^[a-z]([a-z0-9\-]*[a-z0-9])?$
  - spec.componentSpecs.classDefRef.name add validation MaxLength=63
  - spec.componentSpecs.classDefRef.name add validation Pattern=^[a-z0-9]([a-z0-9\.\-]*[a-z0-9])?$
  - spec.componentSpecs.switchPolicy default value: MaximumAvailability -> Noop, enum values{MaximumAvailability, MaximumDataProtection, Noop} -> {Noop}
- clusterdefinitions.apps.kubeblocks.io API
  - spec.componentDefs.name update validation MaxLength: 18 -> 22
  - spec.componentDefs.name update validation Pattern: ^[a-z0-9]([a-z0-9\.\-]*[a-z0-9])?$ -> ^[a-z]([a-z0-9\-]*[a-z0-9])?$
  - spec.componentDefs.scriptSpecs.namespace add validation Pattern=^[a-z0-9]([a-z0-9\-]*[a-z0-9])?$
  - spec.componentDefs.scriptSpecs.volumeName update validation MaxLength: 32 -> 63
  - spec.componentDefs.scriptSpecs.volumeName add validation Pattern=^[a-z]([a-z0-9\-]*[a-z0-9])?$
  - spec.componentDefs.replicationSpec update type: ReplicationSpec -> ReplicationSetSpec
  - spec.componentDefs.horizontalScalePolicy.type: delete Snapshot type and add CloneVolume type
- clusterversions.apps.kubeblocks.io API
  - spec.componentVersions.configSpecs.namespace add validation Pattern=^[a-z0-9]([a-z0-9\-]*[a-z0-9])?$
  - spec.componentVersions.configSpecs.volumeName update validation MaxLength: 32 -> 63
  - spec.componentVersions.configSpecs.volumeName add validation Pattern=^[a-z]([a-z0-9\-]*[a-z0-9])?$
- componentclassdefinition.apps.kubeblocks.io API
  - status.classes update type ComponentClassInstance -> ComponentClass
- componentresourceconstraint.apps.kubeblocks.io API
  - spec
- configconstraint.apps.kubeblocks.io API
  - spec.reloadOptions.unixSignalTrigger.processName delete validation Pattern=^[a-z0-9]([a-z0-9\.\-]*[a-z0-9])?$
  - spec.reloadOptions.shellTrigger
  - spec.reloadOptions.tplScriptTrigger.namespace add validation Pattern=^[a-z0-9]([a-z0-9\-]*[a-z0-9])?$
  - spec.formatterConfig.format add type props-plus
- opsrequests.apps.kubeblocks.io API
  - spec.clusterRef add validation XValidation:rule="self == oldSelf"
  - spec.type add type Switchover and DataScript
  - spec.type add validation XValidation:rule="self == oldSelf"
  - spec.upgrade add validation XValidation:rule="self == oldSelf"
  - spec.horizontalScaling add validation XValidation:rule="self == oldSelf"
  - spec.restart add validation XValidation:rule="self == oldSelf"
  - spec.reconfigure add validation XValidation:rule="self == oldSelf"
  - spec.expose add validation XValidation:rule="self == oldSelf"
  - spec.restoreFrom add validation XValidation:rule="self == oldSelf"
  - status.phase add type Cancelling and Cancelled
- backuppolicytemplates.apps.kubeblocks.io API
  - spec.backupPolicies.componentDefRef update validation MaxLength: 63 -> 22
  - spec.backupPolicies.componentDefRef update validation Pattern: ^[a-z0-9]([a-z0-9\.\-]*[a-z0-9])?$ -> ^[a-z]([a-z0-9\-]*[a-z0-9])?$
  - spec.backupPolicies.[snapshot|datafile|logfile].backupStatusUpdates.updateStage update validation: optional -> required
- backups.dataprotection.kubeblocks.io API
  - status.phase add type Running and Deleting
  - status.manifests.backupTool rename checkSum -> checksum and CheckPoint -> checkpoint
- backuppolicies.dataprotection.kubeblocks.io API
  - spec.[logfile|datafile].persistentVolumeClaim update validation: required -> optional
  - spec.[logfile|datafile].persistentVolumeClaim.name update validation: required -> optional
  - spec.[logfile|datafile].persistentVolumeClaim.storageClassName add validation Pattern:^[a-z0-9]([a-z0-9\.\-]*[a-z0-9])?$
  - spec.[logfile|datafile].persistentVolumeClaim.persistentVolumeConfigMap.name add validation Pattern: ^[a-z0-9]([a-z0-9\.\-]*[a-z0-9])?$
  - spec.[logfile|datafile].persistentVolumeClaim.persistentVolumeConfigMap.namespace add validation Pattern: ^[a-z0-9]([a-z0-9\-]*[a-z0-9])?$
- backuptools.dataprotection.kubeblocks.io API
  - spec.deployKind update value {job,daemon} -> {job, statefulSet}
Deleted API attributes:
- clusters.apps.kubeblocks.io API
  - spec.componentSpecs.primaryIndex
- clusterdefinitions.apps.kubeblocks.io API
  - spec.componentDefs.maxUnavailable
- clusterversions.apps.kubeblocks.io API
  - spec.componentVersions.clientImage
- componentclassdefinition.apps.kubeblocks.io API
  - spec.groups.resourceConstraintRef
- opsrequests.apps.kubeblocks.io API
  - spec.verticalScaling.class
  - status.lastConfiguration.components.class