KeyDB

KeyDB 6.3.4 Release Notes:

• Add keydb_modstatsd, a module for providing keydb stats to a local statsd server. Can be found under src/modules/keydb_modstatsd
• Fixed FreeBSD compile(Thanks to @bra-fsn)
• Fixed a bug in s3 bucket config that blocked startup(Thanks to Alexandre Chichmanian)
• Fixed a bug causing crash if keys command is called after a blocking command (Fixes issues #659, #605, #532)
• Added proper error checking of replica configs, now KeyDB will throw an error if replica-of config is passed before active-replica or multimaster configs
• Fixed double free bug in lazy free
• Added "overload-protect-percent" config, when enabled this will load shed clients whenever CPU usage exceeds configured value
• Added "availability-zone" config, this can be passed any string which will then be reported by info command
• Updated eviction logic to account for total system memory availability using sysinfo
• Fixed a bug where repl-backlog-size config was modified in keydb.conf with the runtime value during config rewrite
• Fixed a possible deadlock when running CLIENT KILL with large number of clients
• Fixed a bug where KeyDB would overcommit memory during fork BG save
• Fixed a bug with disk repl_backlog causing double free
• Fixed integer overflow issue in the temp rdb file naming (Thanks to @karthyuom)
• Fixed compile issue with GCC 13.1.1 (Thanks to @michieldwitte)
• Removed expireset and restored redis expire behaviour
• Fixed a bug causing forked processes to hang, specifically affecting RDB and AOF(Fixes issues #675, #619)
• Added "CLUSTER REPLICATE NO ONE" to turn a replica into an empty primary
• Added RDB-less full sync, can be enabled with config 'enable-keydb-fastsync'

FLASH(Beta):

• Added "flash-disable-key-cache" config to disable key cache which stores every key(but not value) in memory
• Moved cluster slot to key map to being stored by storage provider, rather than in memory
• Moved expires to be stored by storage provider, rather than in memory
• Enabled expiry/eviction from storage provider, previously only data stored in memory could be expired/evicted(Fixes #645 along with the 2 previous updates)
• Fixed a bug where swapdb result was not recovered after keydb restarts in FLASH mode (Thanks to @karthyuom)
• Fixed double free bug when fast sync was canceled early
• Fixed a bug where a temp rdb file with zero bytes is generated in flash mode (Thanks to @karthyuom)
• Fixed a bug where flash CF options are being reset to default after flushall (Thanks to @karthyuom)(Fixes issue #717)
• Updated debug reload command to work with flash

Dockerfile:

• Added support for KEYDB_PASSWORD env variable(Thanks to @einar-pexip)
• Compile with flash in Dockerfile(Thanks to @der-eismann)
• Fix broken redis-cli symlink (Thanks to @pimvandenbroek)
• Use tini for alpine docker(Thanks to @rofafor)

Version 6.3.3 Release Notes

To help accelerate our development efforts for KeyDB, this will be the last release containing support for Centos 7, Ubuntu 16.04, Ubuntu 18.04, Debian 9 and 32-bit builds. For more information click here.

This release contains fixes for 17 issues along with improvements to the KeyDB FLASH feature:

• Fixed race condition with expireset access (Issue #597)
• Fixed keys command with lua, and added keydb as an alternative to redis as lua variable to access db (Issue #562 and #594)
• Fixed hang in aof child (Issue #554)
• Fixed leaking fds from RDB save (Issue #453 and #584)
• Added config for S3 RDB load/save (Issue #584)
• Enabled active defrag during forkless background save to improve average memory efficiency (Issue #460)
• Fixed crash with fork background save during replication (Issue #567)
• Fixed integer overflow in rand family of commands(Issue #631, #632, #633)
• Fixed bad value in hincrbyfloat(Issue #634)
• Fixed OOM hang in rand family of commands(Issue #635)
• Added config to limit count of return values in rand family of commands(Issue #636)

Updates to KeyDB FLASH (Beta):

• RocksDB has been updated to v7.9.2
• Fixed missing slot_to_key map in FLASH cluster mode (Issue #574)
• Added keyspace notifications for keys loaded from FLASH at startup
• Fixed race condition in prefetchKeysAsync with FLASH enabled (Issue #571)

KeyDB v6.3.2 Release

This release contains Beta level support for KeyDB FLASH, new ASYNC commands, latency improvements and a number of bug fixes.

KeyDB FLASH Support

KeyDB FLASH is included as a Beta feature with this release. Enabling this feature avoids the need to store all data in memory, allowing you to store more data at a lower cost. KeyDB will persist data to the storage medium it is written to avoiding the need for AOF/RDB files. KeyDB uses RocksDB as the persistent storage provider and can be enabled with config storage-provider flash /path/to/flash/output. Read more at https://docs.keydb.dev/docs/flash/

New ASYNC Commands

In addition to GET/MGET support released with v6.3.0, ASYNC support has been added for the following commands: HGET, HMGET, HKEYS, HVALS, HGETALL, HSCAN and can be enabled with config enable-async-commands yes

Jammy & Bookworm Support

Packaging support for Ubuntu 22.04 (Jammy) and Debian 12 (Bookworm) has been included with this release and will be maintained moving forwards. For details on installation please refer to https://docs.keydb.dev/docs/ppa-deb

Other Improvements & Bug Fixes

• Added new soft shutdown feature, can be enabled with config "soft-shutdown yes".
• If soft shutdown is enabled, instead of shutting down right away, the server will wait until all clients have disconnected, and will reject all new connection attempts.
• Fixed memory leak with tls certificates when tls allowlist is enabled
• Fixed bug in rdb load with flash enabled to ensure all dbs are safe to load (previously only checked db[0])
• Fixed race conditions in rdb load and replication
• Fixed memory access of rdb file after it should have been deleted
• Fixed integer overflow bug in flash(Issue #486)
• Improve TLS latency by queueing new commands before executing instead of after
• Removed O(n) count of memory usage from info command(replaced by O(1) estimate)
• Improved latency of clearing large number of flash DBs(Thanks to Paul Chen for this fix)(Issue #516)
• replaced sprintf with snprintf to avoid potential security bugs
• Fixed bug where a failed move due to key already existing in move target would result in the key being removed from move source(Thanks to Paul Chen for this fix)(Issue #497)
• Fixed usage of deprecated OpenSSL api in OpenSSL v>3.0.1(Issue #392)
• Imported security fixes from Redis (CVE-2023-22458 CVE-2022-35977)
• Other fixed issues: #480 #477 #454 #452 #303 #425 #492 #541

KeyDB v6.3.1 Release

This point release contains fixes to bugs related to expires, active-rep, and rdb saving.

The following Issues have been resolved in this release: #419, #422, #428
PRs associated with this release: #426, #429, #431, #433

Please see main commit notes below. For full details please reference PRs

• Fix reference counting failure in the dict
• Fix assertion in async rehash
• Prevent crash on exit by avoiding destructors on exit
• Prevent accidental rehash initiation when an async rehash is aborted
• Respond to cancelled rehashes earlier to avoid wasted CPU
• Add memory fence barriers to the replication backlog
• Silence TSAN errors on ustime and mstime
• Resolve incorrect locking during repl backlog resize
• Make async commands "opt-in" as it can change the percieved ordering across clients
• Fix reference counting failure in the dict

KeyDB v6.3.0 Release

KeyDB version 6.3.0 is the first open source release to include previously KeyDB Enterprise Features! This release is the culmination of many years of work to move away from KeyDB’s global lock and enable better scaling across cores.

Active Replication PSYNC

Active Replication has been a feature of many KeyDB releases, but always required a full sync while connecting to a new peer. In this release KeyDB now supports partial synchronization to enable fast cluster healing. In addition KeyDB is able to update its replication offset based upon the knowledge of peer nodes, meaning a full sync from one node is enough to permit partial syncs to other nodes within a mesh.

This change greatly reduces the time to add a new node to an active replication cluster. In our testing we’ve seen the time reduced from well over 10 minutes to a minute and a half for a 12 node mesh cluster.

Async Commands

Async commands are commands which can execute without the global lock. This feature must be enabled first in your configuration file by setting “enable-async-commands yes”. When enabling async commands the consistency is modified slightly, in particular writes from other clients may take a configured amount of time to become visible to other clients. This time is configured with the max-snapshot-slip configuration parameter.

The ordering rules with async commands are as follows:
Writes are always totally ordered among clients
Writes are always immediately visible by the client that performed the write

For most applications the slight modification to the consistency model of Active Replication will not be visible. If your application relies on ordering between different clients this feature is disabled by default. In addition a new command called “lfence” is provided for clients that do rely on consistency across clients but are willing to insert fences to assert total ordering at the correct times.

The following commands support async operation:
GET
MGET

Additional read-only commands will get added over time. If a command is of specific interest to you please post an issue to help us prioritize your use case.

Async Rehash

KeyDB relies upon a spinlock to synchronize threads. In prior versions of KeyDB the CPU time consumed while waiting for the lock to be acquired was wasted. KeyDB now has the ability to perform rehash during this time using otherwise wasted CPU time for a more useful purpose. In many cases this can almost completely hide the overhead of rehashing. No configuration is required to enable this functionality, it operates by default.

In Process Background Saving

Historically Redis has used the “fork” system call to create a new process which performed the background save. This made the code easier to follow as the kernel handled the hard task of copy-on-write of memory while background saving was in progress. However the downside to this approach is that it is not possible to accurately estimate the total amount of memory required including background saving.

KeyDB now uses a new “snapshot” system to create snapshots of the database at a specific period and copy-on-write new changes within the process. As a result the maxmemory setting is now a global setting and will include memory used for background save.

For backwards compatibility the semantics of maxmemory are modified slightly during a background save. While a background save is in progress KeyDB will permit memory consumption to exceed maxmemory by up to 20%. This is to simulate the old behavior of background save memory consumption not being counted towards the maxmemory setting while permitting easy calculation of a total upper bound for memory usage.

IStorage Interface

This release does not yet include KeyDB’s FLASH feature based upon RocksDB. However the release does include KeyDB’s IStorage interface which is the backbone of our persistent storage feature. By implementing this interface for your custom storage solution you will enable all KeyDB features.

Thanks To Our Contributors:

Thank you to everyone who contributed to KeyDB with bug reports, design and testing:

Ben Schermel
Firaenix: Bug Report #393 - Crash in sorted sets with long names
Kerog: #401 Error after include *.confg
Gvsafronov: Fedora 35 Compile Failure
Talkabout: Always providing helpful and detailed bug reports, not limited to #378
Inakisoriamrf: #383 crash
Alebcay: #384 macOS build break
Server2245: #379 Install issue on RHEL7
Antiarchitect: #380, and many others
Tchernomax: #352 missing support for systemd
Heng Kuang: Detailed bug reports and design suggestions
Kajaruban Surendran: Detailed performance analysis of async rehash and other features
Paul Chen: Your enthusiasm and willingness to help debug issues has been deeply appreciated.

Special thanks to Huawei Canada Research Team for your detailed and in-depth bug reports and feature requests and design participation.

Thanks to the EQ Alpha Team that made this release possible: Malavan Sotheeswaran, Vivek Saini, Christian Legge, Peter Liang, Ben Schermel, John Sully

Changes:

• Acquire lock in module.cpp to fix module test break
• Fix usage of nanosleep() on Apple hardware
• Fix networking deadlock (#183)
• Remove redundant zfree() in keydb-benchmark
• Added a read/write lock to fix forking issues

Fixed Issues:
#378 #384

This release of KeyDB is at parity with Redis 6.2.6. In addition, the following changes were included:

• Systemd support for release packages
• Fixes to buffer overflows in the BITOP SHIFT and CRON commands
• Server times are now computed on a seperate thread to improve performance
• Now enforces syslog identity and facility as soon as possible
• Removed erroneous use of LOG_... flags when using syslog
• Fixed erroneous #endif leading to build errors on some platforms
• Fixed the incorrect counting of client connections

Issues resolved:
#355, #370

Changes:

• Removed unused command line options(-a, -d, -P, -r, -q, --csv, -l, -I, -e, --precision, --cluster, --enable-tracking)
• --ms option changed to --time, added --clients, --host, --port, and --threads options
• Added keydb-diagnostic-tool
• Improved test reliability
• Fixed memory leak in keydb-benchmark
• Fixed a number of race conditions
• Fixed OSX build issues
• Feature parity with Redis 6.2.3

Fixed Issues:
#323 #325 #276

Changes:

• Fixed memory leak in MVCC
• Improvements to internal timers
• Reduced redundant synchronization
• Improved test reliability
• Added config option for loading callback interval keys
• Added ascii logo image!
• Added config to disable message of the day
• Improved multithreaded replication performance
• Feature parity with redis 6.0.10

Fixed Issues:
#300 #285 #273 #260 #257 #238

This release has the following enhancements:

• Reduced memory consumption when Active Replication is not used
• Reduced CPU consumption
• Improved TLS performance
• replica-quorum config for multi-master.
  Intended to be used with "replica-serve-stale-data no", if at least N replicas are connected we will consider our master link up and serve data.
• Subkey expires performance improvement for big sets

The following issues were fixed:
#234 #236 #233 #207 #229 #231

New Features!
KEYDB.MEXISTS - return exactly which keys exist in the database not just the count
REPLICAOF REMOVE - Remove specific masters in a multi master setup
multi-master-no-forward configuration option - Reduce network traffic in mesh topology multimaster setups
Redis 6.0.5 feature parity

Fixed Issues:
#194 Bad directive or wrong number of arguments error on 'lazyfree-lazy-user-del no'
#209 Timeout using modules
#210 Databases are not merged on multi-master sync

This is our first release to fully support all Redis 6.0.4 features, including but not limited to:

• TLS Support (fully supports multithreading!)
• Client side caching
• RESP 3 Support

KeyDB Has also added the following new features:

• Improved memory efficiency for short strings
• Fastlock autotuning
• KeyDB.HRENAME - rename a member of a hash

In addition we've spent a lot of time focussing on stability. The following bug fixes are resolved:
#150 - Deadlock in ReplicationFeedMonitors
#170 - KeyDB dying via SIGABORT
#180 - crash after setting maxclients via cmd line
#169 - Write performance of a master dropped sharply when a slave is added

In addition untracked issues were resolved:

• Potential deadlock when entering futex sleep
• Pub/Sub Async messages may not be sent in a timely manner when load is low
• Excessive logging during failed RREPLAY
• KeyDB unresponsive handling clients on different threads during RDB load

Fixes for the following issues:
#155 Crash when a key has multiple sub expires and one is triggered
#154 Failure loading subkey expires
#153 "config get replicaof" returns a corrupt response
#150 Potential deadlock when failing to write to a client
#143 Transancations not always replicated correctly with multi master.

This release also contains additional logging for both deadlocks and replication errors.

Added new KEYDB.CRON command which will allow lua scripts to be scheduled to run in the future. Support is also added for ring topologies with multi-master.

Bug fixes:

• Fix failure to respect max clients configuration option (Issue #147)
• Fix compile issues on raspberry pi and older OS X machines (Issue #141 and #146)
• Fix crash propogating stale keys in active replication
• Fix issue #125
• Fix issue #137
• Fix issue #132

Fixes an issue where we can crash due to AOF rewrite.

Release notes:

• Subkey expires now works with hashes and sorted sets
• Performance fixes for fastlock (Issue #130)
• Fix issue where multimaster is able to run without active replication enabled
• Integration with KeyDB Pro

• Fixed issue where deletes are not merged after temporary disconnect of two or more Active Replicas. (Issue #83
• Fixed Issue #107 where expires are not always replicated with Active Replication
• Fixed deadlock in the assembly spinlock implementation
• Fixed race conditions in PUB/SUB which may cause a crash

This release fixes a few deadlocks than can hit in some rare situations:

• Potential deadlocks when feeding replicas and clients disconnect
• Potential deadlocks feeding monitors and clients disconnect
• Potential deadlock on client disconnection
• Intermittent crash when replica is disconnected

In addition deadlock detection is now added and will bring the server down immediately if one is detected. This will allow faster recovery in the event one is experienced.

Major Changes:

• Added support for expiring members of a set (EXPIREMEMBER, etc)
• Improved expiration algorithm to run in O(n) instead of O (n log n) time
• Reduced memory consumption of expires, up to 10% global memory savings.
• BITOP now supports LSHIFT and RSHIFT shift operators

For more details see our blog: https://docs.keydb.dev/blog/2019/10/20/blog-post/

This release fixes an issue where when under load we may incorrectly commit data to database 0 on the replica instead of the selected database.