Rocksdb Change Log

6.2.2 (6/7/2019)

Bug Fixes

• Disable dynamic extension support by default for CMake.

6.2.1 (6/4/2019)

Bug Fixes

• Fix flush's/compaction's merge processing logic which allowed Puts covered by range tombstones to reappear. Note Puts may exist even if the user only ever called Merge() due to an internal conversion during compaction to the bottommost level.

6.2.0 (4/30/2019)

New Features

• Add an option strict_bytes_per_sync that causes a file-writing thread to block rather than exceed the limit on bytes pending writeback specified by bytes_per_sync or wal_bytes_per_sync.
• Improve range scan performance by avoiding per-key upper bound check in BlockBasedTableIterator.
• Introduce Periodic Compaction for Level style compaction. Files are re-compacted periodically and put in the same level.
• Block-based table index now contains exact highest key in the file, rather than an upper bound. This may improve Get() and iterator Seek() performance in some situations, especially when direct IO is enabled and block cache is disabled. A setting BlockBasedTableOptions::index_shortening is introduced to control this behavior. Set it to kShortenSeparatorsAndSuccessor to get the old behavior.
• When reading from option file/string/map, customized envs can be filled according to object registry.
• Add an option snap_refresh_nanos (default to 0.5s) to periodically refresh the snapshot list in compaction jobs. Assign to 0 to disable the feature.
• Improve range scan performance when using explicit user readahead by not creating new table readers for every iterator.

Public API Change

• Change the behavior of OptimizeForPointLookup(): move away from hash-based block-based-table index, and use whole key memtable filtering.
• Change the behavior of OptimizeForSmallDb(): use a 16MB block cache, put index and filter blocks into it, and cost the memtable size to it. DBOptions.OptimizeForSmallDb() and ColumnFamilyOptions.OptimizeForSmallDb() start to take an optional cache object.
• Added BottommostLevelCompaction::kForceOptimized to avoid double compacting newly compacted files in the bottommost level compaction of manual compaction. Note this option may prohibit the manual compaction to produce a single file in the bottommost level.

Bug Fixes

• Adjust WriteBufferManager's dummy entry size to block cache from 1MB to 256KB.
• Fix a race condition between WritePrepared::Get and ::Put with duplicate keys.
• Fix crash when memtable prefix bloom is enabled and read/write a key out of domain of prefix extractor.
• Close a WAL file before another thread deletes it.
• Fix an assertion failure IsFlushPending() == true caused by one bg thread releasing the db mutex in ~ColumnFamilyData and another thread clearing flush_requested_ flag.

Rocksdb Change Log

6.1.2 (6/4/2019)

Bug Fixes

• Fix flush's/compaction's merge processing logic which allowed Puts covered by range tombstones to reappear. Note Puts may exist even if the user only ever called Merge() due to an internal conversion during compaction to the bottommost level.

Rocksdb Change Log

6.1.1 (4/9/2019)

New Features

• When reading from option file/string/map, customized comparators and/or merge operators can be filled according to object registry.

Public API Change

Bug Fixes

• Fix a bug in 2PC where a sequence of txn prepare, memtable flush, and crash could result in losing the prepared transaction.
• Fix a bug in Encryption Env which could cause encrypted files to be read beyond file boundaries.

6.1.0 (3/27/2019)

New Features

• Introduce two more stats levels, kExceptHistogramOrTimers and kExceptTimers.
• Added a feature to perform data-block sampling for compressibility, and report stats to user.
• Add support for trace filtering.
• Add DBOptions.avoid_unnecessary_blocking_io. If true, we avoid file deletion when destorying ColumnFamilyHandle and Iterator. Instead, a job is scheduled to delete the files in background.

Public API Change

• Remove bundled fbson library.
• statistics.stats_level_ becomes atomic. It is preferred to use statistics.set_stats_level() and statistics.get_stats_level() to access it.
• Introduce a new IOError subcode, PathNotFound, to indicate trying to open a nonexistent file or directory for read.
• Add initial support for multiple db instances sharing the same data in single-writer, multi-reader mode.
• Removed some "using std::xxx" from public headers.

Bug Fixes

• Fix JEMALLOC_CXX_THROW macro missing from older Jemalloc versions, causing build failures on some platforms.
• Fix SstFileReader not able to open file ingested with write_glbal_seqno=true.

6.0.2 (4/23/2019)

Bug Fixes needed for Java users

• Fix build failures due to missing JEMALLOC_CXX_THROW macro (#5053)
• Fix compilation on db_bench_tool.cc on Windows (#5227)

Rocksdb Change Log

6.0.1 (3/26/2019)

New Features

Public API Change

• Added many new features to the Java API to bring it closer to the C++ API.

Bug Fixes

• Make BlobDB wait for all background tasks on shutdown.
• Fixed a BlobDB issue where some trash files are not tracked causing them to remain forever.

6.0.0 (2/19/2019)

New Features

• Enabled checkpoint on readonly db (DBImplReadOnly).
• Make DB ignore dropped column families while committing results of atomic flush.
• RocksDB may choose to preopen some files even if options.max_open_files != -1. This may make DB open slightly longer.
• For users of dictionary compression with ZSTD v0.7.0+, we now reuse the same digested dictionary when compressing each of an SST file's data blocks for faster compression speeds.
• For all users of dictionary compression who set cache_index_and_filter_blocks == true, we now store dictionary data used for decompression in the block cache for better control over memory usage. For users of ZSTD v1.1.4+ who compile with -DZSTD_STATIC_LINKING_ONLY, this includes a digested dictionary, which is used to increase decompression speed.
• Add support for block checksums verification for external SST files before ingestion.
• Introduce stats history which periodically saves Statistics snapshots and added GetStatsHistory API to retrieve these snapshots.
• Add a place holder in manifest which indicate a record from future that can be safely ignored.
• Add support for trace sampling.
• Enable properties block checksum verification for block-based tables.
• For all users of dictionary compression, we now generate a separate dictionary for compressing each bottom-level SST file. Previously we reused a single dictionary for a whole compaction to bottom level. The new approach achieves better compression ratios; however, it uses more memory and CPU for buffering/sampling data blocks and training dictionaries.
• Add whole key bloom filter support in memtable.
• Files written by SstFileWriter will now use dictionary compression if it is configured in the file writer's CompressionOptions.

Public API Change

• Disallow CompactionFilter::IgnoreSnapshots() = false, because it is not very useful and the behavior is confusing. The filter will filter everything if there is no snapshot declared by the time the compaction starts. However, users can define a snapshot after the compaction starts and before it finishes and this new snapshot won't be repeatable, because after the compaction finishes, some keys may be dropped.
• CompactionPri = kMinOverlappingRatio also uses compensated file size, which boosts file with lots of tombstones to be compacted first.
• Transaction::GetForUpdate is extended with a do_validate parameter with default value of true. If false it skips validating the snapshot before doing the read. Similarly ::Merge, ::Put, ::Delete, and ::SingleDelete are extended with assume_tracked with default value of false. If true it indicates that call is assumed to be after a ::GetForUpdate.
• TableProperties::num_entries and TableProperties::num_deletions now also account for number of range tombstones.
• Remove geodb, spatial_db, document_db, json_document, date_tiered_db, and redis_lists.
• With "ldb ----try_load_options", when wal_dir specified by the option file doesn't exist, ignore it.
• Change time resolution in FileOperationInfo.
• Deleting Blob files also go through SStFileManager.
• Remove CuckooHash memtable.
• The counter stat number.block.not_compressed now also counts blocks not compressed due to poor compression ratio.
• Remove ttl option from CompactionOptionsFIFO. The option has been deprecated and ttl in ColumnFamilyOptions is used instead.
• Support SST file ingestion across multiple column families via DB::IngestExternalFiles. See the function's comment about atomicity.
• Remove Lua compaction filter.

Bug Fixes

• Fix a deadlock caused by compaction and file ingestion waiting for each other in the event of write stalls.
• Fix a memory leak when files with range tombstones are read in mmap mode and block cache is enabled
• Fix handling of corrupt range tombstone blocks such that corruptions cannot cause deleted keys to reappear
• Lock free MultiGet
• Fix incorrect NotFound point lookup result when querying the endpoint of a file that has been extended by a range tombstone.
• Fix with pipelined write, write leaders's callback failure lead to the whole write group fail.

Change Default Options

• Change options.compaction_pri's default to kMinOverlappingRatio

Rocksdb Change Log

5.18.3 (2/11/2019)

Bug Fixes

• Fix possible LSM corruption when both range deletions and subcompactions are used. The symptom of this corruption is L1+ files overlapping in the user key space.

5.18.2 (01/31/2019)

Public API Change

• Change time resolution in FileOperationInfo.
• Deleting Blob files also go through SStFileManager.

5.18.0 (11/30/2018)

New Features

• Introduced JemallocNodumpAllocator memory allocator. When being use, block cache will be excluded from core dump.
• Introduced PerfContextByLevel as part of PerfContext which allows storing perf context at each level. Also replaced __thread with thread_local keyword for perf_context. Added per-level perf context for bloom filter and Get query.
• With level_compaction_dynamic_level_bytes = true, level multiplier may be adjusted automatically when Level 0 to 1 compaction is lagged behind.
• Introduced DB option atomic_flush. If true, RocksDB supports flushing multiple column families and atomically committing the result to MANIFEST. Useful when WAL is disabled.
• Added num_deletions and num_merge_operands members to TableProperties.
• Added "rocksdb.min-obsolete-sst-number-to-keep" DB property that reports the lower bound on SST file numbers that are being kept from deletion, even if the SSTs are obsolete.
• Add xxhash64 checksum support
• Introduced MemoryAllocator, which lets the user specify custom memory allocator for block based table.
• Improved DeleteRange to prevent read performance degradation. The feature is no longer marked as experimental.
• Enabled checkpoint on readonly db (DBImplReadOnly).

Public API Change

• DBOptions::use_direct_reads now affects reads issued by BackupEngine on the database's SSTs.
• NO_ITERATORS is divided into two counters NO_ITERATOR_CREATED and NO_ITERATOR_DELETE. Both of them are only increasing now, just as other counters.

Bug Fixes

• Fix corner case where a write group leader blocked due to write stall blocks other writers in queue with WriteOptions::no_slowdown set.
• Fix in-memory range tombstone truncation to avoid erroneously covering newer keys at a lower level, and include range tombstones in compacted files whose largest key is the range tombstone's start key.
• Properly set the stop key for a truncated manual CompactRange
• Fix slow flush/compaction when DB contains many snapshots. The problem became noticeable to us in DBs with 100,000+ snapshots, though it will affect others at different thresholds.
• Fix the bug that WriteBatchWithIndex's SeekForPrev() doesn't see the entries with the same key.
• Fix the bug where user comparator was sometimes fed with InternalKey instead of the user key. The bug manifests when during GenerateBottommostFiles.
• Fix a bug in WritePrepared txns where if the number of old snapshots goes beyond the snapshot cache size (128 default) the rest will not be checked when evicting a commit entry from the commit cache.
• Fixed Get correctness bug in the presence of range tombstones where merge operands covered by a range tombstone always result in NotFound.
• Start populating NO_FILE_CLOSES ticker statistic, which was always zero previously.
• The default value of NewBloomFilterPolicy()'s argument use_block_based_builder is changed to false. Note that this new default may cause large temp memory usage when building very large SST files.
• Fix a deadlock caused by compaction and file ingestion waiting for each other in the event of write stalls.
• Make DB ignore dropped column families while committing results of atomic flush.

Rocksdb Change Log

5.17.2 (10/24/2018)

Bug Fixes

• Fix the bug that WriteBatchWithIndex's SeekForPrev() doesn't see the entries with the same key.

5.17.1 (10/16/2018)

Bug Fixes

• Fix slow flush/compaction when DB contains many snapshots. The problem became noticeable to us in DBs with 100,000+ snapshots, though it will affect others at different thresholds.
• Properly set the stop key for a truncated manual CompactRange
• Fix corner case where a write group leader blocked due to write stall blocks other writers in queue with WriteOptions::no_slowdown set.

New Features

• Introduced CacheAllocator, which lets the user specify custom allocator for memory in block cache.

5.17.0 (10/05/2018)

Public API Change

• OnTableFileCreated will now be called for empty files generated during compaction. In that case, TableFileCreationInfo::file_path will be "(nil)" and TableFileCreationInfo::file_size will be zero.
• Add FlushOptions::allow_write_stall, which controls whether Flush calls start working immediately, even if it causes user writes to stall, or will wait until flush can be performed without causing write stall (similar to CompactRangeOptions::allow_write_stall). Note that the default value is false, meaning we add delay to Flush calls until stalling can be avoided when possible. This is behavior change compared to previous RocksDB versions, where Flush calls didn't check if they might cause stall or not.
• Application using PessimisticTransactionDB is expected to rollback/commit recovered transactions before starting new ones. This assumption is used to skip concurrency control during recovery.

Rocksdb Change Log

5.16.6 (10/24/2018)

Bug Fixes

• Fix the bug that WriteBatchWithIndex's SeekForPrev() doesn't see the entries with the same key.

5.16.5 (10/16/2018)

Bug Fixes

• Fix slow flush/compaction when DB contains many snapshots. The problem became noticeable to us in DBs with 100,000+ snapshots, though it will affect others at different thresholds.
• Properly set the stop key for a truncated manual CompactRange

5.16.4 (10/10/2018)

Bug Fixes

• Fix corner case where a write group leader blocked due to write stall blocks other writers in queue with WriteOptions::no_slowdown set.

5.16.3 (10/1/2018)

Bug Fixes

• Fix crash caused when CompactFiles run with CompactionOptions::compression == CompressionType::kDisableCompressionOption. Now that setting causes the compression type to be chosen according to the column family-wide compression options.

5.16.2 (9/21/2018)

Bug Fixes

• Fix bug in partition filters with format_version=4.

5.16.1 (9/17/2018)

Bug Fixes

• Remove trace_analyzer_tool from rocksdb_lib target in TARGETS file.
• Fix RocksDB Java build and tests.
• Remove sync point in Block destructor.

5.16.0 (8/21/2018)

Public API Change

• OnTableFileCreated will now be called for empty files generated during compaction. In that case, TableFileCreationInfo::file_path will be "(nil)" and TableFileCreationInfo::file_size will be zero.

• Add FlushOptions::allow_write_stall, which controls whether Flush calls start working immediately, even if it causes user writes to stall, or will wait until flush can be performed without causing write stall (similar to CompactRangeOptions::allow_write_stall). Note that the default value is false, meaning we add delay to Flush calls until stalling can be avoided when possible. This is behavior change compared to previous RocksDB versions, where Flush calls didn't check if they might cause stall or not.

• The merge operands are passed to MergeOperator::ShouldMerge in the reversed order relative to how they were merged (passed to FullMerge or FullMergeV2) for performance reasons

• GetAllKeyVersions() to take an extra argument of max_num_ikeys.

New Features

• Changes the format of index blocks by delta encoding the index values, which are the block handles. This saves the encoding of BlockHandle::offset of the non-head index entries in each restart interval. The feature is backward compatible but not forward compatible. It is disabled by default unless format_version 4 or above is used.
• Add a new tool: trace_analyzer. Trace_analyzer analyzes the trace file generated by using trace_replay API. It can convert the binary format trace file to a human readable txt file, output the statistics of the analyzed query types such as access statistics and size statistics, combining the dumped whole key space file to analyze, support query correlation analyzing, and etc. Current supported query types are: Get, Put, Delete, SingleDelete, DeleteRange, Merge, Iterator (Seek, SeekForPrev only).
• Add hash index support to data blocks, which helps reducing the cpu utilization of point-lookup operations. This feature is backward compatible with the data block created without the hash index. It is disabled by default unless BlockBasedTableOptions::data_block_index_type is set to data_block_index_type = kDataBlockBinaryAndHash.

Bug Fixes

• Fix a bug in misreporting the estimated partition index size in properties block.
• Avoid creating empty SSTs and subsequently deleting them in certain cases during compaction.

Rocksdb Change Log

5.15.10 (9/13/2018)

Bug Fixes

• Fix RocksDB Java build and tests.

5.15.9 (9/4/2018)

Bug Fixes

• Fix compilation errors on OS X clang due to '-Wsuggest-override'.

5.15.8 (8/31/2018)

Bug Fixes

• Further avoid creating empty SSTs and subsequently deleting them during compaction.

5.15.7 (8/24/2018)

Bug Fixes

• Avoid creating empty SSTs and subsequently deleting them in certain cases during compaction.

5.15.6 (8/21/2018)

Public API Change

• The merge operands are passed to MergeOperator::ShouldMerge in the reversed order relative to how they were merged (passed to FullMerge or FullMergeV2) for performance reasons

5.15.5 (8/16/2018)

Bug Fixes

• Fix VerifyChecksum() API not preserving options

5.15.4 (8/11/2018)

Bug Fixes

• Fix a bug caused by not generating OnTableFileCreated() notification for a 0-byte SST.

5.15.3 (8/10/2018)

Bug Fixes

• Fix a bug in misreporting the estimated partition index size in properties block.

5.15.2 (8/9/2018)

Bug Fixes

• Return correct usable_size for BlockContents.

5.15.1 (8/1/2018)

Bug Fixes

• Prevent dereferencing invalid STL iterators when there are range tombstones in ingested files.

5.15.0 (7/17/2018)

Public API Change

• Remove managed iterator. ReadOptions.managed is not effective anymore.
• For bottommost_compression, a compatible CompressionOptions is added via bottommost_compression_opts. To keep backward compatible, a new boolean enabled is added to CompressionOptions. For compression_opts, it will be always used no matter what value of enabled is. For bottommost_compression_opts, it will only be used when user set enabled=true, otherwise, compression_opts will be used for bottommost_compression as default.
• With LRUCache, when high_pri_pool_ratio > 0, midpoint insertion strategy will be enabled to put low-pri items to the tail of low-pri list (the midpoint) when they first inserted into the cache. This is to make cache entries never get hit age out faster, improving cache efficiency when large background scan presents.
• For users of Statistics objects created via CreateDBStatistics(), the format of the string returned by its ToString() method has changed.
• The "rocksdb.num.entries" table property no longer counts range deletion tombstones as entries.

New Features

• Changes the format of index blocks by storing the key in their raw form rather than converting them to InternalKey. This saves 8 bytes per index key. The feature is backward compatbile but not forward compatible. It is disabled by default unless format_version 3 or above is used.
• Avoid memcpy when reading mmap files with OpenReadOnly and max_open_files==-1.
• Support dynamically changing ColumnFamilyOptions::ttl via SetOptions().
• Add a new table property, "rocksdb.num.range-deletions", which counts the number of range deletion tombstones in the table.
• Improve the performance of iterators doing long range scans by using readahead, when using direct IO.
• pin_top_level_index_and_filter (default true) in BlockBasedTableOptions can be used in combination with cache_index_and_filter_blocks to prefetch and pin the top-level index of partitioned index and filter blocks in cache. It has no impact when cache_index_and_filter_blocks is false.

Bug Fixes

• Fix deadlock with enable_pipelined_write=true and max_successive_merges > 0
• Check conflict at output level in CompactFiles.
• Fix corruption in non-iterator reads when mmap is used for file reads
• Fix bug with prefix search in partition filters where a shared prefix would be ignored from the later partitions. The bug could report an eixstent key as missing. The bug could be triggered if prefix_extractor is set and partition filters is enabled.
• Change default value of bytes_max_delete_chunk to 0 in NewSstFileManager() as it doesn't work well with checkpoints.
• Fix a bug caused by not copying the block trailer with compressed SST file, direct IO, prefetcher and no compressed block cache.
• Fix write can stuck indefinitely if enable_pipelined_write=true. The issue exists since pipelined write was introduced in 5.5.0.

5.14.3 (8/21/2018)

Public API Change

• The merge operands are passed to MergeOperator::ShouldMerge in the reversed order relative to how they were merged (passed to FullMerge or FullMergeV2) for performance reasons

Bug Fixes

• Fixes DBImpl::FindObsoleteFiles() calling GetChildren() on the same path

5.14.3 (8/21/2018)

Public API Change

• The merge operands are passed to MergeOperator::ShouldMerge in the reversed order relative to how they were merged (passed to FullMerge or FullMergeV2) for performance reasons

Bug Fixes

• Fixes DBImpl::FindObsoleteFiles() calling GetChildren() on the same path

5.14.2 (7/3/2018)

Bug Fixes

• Change default value of bytes_max_delete_chunk to 0 in NewSstFileManager() as it doesn't work well with checkpoints.
• Set DEBUG_LEVEL=0 for RocksJava Mac Release build.

5.14.1 (6/20/2018)

Bug Fixes

• Fix block-based table reader pinning blocks throughout its lifetime, causing memory usage increase.
• Fix bug with prefix search in partition filters where a shared prefix would be ignored from the later partitions. The bug could report an eixstent key as missing. The bug could be triggered if prefix_extractor is set and partition filters is enabled.

5.14.0 (5/16/2018)

Public API Change

• Add a BlockBasedTableOption to align uncompressed data blocks on the smaller of block size or page size boundary, to reduce flash reads by avoiding reads spanning 4K pages.
• The background thread naming convention changed (on supporting platforms) to "rocksdb:", e.g., "rocksdb:low0".
• Add a new ticker stat rocksdb.number.multiget.keys.found to count number of keys successfully read in MultiGet calls
• Touch-up to write-related counters in PerfContext. New counters added: write_scheduling_flushes_compactions_time, write_thread_wait_nanos. Counters whose behavior was fixed or modified: write_memtable_time, write_pre_and_post_process_time, write_delay_time.
• Posix Env's NewRandomRWFile() will fail if the file doesn't exist.
• Now, DBOptions::use_direct_io_for_flush_and_compaction only applies to background writes, and DBOptions::use_direct_reads applies to both user reads and background reads. This conforms with Linux's open(2) manpage, which advises against simultaneously reading a file in buffered and direct modes, due to possibly undefined behavior and degraded performance.
• Iterator::Valid() always returns false if !status().ok(). So, now when doing a Seek() followed by some Next()s, there's no need to check status() after every operation.
• Iterator::Seek()/SeekForPrev()/SeekToFirst()/SeekToLast() always resets status().

New Features

• Introduce TTL for level compaction so that all files older than ttl go through the compaction process to get rid of old data.
• TransactionDBOptions::write_policy can be configured to enable WritePrepared 2PC transactions. Read more about them in the wiki.
• Add DB properties "rocksdb.block-cache-capacity", "rocksdb.block-cache-usage", "rocksdb.block-cache-pinned-usage" to show block cache usage.
• Add Env::LowerThreadPoolCPUPriority(Priority) method, which lowers the CPU priority of background (esp. compaction) threads to minimize interference with foreground tasks.
• Fsync parent directory after deleting a file in delete scheduler.
• In level-based compaction, if bottom-pri thread pool was setup via Env::SetBackgroundThreads(), compactions to the bottom level will be delegated to that thread pool.

Bug Fixes

• Fsync after writing global seq number to the ingestion file in ExternalSstFileIngestionJob.
• Fix WAL corruption caused by race condition between user write thread and FlushWAL when two_write_queue is not set.
• Fix BackupableDBOptions::max_valid_backups_to_open to not delete backup files when refcount cannot be accurately determined.
• Fix memory leak when pin_l0_filter_and_index_blocks_in_cache is used with partitioned filters
• Disable rollback of merge operands in WritePrepared transactions to work around an issue in MyRocks. It can be enabled back by setting TransactionDBOptions::rollback_merge_operands to true.
• Fix bug with prefix search in partition filters where a shared prefix would be ignored from the later partitions. The bug could report an eixstent key as missing. The bug could be triggered if prefix_extractor is set and partition filters is enabled.

Java API Changes

• Add BlockBasedTableConfig.setBlockCache to allow sharing a block cache across DB instances.
• Added SstFileManager to the Java API to allow managing SST files across DB instances.

Bug Fixes

• Fix regression bug of Prev() with ReadOptions.iterate_upper_bound.

Bug Fixes

• Fix regression bug of Prev() with ReadOptions.iterate_upper_bound.

5.13.3 (6/6/2018)

Bug Fixes

• Fix assertion when reading bloom filter of SST files containing range deletions but no data

5.13.2 (5/25/2018)

Public API Change

• Introduced CompressionOptions::kDefaultCompressionLevel, which is a generic way to tell RocksDB to use the compression library's default level. It is now the default value for CompressionOptions::level. Previously the level defaulted to -1, which gave poor compression ratios in ZSTD.

Bug Fixes

• Fix segfault caused by object premature destruction (PR #3898)
• Fix an issue with unnecessary capture in lambda expressions (PR #3904)

5.13.1 (4/30/2018)

New Features

• Add Env::LowerThreadPoolCPUPriority(Priority) method, which lowers the CPU priority of background (esp. compaction) threads to minimize interference with foreground tasks.
• Eliminate use of temporary directories in BackupEngine to improve reliability on distributed file systems.

5.13.0 (3/20/2018)

Public API Change

• RocksDBOptionsParser::Parse()'s ignore_unknown_options argument will only be effective if the option file shows it is generated using a higher version of RocksDB than the current version.
• Remove CompactionEventListener.

New Features

• SstFileManager now can cancel compactions if they will result in max space errors. SstFileManager users can also use SetCompactionBufferSize to specify how much space must be leftover during a compaction for auxiliary file functions such as logging and flushing.
• Avoid unnecessarily flushing in CompactRange() when the range specified by the user does not overlap unflushed memtables.
• If ColumnFamilyOptions::max_subcompactions is set greater than one, we now parallelize large manual level-based compactions.
• Add "rocksdb.live-sst-files-size" DB property to return total bytes of all SST files belong to the latest LSM tree.

Bug Fixes

• Fix a leak in prepared_section_completed_ where the zeroed entries would not removed from the map.
• Fix WAL corruption caused by race condition between user write thread and backup/checkpoint thread.
• Fsync after writing global seq number to the ingestion file in ExternalSstFileIngestionJob.
• Fix memory leak when pin_l0_filter_and_index_blocks_in_cache is used with partitioned filters

Java API Changes

• Add BlockBasedTableConfig.setBlockCache to allow sharing a block cache across DB instances.

5.12.4 (4/23/2018)

Bug Fixes

• Fix memory leak in two_level_iterator

5.12.3 (4/23/2018)

Bug Fixes

• Fix memory leak when pin_l0_filter_and_index_blocks_in_cache is used with partitioned filters

5.12.2 (3/23/2018)

Bug Fixes

• Fsync after writing global seq number to the ingestion file in ExternalSstFileIngestionJob.

Java API Changes

• Add BlockBasedTableConfig.setBlockCache to allow sharing a block cache across DB instances.

5.12.1 (3/16/2018)

Public API Change

• RocksDBOptionsParser::Parse()'s ignore_unknown_options argument will only be effective if the option file shows it is generated using a higher version of RocksDB than the current version.

New Features

• Avoid unnecessarily flushing in CompactRange() when the range specified by the user does not overlap unflushed memtables.

Bug Fixes

• Fix WAL corruption caused by race condition between user write thread and backup/checkpoint thread.

5.12.0 (2/14/2018)

Public API Change

• Iterator::SeekForPrev is now a pure virtual method. This is to prevent user who implement the Iterator interface fail to implement SeekForPrev by mistake.
• Add include_end option to make the range end exclusive when include_end == false in DeleteFilesInRange().
• Add CompactRangeOptions::allow_write_stall, which makes CompactRange start working immediately, even if it causes user writes to stall. The default value is false, meaning we add delay to CompactRange calls until stalling can be avoided when possible. Note this delay is not present in previous RocksDB versions.
• Creating checkpoint with empty directory now returns Status::InvalidArgument; previously, it returned Status::IOError.
• Adds a BlockBasedTableOption to turn off index block compression.
• Close() method now returns a status when closing a db.

New Features

• Improve the performance of iterators doing long range scans by using readahead.
• Add new function DeleteFilesInRanges() to delete files in multiple ranges at once for better performance.
• FreeBSD build support for RocksDB and RocksJava.
• Improved performance of long range scans with readahead.
• Updated to and now continuously tested in Visual Studio 2017.

Bug Fixes

• Fix DisableFileDeletions() followed by GetSortedWalFiles() to not return obsolete WAL files that PurgeObsoleteFiles() is going to delete.
• Fix Handle error return from WriteBuffer() during WAL file close and DB close.
• Fix advance reservation of arena block addresses.
• Fix handling of empty string as checkpoint directory.

5.12.2 (3/23/2018)

Bug Fixes

• Fsync after writing global seq number to the ingestion file in ExternalSstFileIngestionJob.

Java API Changes

• Add BlockBasedTableConfig.setBlockCache to allow sharing a block cache across DB instances.

5.12.1 (3/16/2018)

Public API Change

• RocksDBOptionsParser::Parse()'s ignore_unknown_options argument will only be effective if the option file shows it is generated using a higher version of RocksDB than the current version.

New Features

• Avoid unnecessarily flushing in CompactRange() when the range specified by the user does not overlap unflushed memtables.

Bug Fixes

• Fix WAL corruption caused by race condition between user write thread and backup/checkpoint thread.

5.12.0 (2/14/2018)

Public API Change

• Iterator::SeekForPrev is now a pure virtual method. This is to prevent user who implement the Iterator interface fail to implement SeekForPrev by mistake.
• Add include_end option to make the range end exclusive when include_end == false in DeleteFilesInRange().
• Add CompactRangeOptions::allow_write_stall, which makes CompactRange start working immediately, even if it causes user writes to stall. The default value is false, meaning we add delay to CompactRange calls until stalling can be avoided when possible. Note this delay is not present in previous RocksDB versions.
• Creating checkpoint with empty directory now returns Status::InvalidArgument; previously, it returned Status::IOError.
• Adds a BlockBasedTableOption to turn off index block compression.
• Close() method now returns a status when closing a db.

New Features

• Improve the performance of iterators doing long range scans by using readahead.
• Add new function DeleteFilesInRanges() to delete files in multiple ranges at once for better performance.
• FreeBSD build support for RocksDB and RocksJava.
• Improved performance of long range scans with readahead.
• Updated to and now continuously tested in Visual Studio 2017.

Bug Fixes

• Fix DisableFileDeletions() followed by GetSortedWalFiles() to not return obsolete WAL files that PurgeObsoleteFiles() is going to delete.
• Fix Handle error return from WriteBuffer() during WAL file close and DB close.
• Fix advance reservation of arena block addresses.
• Fix handling of empty string as checkpoint directory.

5.11.2 (02/24/2018)

Bug Fixes

• Fix bug in iterator readahead causing blocks to incorrectly be considered truncated (corrupted).

5.11.1 (02/22/2018)

New Features

• Follow rsync-style naming convention for BackupEngine tempfiles. This enables some optimizations when run on GlusterFS.

5.11.0 (01/08/2018)

Public API Change

• Add autoTune and getBytesPerSecond() to RocksJava RateLimiter

New Features

• Add a new histogram stat called rocksdb.db.flush.micros for memtable flush.
• Add "--use_txn" option to use transactional API in db_stress.
• Disable onboard cache for compaction output in Windows platform.
• Improve the performance of iterators doing long range scans by using readahead.

Bug Fixes

• Fix a stack-use-after-scope bug in ForwardIterator.
• Fix builds on platforms including Linux, Windows, and PowerPC.
• Fix buffer overrun in backup engine for DBs with huge number of files.
• Fix a mislabel bug for bottom-pri compaction threads.
• Fix DB::Flush() keep waiting after flush finish under certain condition.