rocksdb

5.4.7 release.

5.4.6 release.

Bug Fixes

• Fix db_bench with lite build.

Public API Change

• Support dynamically changing stats_dump_period_sec option via SetDBOptions().
• Added ReadOptions::max_skippable_internal_keys to set a threshold to fail a request as incomplete when too many keys are being skipped while using iterators.
• DB::Get in place of std::string accepts PinnableSlice, which avoids the extra memcpy of value to std::string in most of cases.
  • PinnableSlice releases the pinned resources that contain the value when it is destructed or when ::Reset() is called on it.
  • The old API that accepts std::string, although discouraged, is still supported.
• Replace Options::use_direct_writes with Options::use_direct_io_for_flush_and_compaction. See Direct IO wiki for details.

New Features

• Memtable flush can be avoided during checkpoint creation if total log file size is smaller than a threshold specified by the user.
• Introduce level-based L0->L0 compactions to reduce file count, so write delays are incurred less often.
• (Experimental) Partitioning filters which creates an index on the partitions. The feature can be enabled by setting partition_filters when using kFullFilter. Currently the feature also requires two-level indexing to be enabled. Number of partitions is the same as the number of partitions for indexes, which is controlled by metadata_block_size.
• DB::ResetStats() to reset internal stats.
• Added CompactionEventListener and EventListener::OnFlushBegin interfaces.
• Added DB::CreateColumnFamilie() and DB::DropColumnFamilies() to bulk create/drop column families.
• Facility for cross-building RocksJava using Docker.

Bug Fixes

• Fix WriteBatchWithIndex address use after scope error.
• Fix WritableFile buffer size in direct IO.
• Add prefetch to PosixRandomAccessFile in buffered io.
• Fix PinnableSlice access invalid address when row cache is enabled.
• Fix huge fallocate calls fail and make XFS unhappy.
• Fix memory alignment with logical sector size.
• Fix alignment in ReadaheadRandomAccessFile.
• Fix bias with read amplification stats (READ_AMP_ESTIMATE_USEFUL_BYTES and READ_AMP_TOTAL_READ_BYTES).
• Fix a manual / auto compaction data race.
• Fix CentOS 5 cross-building of RocksJava.
• Build and link with ZStd when creating the static RocksJava build.
• Fix snprintf's usage to be cross-platform.
• Fix build errors with blob DB.
• Fix readamp test type inconsistency.

5.4.5 release.

• Fix RocksJava static build issue.
• Fix RocksJava build for IBM/PPC.

• Fix RocksJava static build issue.
• Fix RocksJava build for IBM/PPC.

• Fix Centos5 cross-building of RocksJava
• Build and link ZStd statically in RocksJava
• Facility for cross-building RocksJava using Docker.

• Fix Centos5 cross-building of RocksJava
• Build and link ZStd statically in RocksJava
• Facility for cross-building RocksJava using Docker.

Public API Change

• Remove disableDataSync option.
• Remove timeout_hint_us option from WriteOptions. The option has been deprecated and has no effect since 3.13.0.
• Remove option min_partial_merge_operands. Partial merge operands will always be merged in flush or compaction if there are more than one.
• Remove option verify_checksums_in_compaction. Compaction will always verify checksum.

New Features

• Memtable flush can be avoided during checkpoint creation if total log file size is smaller than a threshold specified by the user.

Bug Fixes

• Fix the bug that iterator may skip keys

Public API Change

• NewLRUCache() will determine number of shard bits automatically based on capacity, if the user doesn't pass one. This also impacts the default block cache when the user doesn't explict provide one.
• Change the default of delayed slowdown value to 16MB/s and further increase the L0 stop condition to 36 files.
• (Experimental) Two-level indexing that partition the index and creates a 2nd level index on the partitions. The feature can be enabled by setting kTwoLevelIndexSearch as IndexType and configuring index_per_partition.

New Features

• Added new overloaded function GetApproximateSizes that allows to specify if memtable stats should be computed only without computing SST files' stats approximations.
• Added new function GetApproximateMemTableStats that approximates both number of records and size of memtables.

Bug Fixes

• RangeSync() should work if ROCKSDB_FALLOCATE_PRESENT is not set
• Fix wrong results in a data race case in Get()
• Some fixes related to 2PC.
• Fix several bugs in Direct I/O supports.
• Fix a regression bug which can cause Seek() to miss some keys if the return key has been updated many times after the snapshot which is used by the iterator.

Bug fixes

• Fixed the bug of IO Error when compaction_readahead_size > 0 in direct I/O
• Fixed the bug of file size mismatch between MANIFEST and real size in direct I/O
• Fix OSX build break after the fallocate change

Public API Change

• Support dynamically change delete_obsolete_files_period_micros option via SetDBOptions().
• Added EventListener::OnExternalFileIngested which will be called when IngestExternalFile() add a file successfully.
• BackupEngine::Open and BackupEngineReadOnly::Open now always return error statuses matching those of the backup Env.

Bug Fixes

• Fix the bug that if 2PC is enabled, checkpoints may loss some recent transactions.
• When file copying is needed when creating checkpoints or bulk loading files, fsync the file after the file copying.

Bug fixes

• Fixed DeleteRange corruption issue in certain compaction cases

Public API Change

• Options::max_bytes_for_level_multiplier is now a double along with all getters and setters.
• Support dynamically change delayed_write_rate and max_total_wal_size options via SetDBOptions().
• Introduce DB::DeleteRange for optimized deletion of large ranges of contiguous keys.
• Support dynamically change delayed_write_rate option via SetDBOptions().
• Options::allow_concurrent_memtable_write and Options::enable_write_thread_adaptive_yield are now true by default.
• Remove Tickers::SEQUENCE_NUMBER to avoid confusion if statistics object is shared among RocksDB instance. Alternatively DB::GetLatestSequenceNumber() can be used to get the same value.
• Options.level0_stop_writes_trigger default value changes from 24 to 32.
• New compaction filter API: CompactionFilter::FilterV2(). Allows to drop ranges of keys.
• Removed flashcache support.
• DB::AddFile() is deprecated and is replaced with DB::IngestExternalFile(). DB::IngestExternalFile() remove all the restrictions that existed for DB::AddFile.

New Features

• Add avoid_flush_during_shutdown option, which speeds up DB shutdown by not flushing unpersisted data (i.e. with disableWAL = true). Unpersisted data will be lost. The options is dynamically changeable via SetDBOptions().
• Add memtable_insert_with_hint_prefix_extractor option. The option is mean to reduce CPU usage for inserting keys into memtable, if keys can be group by prefix and insert for each prefix are sequential or almost sequential. See include/rocksdb/options.h for more details.
• Add LuaCompactionFilter in utilities. This allows developers to write compaction filters in Lua. To use this feature, LUA_PATH needs to be set to the root directory of Lua.
• No longer populate "LATEST_BACKUP" file in backup directory, which formerly contained the number of the latest backup. The latest backup can be determined by finding the highest numbered file in the "meta/" subdirectory.

We have skipped releasing 4.12.x and went directly to 4.13.5

4.13.5

Public API Change

• Fix a regression in compaction performance.
• Disallow calling IngestExternalFile() on a dropped column family.
• Add EventListener::OnExternalFileIngested() event that will be called for files that are successfully ingested.

4.13.4

Public API Change

• Removed flashcache support.

4.13.0 (10/18/2016)

Public API Change

• DB::GetOptions() reflect dynamic changed options (i.e. through DB::SetOptions()) and return copy of options instead of reference.
• Added Statistics::getAndResetTickerCount().

New Features

• Add DB::SetDBOptions() to dynamic change base_background_compactions and max_background_compactions.
• Added Iterator::SeekForPrev(). This new API will seek to the last key that is less than or equal to the target key.

4.12.0 (9/12/2016)

Public API Change

• CancelAllBackgroundWork() flushes all memtables for databases containing writes that have bypassed the WAL (writes issued with WriteOptions::disableWAL=true) before shutting down background threads.
• Merge options source_compaction_factor, max_grandparent_overlap_bytes and expanded_compaction_factor into max_compaction_bytes.
• Remove ImmutableCFOptions.
• Add a compression type ZSTD, which can work with ZSTD 0.8.0 or up. Still keep ZSTDNotFinal for compatibility reasons.

New Features

• Introduce NewClockCache, which is based on CLOCK algorithm with better concurrent performance in some cases. It can be used to replace the default LRU-based block cache and table cache. To use it, RocksDB need to be linked with TBB lib.
• Change ticker/histogram statistics implementations to accumulate data in thread-local storage, which improves CPU performance by reducing cache coherency costs. Callers of CreateDBStatistics do not need to change anything to use this feature.
• Block cache mid-point insertion, where index and filter block are inserted into LRU block cache with higher priority. The feature can be enabled by setting BlockBasedTableOptions::cache_index_and_filter_blocks_with_high_priority to true and high_pri_pool_ratio > 0 when creating NewLRUCache.