lmdb-js

This release includes a significant overhaul of how get and query/range retrievals are performed to reduce the amount of native calls and the number of JS objects that have to be transmitted across the native calls. This also facilitates and accompanies adding support for V8's new fast-api-calls feature, which also improves performance. This can yield over a 50% performance improvement in get operations, and over twice the speed for iterating through ranges, with small payloads where deserialization is not the dominant cost. There are also very large improvements in the performance of count operations and offset handling as well.

Also related to these changes is switching (back) to serializing and deserializing keys in JS (instead of C++), which reduces the number of JS value/objects cross the native call barrier. This also opens the door for more customized JS key serialization strategies (negation, custom UUID handlers, and potentially more performant little endian format).

The latest point releases include performance improvements for accessing binary data. This should improve performance when using encoding: 'binary'. There are also now dedicated binary methods, getBinary/getBinaryFast that are available for getting binary data even if the store is using a different encoding.

There are now count functions (getCount, getKeysCount, getValuesCount) for faster access to getting a count of entries in a range.

ordered-binary is now available as an encoding option. This may be useful when create index stores that are primarily use to store references to keys in other stores (and this is often done in combination with the dupSort option).

Database resizing has now been completely internalized and done on the fly within LMDB itself, and does not require throwing any errors or restarting any transactions, which makes it more efficient, easy and deterministic to run transaction callbacks.
There are now new defaults and options for ordering asynchronous transactions and standalone operations.
Synchronous transactions now execute as a child transaction where possible (inside an existing transaction without writemaps or caching).

Changes to the default LMDB configuration to use the new remapping functionality by default, which provides a simpler approach to database growth without have to use resize events. Also, writemaps (with syncing) have been disabled by default in Windows since they are not syncing properly.

The major new addition for this release is support for asynchronous transactions and child transactions. This significantly expands the ability to define more complex transactions with arbitrary code, while still taking advantage of asynchronous, batched transaction handling with off-thread start and commit operations. In addition, there is also support for child transactions (which are also asynchronous by default), which allows for asynchronous transaction operations to have full abort/roll-back functionality. See the documentation for the new transactionAsync and childTransaction methods.

As part of these changes the transaction method is now deprecated (which runs a synchronous transaction), as it will be converted to running asynchronous transactions in the future. Also the batch thresholds have been removed as there are too many complications with trying to run an asynchronous batch synchronously.

This version includes improvements for checking reader locks, cleaning up stale reader locks. It also provides an option for whether or not to use a reader lock-based snapshot for iterators (can be disabled for long-running iterators).
This version includes more expansive TypeScript definitions.

This release includes more comprehensive support for interacting with duplicate key databases (which are enabled with dupSort flag):

• remove/removeSync now support specifying a value as the second argument for deleting individual entries that may share a duplicate key
• getValues(key) was added to iterate over all values of a given key
• getKeys(rangeOptions) was added to easily iterate over all keys in a range, without returning duplicate keys

Caching is now available, and can be enabled with the cache flag. This uses a weak-referenced based LRFU cache that can improve performance, guarantee key correlation with object identity, and provide immediate access to put values.

The getRange function for traversing over a range of keys now uses a true snapshot-based cursor to traverse the entire range. Previously, getRange would get blocks of 100 entries and reset the cursor between retrievals so it wasn't truly snapshot based, but now getRange should return a range of entries based on the exact snapshot/state of the database at the time when it was called.

We have also improved the TypeScript type definitions and added a test for them.

lmdb-store now includes type definitions for TypeScript. This release also imposes a maximum search for free-space in LMDB to better handle databases with large fragment free spaces.

lmdb-store now includes support for on-disk encryption. This is enabled by using the new encryptionKey option/property, and specifying a key to use for encryption. lmdb-store encrypts using the Chacha8 cipher, which provides a very high performance means of encrypting a database (this is the only type of encryption currently supported).

This major version update includes an update to LMDB 1.0, which has incompatible database format with LMDB 0.9.x. This release includes an automated upgrade when opening a LMDB 0.9.x database and attempts to upgrade the database to LMDB 1.0 (by copying all the records from the older format to newer format). This involves the use of the lmdb-store-0.9 package to handle reading from the old format. In the future, this package will not be included, and upgrade will require explicitly including that package.