Fixes #1760: ReadonlyError: Write transactions not allowed within liveQueries

Bug fixes

• dexie-cloud-addon: Behave when resetting database without reloading page. c3705a784d543798eef2a05c686369c8e699065a

• dexie: Fix #1736 Optimistic cache not purged on database deletion

General bugfixes

• #1740 Where-clauses with object-syntax fail to reuse virtual indexes - (there's a corresponding fix in the 3.x track)

Bugfixes for cache and optimistic updates

• Unhandled AbortErrors in console log (#1743)
• Improvements of optimistic updates (#1733)
• Move cache-middleware to level 0 (#1744)

Thanks to everyone that tried out 4.0.1-alpha.17 and found the issues that are now fixed!

This release replaces 4.0.1-alpha.12, 4.0.1-alpha.13 and 4.0.1-alpha.14 due to issues #1730, #1731 and a few other issues fixed in PR #1733

Cache and Optimistic Updates (v0.1)

The biggest news with this release is the initial support for common cache for live queries that enables multiple liveQuery observables targeting the same set of queries to reuse a single request towards indexedDB and to be immediately triggered as soon as data is being mutated. In practice, this will become a big game changer for large apps that has many ongoing liveQueries going on targeting large amount of data and possibly targeting the same data from different components.

Before the flow was pretty simple but could take some time from a mutation until a change was visible in the app if the app had many components using liveQuery and where the results of the queries were large.

Flow prior to this relase:

1. All live queries for the app is run initially (and the components are rendered).
2. User does som action so that data is being updated (such as db.friends.update(1, {age: 23}).
3. Write-transaction is committed --> onstoragemutated event is broadcasted to all tabs and workers
4. liveQueries that has touched the affected ranges are re-executed. If we have 10 components with affected liveQueries, all 10 of them will simultanously query IndexedDB for their data again.

Flow with this release:

1. All live queries for the app is run initially their promises are immediately cached in memory so that several liveQueries requesting the same data will await the same single promise instead of each one of them requesting indexedDB. When result arrives, the result is kept in the cache as long as there are components subscribing to the query.
2. User does som action so that data is being updated (such as db.friends.update(1, {age: 23}). The cache is immediately recomputed by applying the mutation to the previous result and triggering subscribers immediately to rerender.
3. Mutation promise resolves or reject. If it rejects, the optimistic update is rolled back and components rerender.
4. Transaction commits or rejects. If it rejects, the optimistic updates are rolled back and components rerender. If it was successful, components doesn't need to requery indexedDB because they already have the changes in the cache.
5. All mutations are still broadcasted to other tabs and workers. When a mutation comes from another tab or worker, it will not be optimistic but require a re-execution of the query.

So basically, liveQueries will only request IndexedDB at startup and two identical queries will only result in a single query towards IndexedDB.

Now this is the initial version of this and only certain queries can utilize this:

• If your liveQuery callback explicitely does db.transaction('r', ...), the user expects isolation, so optimistic updates won't be used.
• Only simple range queries are supported in this initial version: equals, above, aboveOrEqual, below, belowOrEqual, between, startsWith.
• Only tables with inbound keys are supported.
• Only toArray() queries (not .keys(), .primaryKeys() or .count())
• Not offset based queries.
• Not ignoreCase queries
• Not anyOf or inAnyRange

Example of optimised queries:

• db.friends.toArray()
• db.friends.where('age').between(18,65, true, true).limit(10).toArray()

Example of non-optimized queries in this initial version:

• db.friends.primaryKeys()
• db.friends.count()
• db.friends.offset(25).toArray()
• db.transaction('r', db.friends, () => db.friends.toArray())

PRs:

• #1718 Optimistic updates
• #1727 Fix optimistic updates

Other changes:

• Dexie will throw if trying to mutate data from a liveQuery callback ( liveQuery(()=>db.friends.add({...})) )
• A new constructor option to Dexie is {cache: 'immutable' | 'cloned' | 'disabled'}. The default is 'cloned' but 'immutable' would give better performance because the cached results can be returned directly to the caller without having to deep clone it. However, the data will be frozen so if the liveQuery callback would try to set properties or call mutating array operations on the results, it will fail to do so if having { cache: 'immutable' }. array.sort() is one example of a commonly used mutable operation. Workaround is to use array.slice().sort() instead.

Other PRs in this release

• #1711 Add .d.mts files to support import types properly.
• #1716 Remove custom unhandledrejection event propagation

This release is replaced by 4.0.1-alpha.17

This release is replaced by 4.0.1-alpha.17

Fixes in this maintenance release:

• #1728 Make the optimisation in [email protected] work with [email protected]. In the latest version of dexie-react-hooks (version 1.1.5), live queries are optimised to not fire of unnecessary on render. However, this optimisation is dependant on that the observable returned from liveQuery has a method revealing whether it may be resolved synchronously or not (hasValue()). The optimisation requires dexie@>=3.2.4 or [email protected] together with [email protected].
• #1677 Add //# sourceMappingUrl= to minified files. Fixes #326.
• #1712 Add .d.mts files to support import types properly in preparation for future typescript release where this might be needed.

Better SSR support for NextJS and SvelteKit

This release (and related dexie-react-hooks release) comes with better behavior of liveQuery(). useLiveQuery() and useObservable() in SSR environments, as it makes sure that calling these from a node runtime without any indexedDB envirnomnent will result in a no-op. The result: Use these tools the same in Next.js as in vanilla React and in SvelteKIT as in vanilla Svelte - no need to check for SSR in application code anymore and no need to dynamically imported components doing dexie queries.

We also fixed the typings of our Observables returned from liveQuery() to be type-compatible with Svelte Readables (this was an issue when consuming liveQuery() results in Svelte using typescript).

Svelte users should install this prerelase of dexie instead of the stable version, but NextJS users may stay on the stable version of dexie if they prefer, and just upgrade dexie-react-hooks.

Full Next.js Support

• Fixed more node errors that prevented nextjs SSR https://github.com/dexie/Dexie.js/pull/1675/commits/19cdf0b3a45d9d386edbfd331d07037246d63d82
• Better nextjs support for useLiveQuery(): https://github.com/dexie/Dexie.js/pull/1675/commits/39484a3c19355595b2b57bd042dbe22b55f97301
• Allow normal import of dexie-cloud-addon in nextjs: https://github.com/dexie/Dexie.js/pull/1675/commits/e18ee07c5b248143d4936b491a41b6d6679bcddd

Full SvelteKit Support

• #1691 Sveltekit fixes

Other fixes

• Fix transaction typings #1685

We've also released...

[email protected]

• Better nextjs support for useLiveQuery(): https://github.com/dexie/Dexie.js/pull/1675/commits/39484a3c19355595b2b57bd042dbe22b55f97301

[email protected]

• move rxjs deps out from peerDependencies and into ordinary dependencies
• #1675 Dexie Cloud and NextJS improvements
• #1691 Sveltekit fixes

[email protected]

• progressCallback Problem? #1678

Minor fix

Append sourceMappingUrl in minified files

This fix makes the all JS files in the dist folder have the source mapping file pointed out so that bundlers can generate a correct final map file for the application or library that bundles dexie into it.

If dexie was used without bundling it into the app, this change will have no benefit as chrome devtools is still able to locate the map files without the mapfile comment-line.

Typings fix for Table.update(), Table.bulkUpdate() and Collection.modify()

In Dexie 4.0.1-alpha.6, Table.bulkUpdate() was introduced as well as improving the typings of Table.update(), Collection.modify() to using typescript template literals that would correcltly type the changes argument and provide a nice code completion. However, there was a typings bug in this release that made the typings unusable for updating nested properties.

This version fixes the typings of Table.update(), Table.bulkUpdate() and Collection.modify() so that they work according to the expected format.

Requires Typescript 4.8 or later

The typings in this release requires Typescript 4.8 or later in order to accept numeric keypaths and allow updating individual array items.

The UpdateSpec type

A new type UpdateSpec<T> is expected as the second argument to Table.update(). This type can also be imported from dexie when the library user need to build the updateSpec using custom code before finally send along to Table.update() or in an array keysAndChanges to Table.bulkUpdate().

import type { UpdateSpec } from 'dexie';

interface Contact {
  name: string;
  address: Address;
}

interface Address {
  city: string;
  street: string;
  streetNo: number;
}

let updateSpec: UpdateSpec<Contact> = {};

updateSpec["address.streetNo"] = 44;

db.contacts.update(1, updateSpec);

Bugfixes:

• Node.js process not exiting when Dexie.js is imported

This was fixed for 4.x but with this release it is also fixed in the official latest stable version of dexie.

News:

• Table.bulkUpdate()

Typings:

• Revert to dexie 3 style Table<T, TKey>
• Updated Collection.filter type (PR 1658)
• feat(d.ts): creating hook subscriber return value

dexie-cloud addon (4.0.1-beta.26)

• Check BigInt availability without getting errors
• Bugfix when Buffer is polyfilled by webpack
• Imported rx operators wrong
• Bugfix default GUI fail if the JS is included in
• Bugfix "shopping card" use case: Failed to sync local data after logging in user
  
• Bugfix synk of ArrayBuffer data failed.

Other addons

• Release version-bumps of dexie-observable, dexie-syncable, dexie-export-import with updated deps to [email protected].

Misc

• Stop testing on IE11
• Updated browser test matrix
• Converted to pnpm
• Removed some still-remaing uses of indexedDB.cmp() and make use of the faster js version everywhere
• Removed link to feathub, as that app is gone
• Tests: Avoid Safari issue for IDBObjectStore.put()
• Vue sample: fix: lint error
• Vue sample: refactor: use vue composition api script setup syntax and vite
• Removing the "engines" field

• Fix for #1576 Node.js process not exiting when Dexie.js is imported
• PR #1559 Support for FileSystemDirectoryHandle when cloning

Security fix

Prohibit possible prototype pollution in Dexie.setByKeyPath() (https://github.com/dexie/Dexie.js/commit/1d655a69b9f28c3af6fae10cf5c61df387dc689b)

Bugfix

Fix #1473 Cannot use Dexie in react-native

A corresponding release 3.2.2 contains the same fixes for 3.x.

Security fix

Prohibit possible prototype pollution in Dexie.setByKeyPath() (https://github.com/dexie/Dexie.js/commit/1d655a69b9f28c3af6fae10cf5c61df387dc689b)

Bugfix

Fix #1473 Cannot use Dexie in react-native

A corresponding release 4.0.0-alpha.3 contains the same fixes for 4.x.

• Workaround for issue #613: Automatically reopen IndexedDB connection in case it was unexpectedly closed, and redo the operation. When a transaction couldn't be created due to invalid state, Dexie will reopen the IndexedDB connection and retry creating the transaction.
• Resolves #1439 and #1369 by extending the "exports" field to include "require" compliant version of dexie.

Should resolve #1439 and #1369 by extending the "exports" field to include "require" compliant version of dexie.

The initial release on 4.0. Currently pretty much identical to 3.2.1+ but with:

• More precise typings for Collection.modify() and Table.update() using template literal types for keyPaths.
• Typings: Table.add() and Table.put() won't require methods to be present on the object to add. Allows for adding POJO objects rather than having to construct the full class to add a row.
• Typings of Table generic: A property name can be used as 2nd generic argument (TKey) rather than the direct type. This will infer the key type from the type of that property and make it optional in Table.add() and Table.put() (see sample below)
• New class exported class Entity that can be optionally used as a base class in order to resolve dependency issues when mapping tables to classes. A subclass of Entity has a private property this.db which is the Dexie instance it originates from. Entity subclasses are not meant to be constructed by application code.

  
  export class Friend extends Entity<FriendsDB> {
    id!: number;
    name!: string;
    age!: number;
  
    birthday() {
      return this.db.friends.update(this.id, friend => ++friend.age);
    }
  }
  
  export class FriendsDB extends Dexie {
    friends!: Table<Friend, 'id'>;
  
    constructor() {
      super("FriendsDB");
      this.version(1).stores({
        friends: '++id, name, age'
      });
      this.friends.mapToClass(Friend);
    }
  }
  
  

Contains a workaround for Chrome issue #613. Needs to be tested in the field a while before we can release this publicly.

Dexie.js has become Reactive

After one year in alpha, beta and RC, Dexie.js with liveQuery() is now officially released. The main reason for this new feature is better integration with frontend libraries like React, Svelte, Vue and Angular.

Together with this release, the website https://dexie.org also got a face lift with tutorials for React, Svelte, Vue and Angular.

dexie.org

Take a look past the updated website. Old tutorials are replaced with modern relevant framework specific ones. We've added React, Svelte, Vue and Angular samples on the landing page.

All changes since 3.0.3 in chronological order

• PR 1104: dbName follows dependencies.indexedDB (II)
• Option {allKeys: true} to bulkPut() and bulkAdd() will be equally fast as not providing that option.
• Code cleanup and optimizations.
• Expose IDB 'close' event: https://github.com/dfahlander/Dexie.js/pull/1212
• BulkError: Possible to track individual errors. Add failuresByPos property: https://github.com/dfahlander/Dexie.js/pull/1209
• Dexie.getDatabaseNames(): Small optimization for our workaround for non-chromium browsers lacking the IDBFactory.databases(). commit.
• Argument to on.ready() callback will get a special Dexie instance that is not blocked (vip Dexie). This was the case also before but then we had to rely on zone state. This change makes it possible to perform non-dexie operations in on.read() callback (such as fetch()), loosing the zone state (PSD) but still have VIP access to the Dexie instance. This makes the code in a on.ready() callback not having to deal with wrapping all non-Dexie calls with Promise.resolve().
• Allow multiple calls to Version.upgrade() on the same version - will run all of them instead just of the latest registered.
• Retiring old workaround for safari 8 bug not allowing array argument to IDBDatabase.transaction().
• Dexie.delete() specifies an empty addons list to ensure no addons are involved when deleting a database using that static method.
• Minor extended the DBCore interface to make it possible for Dexie Cloud to sync certain operations consistently. Specifically, middlewares that implement the DBCoreTable.mutate() endpoint now also gets information on the where-critera and the update specification when originating from Collection.modify() or Collection.delete().
• Support for Chrome's transaction durability option in Dexie constructor. PR #1367
• Official event Dexie.on('storagemutated')
• Typings: Stop exporting Dexie as namespace. Enables VSCode's ergonomic auto-import feature for dexie. The namespace export was not even working as expected. The intent had been to support those that code Typescript without using modules. But even they must have been disappointed because the typings did not only reveal parts of the Dexie API.
• Let liveQuery() be type-wise compable with RxJS (PR #1417)
• Switch to JS implementation of indexedDB.cmp() (PR #1412)

Fixed Bugs since 3.0.3 in chronological order

• #1195 The 'update' crud-hook interpreted array properties as object with number keys.
• #1280 Cannot add CryptoKeys to table with auto-incrementing primary key
• Typing correction: Fix version signatures (PR #1287)
• Fix #1236 - problems updating FileSystemFileHandle properties.
• Bugfix: Transaction.abort() does not rollback changes (filed as a repro PR in #1329). Fixed in PR #1330.
• Bugfix in Virtual Indexes (the reuse of compound indexes as plain indexes): Couldn't reuse parts of primary keys - only parts of indexes.
• PR #1379 fix(bulk-delete): correctly define keys type. Fixes #1364.
• Bugfix for #1381 Collection.delete() fails silently for queries on virtual indexes that use .filter().
• Deleting multiple tables in a new version using null (#1418)
• (Another) mysterious MissingAPI error (#1400)