electric

Patch Changes

• 5bec40a: Fixed multiple operations touching the same row within a tx not being applied properly on PG
• 212ec8d: Fixed a bug with lost writes when PG dropped the replication connection

Patch Changes

• 7cc6e27: Use correct version of electric-sql on the starter template

Minor Changes

• 69d13c4: Rewritten the Electric sync layer to remove Vaxine and enable partial replication

  This release encompasses quite a bit of work, which resulted in a complete overhaul of the current system. It's hard to note all changes, see git history for that, but this public release
  marks the first somewhat stable version of the system for which we intend incremental changes.

  That said, this is still considered an unstable release and so this is only a minor version bump.
  We intend to keep this and the electric-sql typescript library somewhat in sync on minor versions up to 1.0.0: new features that require both server-side support as well as client-side support are going to result in a minor-level version bump on both, while features/improvements that don't need both sides will be marked as a patch version.

  Rough change list between previous version and this one:

  • Removed Antidote (Vaxine) from the system
  • Moved CRDT conflict resolution into GitHub using special observed operations CRDT form
  • Streamlined the deployment heavily by relying now on unpatched Postgres
  • Removed the Postgres-to-Postgres replication
  • Added the concept of "electrified" tables - only those can be synced to the client
  • Added PG-managed migrations instead of relying on an external system (previously, our cloud console) to manage them
  • Added migration propagation to the client so that the server may dictate table creation to the clients
  • Heavily improved initial sync story: we don't send the entire Antidote write-ahead log on connect, we send actual data queried from Postgres and then start streaming the changes
  • Added the first iteration of partial replication, where the client subscribes to tables they are interested in instead of the all available tables

Patch Changes

• f8cf910: Remove SchemaRegistry and use our cached knowledge of the pg schema to fulfill all requirements for knowledge of the replicated tables

• 5fe00af: Fixed sync from subscription not respecting electric_user_id filtering

• afaedcd: Make some configuration option names more descriptive. Namely,

  • rename ELECTRIC_HOST to LOGICAL_PUBLISHER_HOST
  • rename POSTGRES_REPLICATION_PORT to LOGICAL_PUBLISHER_PORT
  • rename STATUS_PORT to HTTP_API_PORT

  The first two options together define the host:port pair that the PostgreSQL database will connect to
  in order to start following Electric's logical replication stream that publishes inbound changes from all clients.

  The HTTP_API_PORT option is now more aptly named since Electric exposes more than just the status endpoint
  on that port.

• 6026ced: Update all deps that had new backwards-compatible versions

• 11acadd: Allow for enabling SSL for PG connections via an env variable

• 57324c4: Add server-side validation of row values incoming from Satellite clients

• 089968d: Fixed the issue whereby calling electrify() on a previously electrified table caused a duplicate migration to be created and put onto the replication stream.

• f60ce16: Implemented correct semantics for compensations to work across the stack

• 3cf2bc2: Support for intersecting shape subscriptions

• 8b8cc93: Fix a bug in schema version validation

• e17b37e: Fix the bug where the client failed to restart the replication connection after completing the initial sync once.

• 2e8bfdf: Fixed the client not being able to reconnect if the migrations were preloaded and the only operation was a subscription. In that case the client have never received any LSNs (because migrations didn't need to be sent), so reconnection yielded errors due to missing LSN but existing previously fulfilled subscriptions. We now send the LSN with the subscription data so even if it's the first and only received message, the client has enough information to proceed.

• e4cbf80: Update Elixir to 1.15.4 to fix remote shell

• 571119a: Add a validation step in the electrify() function that only lets tables with supported column types to be electrified

• 3ca4917: Fixed an issue where sometimes subscription data would not be sent in absence of other writes to PG

• 00d5a67: Add server-side enforcement of "NOT NULL" for values incoming from Satellite clients

Patch Changes

• 5567869: Use PascalCased model names in generated Prisma schema and map them to the original table names.
• 9db6891: Also fix casing in types that refer to model names

Patch Changes

• 1c20e29: Improve starter template output and introduced new db:connect command.
• d9344b9: Use a tab scoped dbName so the starter example syncs across tabs.
• 345cfc6: Added auth.insecureAuthToken function and updated examples to use it.
• 29c7cc3: Starter template for bootstrapping Electric applications.
• 158431b: Include template in packaged files

Minor Changes

• 69d13c4: Large rewrite, that introduces Data Access Library, client-side migration application, and subscriptions.

  This release encompasses quite a bit of work, but this is still considered an unstable release and so this is only a minor version bump.
  We intend to keep this and the components/electric server sync layer somewhat in sync on minor versions up to 1.0.0: new features that require both server-side support as well as client-side support are going to result in a minor-level version bump on both, while features/improvements that don't need both sides will be marked as a patch version.

  Data access library exposes a prisma-like interface to interact with SQLite (any platform works, including web). On top of that, we introduce a .sync() interface which establishes a subscription with the server to get initial data and changes for the tables we're interested in.

  Server now knows how to send migrations to the clients as soon as they get applied to the underlying PostgreSQL database, and this client version applies them to the local database.

  Server now knows how to handle subscriptions and reconnection a bit better, which the client makes use of (in particular, this heavily improves initial sync performance).

Patch Changes

• 5567869: Use PascalCased model names in generated Prisma schema and map them to the original table names.
• dc48f1f: Fixed liveMany, liveUnique, and liveFirst functions not exposing the include tables properly, making useLiveQuery miss some relevant updates
• c588bdf: Fixed not sending all the transactions if more than one was done within a throttle window
• 0bcc92c: Added a uniqueTabId utility function for apps that want to use tab-scoped DB names.
• 1c20e29: Improve starter template output and introduced new db:connect command.
• 4531dde: Fix unreliable behaviour in the React useConnectivityState hook.
• 49c4b35: Allowed https: protocol in service connections for CLI
• f60ce16: Implemented correct semantics for compensations to work across the stack
• b29693e: Modify generated migrations file to be a .ts file instead of .js file
• 18619ef: Fixed race condition in throttled perform snapshot
• 232f7a5: Updated snapshotting function to be more efficient when handling a large oplog
• edfb298: Improved subscription data insertion to do batched inserts instead of one-by-one statements. Inserting a lot of data should be much faster.
• 525c8d1: Moved CLI dependency from dev to prod dependency list
• a112a03: Fixed Satellite not handling mutliple concurrent subscription calls properly
• bd02d79: Fixed garbage collection on shape unsubscribe (also called on subscription errors), which caused the DELETEs performed by GC get noted in operations log and sent to the server, causing data loss
• 49cbe27: Fixed using .sync() calls before the replication is established. Now we defer trying to subscribe until the replication is succesfully established if we're already in the process of connecting and starting.
• 3345cf8: Bugfix: update the wa-sqlite driver to use the WebSocketFactory.
• 2e8bfdf: Fixed the client not being able to reconnect if the migrations were preloaded and the only operation was a subscription. In that case the client have never received any LSNs (because migrations didn't need to be sent), so reconnection yielded errors due to missing LSN but existing previously fulfilled subscriptions. We now send the LSN with the subscription data so even if it's the first and only received message, the client has enough information to proceed.
• 345cfc6: Added auth.insecureAuthToken function and updated examples to use it.
• c30483f: Fixed conflicts when using multiple instances of Electric on the same page
• 93d7059: Fixed calls to the .sync() function when page reloads establishing a new subscription alongside existing one. Now we deduplicate subscription requests by hash of their contents, so no two exactly the same subscriptions are possible. This also solves calling <table>.sync() many times in multiple components - they will return the same promise.
• 80531f0: Fixed subscription being registered too late preventing deduplication
• 10bbae9: Moved better-sqlite3 to dependencies because CLI command uses it
• 3cb872d: Chore: made _getMeta types more precise
• 9db6891: Also fix casing in types that refer to model names
• e165048: Fixed subscription data not triggering data changed notification and thus liveQuery not working as expected
• d359cae: Made argument of liveMany optional.
• 7eab08e: Improved config.url parsing and SSL support.
• 8209293: Fixed reconnect issues when the client is too far behind the server
• f4184b1: Fix: ensure we do much more cleanup in useEffect returned functions and in close method of Satellite

Patch Changes

• d9344b9: Use a tab scoped dbName so the starter example syncs across tabs.
• 345cfc6: Added auth.insecureAuthToken function and updated examples to use it.

Patch Changes

• 0bcc92c: Added a uniqueTabId utility function for apps that want to use tab-scoped DB names.
• 345cfc6: Added auth.insecureAuthToken function and updated examples to use it.

Patch Changes

• c30483f: Fixed conflicts when using multiple instances of Electric on the same page

Patch Changes

• 5fe00af: Fixed sync from subscription not respecting electric_user_id filtering

• afaedcd: Make some configuration option names more descriptive. Namely,

  • rename ELECTRIC_HOST to LOGICAL_PUBLISHER_HOST
  • rename POSTGRES_REPLICATION_PORT to LOGICAL_PUBLISHER_PORT
  • rename STATUS_PORT to HTTP_API_PORT

  The first two options together define the host:port pair that the PostgreSQL database will connect to
  in order to start following Electric's logical replication stream that publishes inbound changes from all clients.

  The HTTP_API_PORT option is now more aptly named since Electric exposes more than just the status endpoint
  on that port.

Patch Changes

• 29c7cc3: Starter template for bootstrapping Electric applications.

Patch Changes

• 11acadd: Allow for enabling SSL for PG connections via an env variable
• 57324c4: Add server-side validation of row values incoming from Satellite clients
• f60ce16: Implemented correct semantics for compensations to work across the stack
• 3cf2bc2: Support for intersecting shape subscriptions
• 571119a: Add a validation step in the electrify() function that only lets tables with supported column types to be electrified
• 00d5a67: Add server-side enforcement of "NOT NULL" for values incoming from Satellite clients

Patch Changes

• 1c20e29: Improve starter template output and introduced new db:connect command.
• 158431b: Include template in packaged files

Patch Changes

• f8cf910: Remove SchemaRegistry and use our cached knowledge of the pg schema to fulfill all requirements for knowledge of the replicated tables

Patch Changes

• 5567869: Use PascalCased model names in generated Prisma schema and map them to the original table names.
• 9db6891: Also fix casing in types that refer to model names

Patch Changes

• 5567869: Use PascalCased model names in generated Prisma schema and map them to the original table names.
• dc48f1f: Fixed liveMany, liveUnique, and liveFirst functions not exposing the include tables properly, making useLiveQuery miss some relevant updates
• c588bdf: Fixed not sending all the transactions if more than one was done within a throttle window
• 4531dde: Fix unreliable behaviour in the React useConnectivityState hook.
• b29693e: Modify generated migrations file to be a .ts file instead of .js file
• 18619ef: Fixed race condition in throttled perform snapshot
• 232f7a5: Updated snapshotting function to be more efficient when handling a large oplog
• 10bbae9: Moved better-sqlite3 to dependencies because CLI command uses it
• 3cb872d: Chore: made _getMeta types more precise
• 9db6891: Also fix casing in types that refer to model names
• 7eab08e: Improved config.url parsing and SSL support.
• f4184b1: Fix: ensure we do much more cleanup in useEffect returned functions and in close method of Satellite

Patch Changes

• e4cbf80: Update Elixir to 1.15.4 to fix remote shell

Patch Changes

• edfb298: Improved subscription data insertion to do batched inserts instead of one-by-one statements. Inserting a lot of data should be much faster.
• a112a03: Fixed Satellite not handling mutliple concurrent subscription calls properly
• 3345cf8: Bugfix: update the wa-sqlite driver to use the WebSocketFactory.
• 80531f0: Fixed subscription being registered too late preventing deduplication
• e165048: Fixed subscription data not triggering data changed notification and thus liveQuery not working as expected

Patch Changes

• 089968d: Fixed the issue whereby calling electrify() on a previously electrified table caused a duplicate migration to be created and put onto the replication stream.
• 8b8cc93: Fix a bug in schema version validation
• e17b37e: Fix the bug where the client failed to restart the replication connection after completing the initial sync once.
• 2e8bfdf: Fixed the client not being able to reconnect if the migrations were preloaded and the only operation was a subscription. In that case the client have never received any LSNs (because migrations didn't need to be sent), so reconnection yielded errors due to missing LSN but existing previously fulfilled subscriptions. We now send the LSN with the subscription data so even if it's the first and only received message, the client has enough information to proceed.
• 3ca4917: Fixed an issue where sometimes subscription data would not be sent in absence of other writes to PG

Patch Changes

• bd02d79: Fixed garbage collection on shape unsubscribe (also called on subscription errors), which caused the DELETEs performed by GC get noted in operations log and sent to the server, causing data loss
• 49cbe27: Fixed using .sync() calls before the replication is established. Now we defer trying to subscribe until the replication is succesfully established if we're already in the process of connecting and starting.
• 2e8bfdf: Fixed the client not being able to reconnect if the migrations were preloaded and the only operation was a subscription. In that case the client have never received any LSNs (because migrations didn't need to be sent), so reconnection yielded errors due to missing LSN but existing previously fulfilled subscriptions. We now send the LSN with the subscription data so even if it's the first and only received message, the client has enough information to proceed.
• 93d7059: Fixed calls to the .sync() function when page reloads establishing a new subscription alongside existing one. Now we deduplicate subscription requests by hash of their contents, so no two exactly the same subscriptions are possible. This also solves calling <table>.sync() many times in multiple components - they will return the same promise.