libsql v0.2.1

• WasmEdge and Wasmtime runtimes for user-defined functions now support WASI; WasmEdge can also work with custom plugins
• CREATE FUNCTION and DROP FUNCTION is now unconditionally supported in the grammar; no change in existing semantics, it simply makes the amalgamation distribution simpler
• the new Rust libSQL shell is now included in the source code - contributions welcome!
• additional metrics - LIBSQL_STMTSTATUS_ROWS_READ and LIBSQL_STMTSTATUS_ROWS_WRITTEN are now available in sqlite3_stmt_status
• and more!

The libSQL team is pleased to announce libSQL 0.2.0!

This release comes with a bunch of features and fixes, including:

Virtual WAL

Write-ahead log is a journaling mode which enables nice write concurrency characteristics - it not only allows a single writer to run in parallel with readers, but also makes BEGIN CONCURRENT transactions with optimistic locking possible (once BEGIN CONCURRENT gets merged upstream*). In SQLite, WAL is not a virtual interface, it only has a single file-based implementation, with an additional WAL index kept in shared memory (in form of another mapped file). In libSQL, akin to VFS, it's possible to override WAL routines with custom code. That allows implementing pluggable backends for write-ahead log, which opens many possibilities (again, similar to the VFS mechanism).
Virtual WAL is already used in sqld and bottomless S3 storage for libSQL.

WasmEdge support for user-defined functions

It is now possible to pick WasmEdge as the WebAssembly runtime powering our user-defined functions. WasmEdge is a very lightweight runtime, and yet fully feature-complete, with a slim build that takes only around 1MiB. That makes it a perfect choice for environments with storage limits, like embedded systems, but also serverless infrastructure. In order to compile libSQL with WasmEdge, install the libwasmedge library and configure libSQL with ./configure --enable-wasm-runtime-wasmedge. A WasmEdge-flavored build is also available at our 0.2.0 release page.

auto-initialization for WebAssembly user-defined functions

It is no longer required to initialize the WebAssembly UDF function table explicitly before executing a CREATE FUNCTION statement - the initialization will be performed automatically.
Note that for backward compatibility purposes it's still required to run try_initialize_wasm_func_table (or .init_wasm_func_table in shell) prior to the first execution of a Wasm-backed user-defined function.

New method in sqlite3_module: xPrepareSql (contributed!!!)

A new method named xPrepareSql is added to the sqlite3_module struct. An
associated opcode VPrepareSql is added as well. That opcode is emitted immediately before a VFilter opcode when querying virtual tables.
xPrepareSql gives an opportunity for the virtual table implementation to
inspect the query string and selectively choose which columns from the
remote server to pull and cache locally.

liblibsql.so and liblibsql.a libraries

Release 0.1.0 shipped the libSQL library in the form of libsqlite3.so and libsqlite3.a libraries, under the assumption that they should be used as drop-in replacement for an existing SQLite installation. It is however useful to be able to keep SQLite intact and use libSQL libraries only for a specific project. For that purpose, 0.2.0 comes with more artifacts: liblibsql.so and liblibsql.a. They contain exactly the same binary data as their sqlite3 counterparts, but there's no longer a name clash and thus libSQL 0.2.0 can be used alongside any existing SQLite installation.
This change was inspired by our brand new Python driver, https://github.com/libsql/pylibsql/, which now depends on the libsql dynamic library.

libsql_api_routines

In order to keep libSQL extension mechanism compatible with the one in SQLite, additional API routines are now kept in a separate structure - libsql_api_routines. The original sqlite3_api_routines structure is left intact.

Fixes, fixes, fixes

Lots of fixes of our code and documentation, some of them contributed by the community, are part of the 0.2.0 release as well!

Hall of Fame

libSQL is not only Open-Source, but also Open-Contribution. Huge kudos to all contributors, thanks for making this release happen!

69  Piotr Sarna
19  Pekka Enberg
 5  Lucas C. Villa Real
 4  Jan Špaček
 3  ad hoc
 2  Glauber Costa
 1  T.J. Yang
 1  Doug Stevenson
 1  belzecue

libSQL 0.1.0 release notes

The libSQL team is pleased to announce libSQL 0.1.0!

This release comes with the following features and fixes:

CREATE FUNCTION: support for WebAssembly user-defined functions

User-defined functions are here, available straight from SQL! The new CREATE FUNCTION and DROP FUNCTION syntax allows to manage functions dynamically right from the SQL interpreter, without having to register them via any API calls.
Support for user-defined functions is experimental and needs to be explicitly enabled via configure:

./configure --enable-wasm-runtime.
WebAssembly is a compiled language, so with the release come helper projects to make creating new user-defined functions easy:

• https://bindgen.libsql.org/
  an interactive playground for compiling Rust code to WebAssembly user-defined functions for libSQL
• https://github.com/libsql/libsql_generate
  source-code for the playground above
• https://crates.io/crates/libsql_bindgen
  a Rust crate used to transform native Rust code to WebAssembly user-defined functions for libSQL

Current implementation runs on Wasmtime and can be compiled either as a separate dynamic library, or as monolith. The support is implemented directly in Wasmtime’s native language — Rust — and linked with existing C code. Next on our plate: integration with V8) API as well as WasmEdge, [Wasmer.io](http://wasmer.io/) and more!

New test suite

SQLite is famous for its extensive test coverage, which is the foundation of its robustness and reliability. Existing SQLite tests are already part of our continuous integration. What’s less broadly known is that one of the test suites providing 100% branch coverage to SQLite — TH3 — is proprietary.

In libSQL we insist on making the whole source code available, tests included, and that’s why we added yet another test suite to the pack. SQLite OSS tests are written in Tcl. We are not experts in Tcl, so the new suite is based on the Rust toolchain.

We acknowledge that this test suite is not extensive as TH3, but we do plan to work towards it, and your contributions for new tests are always welcome! The new suite is available at test/rust_suite. If you are new to libSQL, and want to contribute, that’s a great start.

optimization of ext/wasm build

ext/wasm extension allows building libSQL to Wasm. It's a fantastic initiative kickstarted back at SQLite. We applied a few rounds of optimization on top of the project to make it even more lightweight and fast - reducing the module size from over 3MiB to less than 1MiB. A preview for the optimized playground application is hosted here: https://playground.libsql.org/.

RANDOM ROWID

Tables can now be created with an additional RANDOM ROWID directive, which switches the rowid generation algorithm from sequential numbers to trying pseudorandom values until an available one is found. This feature was requested in context of mvSQLite, a distributed SQLite-compatible database, where sequential row ids are much more likely to cause contention with multiple writers.

Lots of fixes and amendments

• improved make clean to cover more files
• added explicit I/O flush in iotrace mode
• added a .gitignore file
• dropped .fossil-settings
• and more!

Contributions

libSQL is not only Open-Source, but also Open-Contribution. Huge kudos to all contributors, thanks for making this release happen!