Bot releases are hidden (Show)

chai - v0.16.0 Latest Release

Published by asdine 10 months ago

Genji is now ChaiSQL ! 🫖

A name change that comes with a change in objectives: ChaiSQL strives to become a modern SQL embedded database.
The goal is to follow the SQL standard as much as possible, without sacrificing schema flexibility.

API changes

Support for DESC composite primary keys and indices
Add TIMESTAMP type
The DOCUMENT type is now OBJECT

Other notable changes

StructScan check interface parameter is a pointer to a struct by @abramlab in https://github.com/chaisql/chai/pull/487
Add LOWER builtin function by @agonist in https://github.com/chaisql/chai/pull/510
Add UPPER function by @agonist in https://github.com/chaisql/chai/pull/511
add trim, ltrim, rtrim functions by @agonist in https://github.com/chaisql/chai/pull/512
Add coalesce function by @yaziine in https://github.com/chaisql/chai/pull/513
Add random function by @agonist in https://github.com/chaisql/chai/pull/514

New Contributors

@agonist made their first contribution in https://github.com/chaisql/chai/pull/510
@abramlab made their first contribution in https://github.com/chaisql/chai/pull/487

Full Changelog: https://github.com/chaisql/chai/compare/v0.15.3...v0.16.0

chai - v0.15.3

Published by asdine 10 months ago

Update Pebble to v1.0.0
Remove encryption experiment

chai - v0.15.2

Published by asdine over 1 year ago

This release contains one bug fix:

StructScan: deep copy strings during scan #499

chai - v0.15.1

Published by asdine over 2 years ago

This release contains a few bug fixes:

orderby: fix evaluation of ORDER BY expr when used with projection #471
errors: publish error functions to easily determine if an error is a NotFoundError or an AlreadyExists error. 492cdbe26b2f2f415a37912f291e552af7484eaa
pebble: use custom Separator function. Without this, Pebble would sometimes generate an incompatible key that would cause a panic a35719a72759576a2527bfda0007f7c157354bb8

chai - v0.15.0

Published by asdine over 2 years ago

Genji v0.15.0 is out and comes a few big architectural changes 🚀

Goodbye Bolt and Badger, hello Pebble

Historically, Genji started as a layer on top of the amazing Bolt and Badger.
As features were being added, Genji is pretending to become more and more like a full featured SQL database, and thus needed more flexibility than what Bolt and Badger could offer.

Things like custom key comparators (to control how keys are organized in the tree), snapshots, concurrent read/writes (in Bolt only), were missing greatly. Both came with their own transaction mechanism which were a bit limiting for Genji (limited number of writes per transaction, no support for transaction promotion, etc.).
Also, having to maintain both data layers came with lots of challenges that we believe wasn't worth it.

That's why we decided to use a single backend: Pebble.

Pebble comes with lots of nice features, here are the ones we consider particularly interesting for Genji:

No transactions (this is a good thing, we have our own now!)
Batches
Snapshots
Can also be used in-memory only
Full control on how the tree is organized

New transaction mechanism

Pebble doesn't come with its own transactions mechanism, so we had to implement our own, which comes will lots of advantages:

Reads don't block writes, writes don't block reads: Read transactions always operate on the latest snapshot and don't require any locking. Write transactions use small in-memory batches which are atomically persisted alongside a rollback segment. The rollback segment contains undo changes to apply if the transaction is rolled back or if the database crashes.
Fully serializable transactions: Genji supports multiple readers at a time and a single writer. This doesn't change from the previous releases, however we believe having a single writer should be more than enough in a embedded setting.
Arbitrarily large transactions: Write transactions can contain a virtually unlimited number of changes while guaranteeing they won't be taking more than 10MB of ram (hardcoded value for this release). This allows to run Genji on small devices without worrying about the ram usage.
Super fast writes: This release doesn't come with any official benchmark but our own biased ones showed that when inserting large documents, writes are twice as fast as the amazing SQLite (with WAL mode). Again, this is on a very specific benchmark, on a very specific device. But it is promising.

New CREATE TABLE API

The CREATE TABLE statement introduces new rules for table creation that gives much more control.

By default, a table now only inserts fields that have been declared. Any other field will be ignored and not inserted.

CREATE TABLE foo (a INTEGER, b TEXT);
INSERT INTO foo (a, b, c) VALUES (1, 'hello', 'not inserted');
INSERT INTO foo VALUES {a: 2, b: 'bye', c: 'not inserted'};
SELECT * FROM foo;

{
  "a": 1,
  "b": "hello"
}
{
  "a": 2,
  "b": "bye"
}

If you want to insert any field, and only care about declaring a few ones, you can use the ... notation.

CREATE TABLE foo (a INTEGER, b TEXT, ...);
INSERT INTO foo (a, b, c) VALUES (1, 'hello', 'inserted');
SELECT * FROM foo;

{
  "a": 1,
  "b": "hello"
  "c": "inserted"
}

Nested fields are now declared like this:

CREATE TABLE user (
    name TEXT,
    address (
        number INTEGER,
        street TEXT,
        zipcode TEXT NOT NULL
    )
);
INSERT INTO user (name, address) VALUES (
    "Ilias",
    {
        number: 10,
        street: "rue de la paix",
        zipcode: "12345"
    }
);

Drop support for WASM and TinyGo

Historically, Genji has invested a lot of efforts in supporting TinyGo in the hopes of being compiled in WASM and being used in the browser.
However, because of the lack of support for certain packages of the standard library (like reflect, fmt, encoding/jsom, etc.), it was very difficult for us to keep maintaining compatibility across different releases.
Even when we managed to compile, TinyGo would produce very large files (> 3MB), which defeats the purpose of using it.
We may reconsider this in the future as the support for Wasm in TinyGo becomes more mature.

Breaking changes and data format stability

⚠️ This release introduces a big refactoring and thus is not compatible with previous versions.
⚠️ It also only support Go 1.18+ as it relies on the new generics feature introduced in Go 1.18.

However, the database format is almost stable and future releases should not introduce any breaking changes.
If that happens, we will over communicate and provide ways to make it transparent as much as possible.

Our goal is to reach v1 very soon 🚀

Other notable changes

index: treat all NULL values differently in UNIQUE indexes by @asdine in https://github.com/genjidb/genji/commit/2d4df658e47e276364be3d33c42ebc6856e86f70
index: use index with BETWEEN operator by @asdine in https://github.com/genjidb/genji/commit/a304e806ff2f0934b8fcd75762d6bd65aadc32ff
index: prevent index creation on undeclared paths by @asdine in https://github.com/genjidb/genji/commit/3f32bcbf19161b856c7dcbb47dcc0657790ad2e4
cli: fix genji restore command with certains whitespace characters by @tzzed in https://github.com/genjidb/genji/pull/439
cli: add .timer shell command by @tzzed in https://github.com/genjidb/genji/pull/445
cli: ensure the inserts are run in the same transaction in genji insert by @tzzed in https://github.com/genjidb/genji/commit/b4606f7cdfd9c025b83b85e1c3c28f05b277a56a
sql: fix UPDATE behavior with primary keys by @asdine in https://github.com/genjidb/genji/commit/f81f3f18f04df59fd6fa0c13b8c97e38bb3a1228
sql: support params in LIMIT and OFFSET by @asdine in https://github.com/genjidb/genji/commit/6178d1acfb341950f83631a48132fde3b78382bf

Full Changelog: https://github.com/genjidb/genji/compare/v0.14.0...v0.15.0

chai - v0.14.1

Published by asdine almost 3 years ago

Fix JSON encoding of msgpack documents #438

chai - v0.14.0

Published by asdine almost 3 years ago

SQL

Add more math functions by @jhchabran in https://github.com/genjidb/genji/pull/421
Parse blob literal by @asdine in https://github.com/genjidb/genji/pull/423
Add typeof function by @asdine in bba4c0a63006e936644301483d6f7a0d29283f37
Handle integer conversion errors by @jhchabran in https://github.com/genjidb/genji/pull/425
Add support for scientific notation for doubles by @asdine in d3b81ad6f1a3c58b471123dc38ada766fc215f14
Parse optional parentheses on DEFAULT clause by @asdine in 1654f358d40b2d611afdffb48d95045fc0bc6461
Add support for UNION by @asdine in 412d30030d97c9d6414a91e08ad0f114a75c6f4b
Add support for CHECK by @asdine in https://github.com/genjidb/genji/pull/436
Add support for composite primary keys by @asdine in 8983d68daf1d030e5a2282ac39cdf76d637bdac8

CLI

Add bench command by @asdine in 74603aa55e815bc97dd5d5aca5a9fea9fc8f78ed
Fix .indexes command when transaction is active by @asdine in d5fd8796d760e12bb5adcf23fdf3fdc956a57f2f

Other

Improve failures readability when testing exprs by @jhchabran in https://github.com/genjidb/genji/pull/426
Add support for transient databases by @asdine in a45d2c1ae57cd6605a630888b55813a65e2b4d7a
Improved planner logic by @asdine in https://github.com/genjidb/genji/pull/428
Add errors package with optional stacktrace capturing by @jhchabran in https://github.com/genjidb/genji/pull/431
Refactor to handle errors with internal/errors by @jhchabran in https://github.com/genjidb/genji/pull/432
Parse path fields with brackets by @asdine in cfdb78487fe124ed804c9627168ed22d26888a92
Fixed filling index types when loading indices from existing db by @KudinovKV in https://github.com/genjidb/genji/pull/434
Use same tree structure for both tables and indexes by @asdine (⚠️ Breaking change) in a7309a70c50167679b4ec1152a36ab559de8d8ab
Fix goacc and CI by @asdine in https://github.com/genjidb/genji/pull/435

New Contributors

@KudinovKV made their first contribution in https://github.com/genjidb/genji/pull/434

Full Changelog: https://github.com/genjidb/genji/compare/v0.13.0...v0.14.0

chai - v0.13.0

Published by asdine about 3 years ago

SQL

Add concat operator
Add NOT operator
Add BETWEEN operator
Add INSERT ... RETURNING
Add ON CONFLICT
Add UNION ALL #408 (@jhchabran)
Add CREATE SEQUENCE
Add DROP SEQUENCE
Add NEXT VALUE FOR

Core

Add document.NewFromCSV
Add prepared statements
Add new __genji_catalog table and drop __genji_table and __genji_indexes
Remove engine.NextSequence

CLI

Add .import command

This release also contains various bug fixes.

chai - v0.12.0

Published by asdine over 3 years ago

SQL

Added support for composite indexes #376 (⚠️Breaking change) @jhchabran
Added support for UNIQUE table constraint #387
Added support for PRIMARY KEY table constraint #333
Added support for INSERT ... SELECT #385
Indexes can be created with a generated name now #386

Core

Fix indexed array comparisons with numbers #378
Fix a bug preventing the use of time.Time with driver.Scanner #258

CLI

Added a new way of writing integration tests: Examplar #380 @jhchabran
Added .schema command #339 @tzzed

chai - v0.11.0

Published by asdine over 3 years ago

SQL

Add OFFSET, LIMIT and ORDER BY to DELETE statement #318 #319 #320 @jhchabran
fix: query returning no result when using ORDER BY with sql/driver implementation #352 @tzzed

Core

New Stream API #343
Update Badger to v3 #350
Integers are now compacted when encoded #351 (⚠️Breaking change) @jhchabran
Infer constraints based on user-defined ones #358
Queries using primary keys are much faster #310
fix: Encoding of consecutive integers in arrays #362 @jhchabran
fix: Incorrect index state with UPDATE queries on indexed fields #355 #368
fix: Build index upon creation #371
fix: BLOB to byte array decoding #305

CLI

New genji dump command #315 @tzzed
New genji restore command #321 @tzzed
fix: .dump not outputting DEFAULT clause #329 @jhchabran
fix: Deadlock when trying to open a locked database #365
fix: Panic on genji insert when specified engine doesn't exist #331

chai - v0.10.0

Published by asdine over 3 years ago

SQL

fix: COUNT(*) with empty set #284
fix: Selection of field in GROUP BY expression #317
fix: Index selection of IN operator #281

Core

Update msgpack #340 @kenshaw
fix: Panic when optimizer returns empty tree #283 @goku321

CLI

Add .save command #311 @jhchabran

chai - v0.9.0

Published by asdine almost 4 years ago

SQL

Drop Duration type #212 @asdine
Expose the __genji_indexes table #214 @tzzed
Add ALTER TABLE ... ADD FIELD ... #96 @tdakkota
Prevent creating a table if constraints are incoherent #222 @tdakkota
Add LIKE operator #241 @tdakkota
Add SELECT DISTINCT #264 @tdakkota
fix: IN operator behavior with parentheses #207 @tdakkota
fix: Panic when parsing large integers #256 @goku321
fix: Ignore currently unsupported regex operators #252 @tie
fix: Panic on SELECT with LIMIT min(0)/max(0) #257 @goku321
fix: Panic when running GROUP BY on array values #208 @asdine

Core

Implement driver.DriverContext and driver.Connector interfaces in sql/driver #213 @tie
Add support for context.Context #206 #224 @tie @asdine
Slightly more efficient conversion from JSON to document #270 @asdine
Add support for embedded structs to document.NewFromStruct #225 @tzzed
Numbers are stored as double by default, unless a field constraint is specified during table creation #312 @asdine (⚠️ Breaking change)
fix: CI tests with nested modules #232 @tie
fix: badgerengine benchmarks #209 @tie
fix: Be more forgiving when scanning null values in StructScan #211 @tie
fix: Badger tests on Windows #288 @tie
ci: Add support for fuzzing #253 @tie

CLI

Respect NO_COLOR and NO_HISTORY environment variables #215 @tie
Add .dump command #181 @tzzed
Add version command #184 @cvhariharan
Add support for exit and help commands without leading dot #180 @Amirhossein2000
fix: Autocompletion panic when there is no suggestion #178 @tdakkota
fix: Panic on genji version when compiled in GOPATH mode #261 @tdemin

chai - v0.8.1

Published by asdine almost 4 years ago

Bug fixes

Fix deadlock when running concurrent transactions #298

chai - v0.8.0

Published by asdine about 4 years ago

SQL

Add BEGIN, ROLLBACK, and COMMIT statements #78
Add support for paths in UPDATE statement #84
Add GROUP BY clause #6
Add COUNT aggregator #5
Add MIN aggregator #165
Add MAX aggregator #166
Add SUM aggregator #4

Core

Add codec system #177
Remove introduction text when reading from STDIN #179
Improved how values are indexed #194
Indexes created on field who has a typed constraint are now typed #195
Add support for array and document indexes #199
Removed encoding/json dependency and use jsonparser instead #203

CLI

Add table autocompletion #143
Add index autocompletion #157
Add command suggestions #163

Bug fixes

Fix order of SELECT ast nodes #188
MapScan now decodes nested documents #191
Fix saving of history when .exit was called #196

chai - v0.7.1

Published by asdine about 4 years ago

Bug fixes

Fix how list indexes are stored #170
Fix projection of __genji_tables primary key #173

chai - v0.7.0

Published by asdine about 4 years ago

SQL

Add REINDEX statement #72
Add ALTER TABLE ... RENAME TO ... statement #95
Add new __genji_tables table #152
Allow referencing current document in expression #147

Core

Removed fixed size integers #130
Renamed float64 type to double #130
Integers are now converted to double prior to comparison and indexing #146
Encode documents using MessagePack #117
Move badgerengine into its own module #140
Replaced memoryengine with custom implementation #139
Store table information in memory #142
Add support for time.Time in document.StructScan

CLI

Add .help command #160
Add .indexes command #100
Add table suggestions after FROM keyword #159
Ignore input with whitespace only #106

Bug fixes

Prevent primary key overlap with concurrent inserts #74
Fix behavior of ValueBuffer#Copy #111
Fix UPDATE ... SET clause setting the wrong array indexes #91
Fix display of field names with backquotes #64
Arithmetic operators return null for incompatible types #105
Fix parentheses behavior #131
Fix CAST behavior #138
Remove transaction promotion #150

chai - v0.6.0

Published by asdine over 4 years ago

SQL

Added support for IS and IS NOT #75
Added support for IN and NOT IN #76 #81
Added support for UPDATE ... UNSET #68
Added support for field constraints for missing types #85
Added support for EXPLAIN #102

Core

Added support for Cursors to Engines #40
Added query planner #88

Bug fixes

Fix null to text conversion #66
Make UPDATE ... SET set a field to every matching document #82
Fix panic when ORDER BY is used with an indexed field #71
Fix DROP TABLE behavior that used to remove all database indexes #99
Normalize behavior of boolean field constraint with incompatible types #89

chai - v0.5.0

Published by asdine over 4 years ago

Support NOT NULL field constraints
Support Bitwise operators
Support Duration values
Support SELECT without table reference
Support arithmetic operators
Support AS
Support CAST
Use Badger as main in memory engine

chai - v0.4.0

Published by asdine almost 5 years ago

Core

Support for Documents
Functions for translating structs and maps into documents
Renamed references to Record to Document
Moved database logic to database package

SQL

Insert documents using document notation
Select sub fields
Support for ORDER BY
Support for field constraints

Misc

Removed code generation temporarily
Improved shell auto completion

chai - v0.3.0

Published by asdine almost 5 years ago

Changelog

Core

New (db/tx).QueryRecord method to query a single record
New db.SQLDB method that wraps the db into a database/sql db
New tx.ListTables method
New tx.ReIndex method
New tx.ReIndexAll method
New index implementation
Moved package recordutil to record
New record.Scan method

SQL

Support key() function to return the primary key
Support specifying primary key of any type during a CREATE TABLE statement
Generate autoincrementing default key instead of uuid
Support multiple wildcards, fields and functions in SELECT statement
Support != operator
Support == operator as an alias for =
Better support for NULL
Parsed integers are converted to the smallest int size that fits
Double quoted strings are now treated exclusively as identifiers
Where expression containing only the primary key will benefit from a primary key optimization
Float32 numbers no longer supported