Bot releases are visible (Hide)

dolt - 0.35.3

Published by github-actions[bot] almost 3 years ago

Merged PRs

dolt

2579: /go/doltcore/sqle: Fixed write perf regression

Closed Issues

dolt - 0.35.2

Published by github-actions[bot] almost 3 years ago

Merged PRs

dolt

2576: Fixed bug in using TRIM function in CHECK constraints

go-mysql-server

722: Bug fix for using TRIM function in CHECK constraints
720: Allow deterministic functions in CHECK constraints

Closed Issues

2554: dolt_status is panicking after changes

dolt - 0.35.1

Published by github-actions[bot] almost 3 years ago

Merged PRs

dolt

2573: Allow most function calls in CHECK constraints
2570: Fix bad error message in SHOW TABLES when no database selected
This fixes https://github.com/dolthub/dolt/issues/2559
Also fixes https://github.com/dolthub/dolt/issues/2534
2569: /go/doltcore/doltdb: Replaced doltdb.Table.IndexData types.Map with durable.IndexSet
2564: /go/store/{prolly,skip,val}: New Storage Format
2560: Fix bug with import update dropping rows
This PR addresses a customer issue where --update,--continue on a table import were dropping rows. The error comes from the facts that INSERT IGNORE semantics were not correctly handling issues with data conversions (type conversions, string lengths, etc.)
2555: go/utils/copyrighthdrs: Allowlist 2022.
2529: /go/libraries/doltcore/sqle/writer: Extracted Interface for WriteSession
Extracted an interface from TableEditSession to allow for multiple implementations for different formats

go-mysql-server

720: Allow deterministic functions in CHECK constraints
719: Fixed a bug in trigger execution involving update statements in BEGIN blocks
This fixes https://github.com/dolthub/dolt/issues/2534
717: add TIMESTAMPDIFF() function
716: Detect and give a reasonable error message for SHOW TABLES with no current db
Fixes https://github.com/dolthub/dolt/issues/2559
This also moves the resolve databases rule into a different analyzer phase and fixes a string of issues that arise when that move occurs.
715: Add lag function.
Spec: LAG(expr, offset(default = 1), default(default=nil)).
712: INSERT IGNORE data conversion behavior
This PR addresses behavior for INSERT IGNORE with regards to data conversion errors.
1. Changes column value to 0 values on incorrect types
2. Truncates strings that are too long
  cc. https://dev.mysql.com/doc/refman/8.0/en/sql-mode.html#sql-mode-strict
708: sql/parse: Update sql/parse/parse.go for dropping vitess partial ddl support.
701: add DATEDIFF() func
697: sql/analyzer: Fix a bug where we do not use indexed joins for some queries with offset or sort+limit[+offset].
696: Fix boolean SQL conversion panic
Fixes a panic happening when returning Boolean values from the database.
690: Adding spatial types: point, linestring, and polygon
Able to create table with column of point, linestring, and polygon type.
Able to call some geometry constructors to create and display a point, linestring, and polygon.
Added functions: st_pointfromtext, st_linestringfromtext, st_polygonfromtext and tests
686: server/handler,sql/plan/row_update_accumulator: Move the logic for generating proper OkResult values for RowsAffected and InsertId to the engine.
This makes RowUpdateAccumulator responsible for returning the right OkResult,
instead of having the server/handler hack up the values in the result after the
fact. A few enginetests needed to be updated because they asserted the wrong
values for InsertId coming back in the OkResult.
It also makes RowUpdateAccumulator responsible for setting FoundRows, RowCount
and LastInsertId. LastInsertId handling is pretty gnarly becuase we currently
don't have it coming back from the insert iterator. It should be moved there.

vitess

109: go/mysql: Make all tests pass on macOS.
Bumps expected TLS version from 1.2 to 1.3.
Adds some string matching for strerror strings that are different between Linux
and macOS.
108: add support for constraint named unique
This is an enhancement for #95
According to https://dev.mysql.com/doc/refman/8.0/en/create-table.html, there will be a format like constraint namedx unique (full_name) in create table statement.
107: go/vt/sqlparser: Remove PartialDDL support.
There's no need to support partial DDL in dolt.
104: Window function frames, extent, and bounds
Adds parsing capabilities for window frames, including one sided frames, two sided frames, RANGE value frames, integral and interval based frames, and errors for semantically incorect combinations of frame options. Only value based intervals are currently supported. No prepared statement support for expression bounds.
103: go/mysql: conn: Add LoadInfile method which returns a ReadCloser for reading the remote file from a client.
We will migrate the implementation in go-mysql-server from HandleLoadDataLocalQuery to this interface. This keeps the responsibility for writing the file and the likes out of the vitess/conn layer.

Closed Issues

2520: Feature Request: timestampdiff sql function
2534: Panic on inserting empty tuple with trigger
2559: show tables produces inscrutable error in sql-client
2531: Feature request: lag
516: Datagrip cannot connect
2020: Non-deterministic import behavior on dolt table import
2519: Feature Request: datediff sql function
2544: Broken Link in hint text
2539: mysql client connection gets disconnected from shell when running queries
703: undefined: sql.NewPrimaryKeySchema

dolt - 0.35.0

Published by github-actions[bot] almost 3 years ago

Merged PRs

dolt

2538: doltcore/table/editor: index_edit_accumulator: Correctly order the most recently accumulated edits after all flushed edits.
IndexEditAccumulator has the same bug TableEditAccumulator had in #2278. The result is that large edits of indexes can drop index entries.
2530: Add DOLT_BRANCH() function.
2514: Add the ability to modify a table with AUTO_INCREMENT.
2511: Add prometheus metrics to sql-server.

go-mysql-server

699: server/handler.go: Fix a bug introduced in 0.34.9 which causes some LIMIT queries to fail non-deterministically under sql-server.
697: sql/analyzer: Fix a bug where we do not use indexed joins for some queries with offset or sort+limit[+offset].
682: Alter table can now modify the auto increment attribute of a column

Closed Issues

2536: Do not create temp_log file when running dolt sql-server
2442: read-only flag not being respected in config.yaml file
2492: Trying to connect with DBeaver to sql-server results in panic of sql-server
662: Automatically generate function docs

dolt - 0.34.9

Published by github-actions[bot] almost 3 years ago

Merged PRs

dolt

2507: Bump GMS and Vitess
2504: /go/libraries/doltcore/doltdb: Deleted key itr
2503: Clone sets upstream for main
2500: /go/libraries/doltcore/table/noms: Removed Noms Map Creator and Updater
2497: /go/libraries/doltcore/doltdb: Extracted Persistent Object from doltdb.Table
This refactor is the first step in supporting a new storage format. The primary focus of this change is to modify doltdb.Table to extract its backing types.Struct.
Throughout the doltdb package, in-memory objects are coupled with the persistent types objects that are used to de/serialize them to/from disk. The intention of this refactor and subsequent work is to decouple storage details from persistent objects. For example, schema.Schema is an in-memory abstraction that has a separate utility in the encoding package to read/write schemas to/from disk. This has created a clean separation of concerns for schema.Schema, and makes it easy to swap out serialization formats.
2495: Refactor the Export path to read from the engine
This pr removes the need of TableDataLoc from export and moves it towards an engine based table reader. It also modifies the writer implementations to work with sql.Row
TODO: I don't think export should support either --continue or --mapping. They were never tested nor are they relevant. If there is an error reading a row from a table, you should fail hard.
2494: bats test for failed window agg query
bats test and GMS bump for https://github.com/dolthub/dolt/issues/2448
https://github.com/dolthub/go-mysql-server/pull/680
2490: Fix auto fwt transform to not drop a row on flush
2488: Fixed bug with incorrectly applied index lookups for multi column indexes
2486: Improve 'Data Mover' code on import path
2485: Skipped test for Index overuse issue
Added skipped bats test for the issue where an index is used inappropriately when a function modifies the column producing incorrect results.
2482: add pager to dolt log
dolt log output is paginated
2472: Modified bats for SQL checkout to assert that the branch should not change on the CLI.
SQL Checkout should not change the branch in command line context. It only changes the branch for the session which ends after the statement. Resolves #2468.

2457: Add import benchmarks and add in import performance fix
Two noticeable problems.

No longer using BulkImportTableEditor. Needs to be threaded through in case the batch mode is set in the session.
Performance overhead from DoltRow to SqlRow conversion throughout the insert process. This should be reduced in the future as row.Row is deprecated
Current results

generating test file
Running the current version of import
Successfully initialized dolt data repository.
Rows Processed: 1000000, Additions: 1000000, Modifications: 0, Had No Effect: 0
Import completed successfully.
real	0m47.759s
user	1m16.795s
sys	0m25.848s
Running version 0.34.5
Rows Processed: 1000000, Additions: 1000000, Modifications: 0, Had No Effect: 0
Import completed successfully.
real	0m37.736s
user	1m12.005s
sys	0m30.281s

2369: Async push for replication
Async hook overview:
- Calling the commit hook (Execute) pushes a (dataset, database) tuple into a channel
- go routine 1 is responsible for reading from this channel, and setting the dataset id -> pull args in a shared map. Multiple commits to the same dataset override old values in this map.
- goroutine 2 has two phases that are run iteratively. Phase 1 copies the current state of the commit map into a local value. Phase 2 iterates through daatsets in the local commit map, pushing the most recent dataset commit if its hash is not the deduplication map's value. The deduplication map is private to gouroutine 2, and saved the pushed commit hash for each dataset.
  lifecycle management:
- new engine interface maintains BackgroundThreads, finalizing when we are done with the engine
  Initial perf comparison:
- PushOnWrite: 7.9 sec / op
- AsyncPushOnWrite: 6.5 sec / op
- diff ~ 20% speedup
  The perf comparison depends on many factors (the transaction throughput, commit size, network latency, etc), but this isolated test supports the idea that amortizing commits is better than individually pushing every commit. In practice, the time/memory overhead of coordinating the amortization with threads should always be smaller that the round trip of pushing every individual dataset.
  edge cases:
- commits to a dataset move backwards in time or delete values. The older commits contains chunks that fail to push (? but push synchronizes ancestors commits? so this should be OK unless we reset hard?)
- every commit creates a unique dataset id, and the amortization map blows up

go-mysql-server

689: Added test utility method for IndexTableAccess
684: sql/plan: load_data: Move LoadInfile logic from server/handler to the plan node.
This puts the LoadInfile functionality as a Service on the sql.Context and has
the plan node call into it. This is a better place for it in general, rather
than inspecting the resulting plan tree in server/handler and calling adhoc
methods on the *mysql.Conn based on what we find.
Also addresses a number of (potential) issues with the existing load data local
functionality including:
- Race-y file writes and reads from a single file (.LOADDATA) for concurrent
  requests.
- Incorrect handling of scanner.Err() checking; we now correctly return the
  errors when they occur, instead of deferring them to the Close() call.
- Avoids sending LoadInfilePacket for a non-LOCAL LOAD DATA call.
  The most complected part of the new implementation is the ordering constraint
  around RowUpdateAccumulator iterator calling Close() on the row source before
  it returns an Ok result from the Next() call. An alternative approach would be
  to copy the contents to a temporary file in LoadData.RowIter().
683: Changed Range overlap algorithm
Implemented a tree structure for ranges based on https://en.wikipedia.org/wiki/Interval_tree#Augmented_tree. I tested this against a benchmark that replicated the slowdown seen in Dolt release 0.34.5 and saw linear scaling, just as we'd hope. There are some cases that are handled very inefficiently, and I left some strategies on how to solve them listed in the comments, which I will probably implement at a later time.
681: Return Uint64 instead of int64 in charsets table
680: Window aggregation buffer sharing test
Re: https://github.com/dolthub/dolt/issues/2448
Dolt diff table aggregation queries were bleeding results. We still use buffer sharing, but this test should prevent regressions for the same issue.
679: server/handler.go: Migrate doQuery rows loop to errgroup.
Previously this used a |quit| signaling channel and an |errChan| to
communicate any errors. We didn't get context cancelation in all cases and our
call to |rows.Close()| raced with the background thread's call to
|rows.Next()|.
678: server,sql/p{arse,lan}: Move handling of KILL statements to plan nodes instead of adhoc regex parsing in server handler.
676: Bug fix for multi-column index matching and test for same
675: Design for RowFrame

673: fix cross join panic
The cross join expression check would panic for any Comparator outside of the whitelist, instead of returning false to indicate an unmatchable expression.

> SELECT a6, x5, b2+a6, e4  FROM t6, t4, t5, t2 WHERE b4 in (434,267,535,407,331,2,921,469)   AND c5=819   AND d6 in (192,129,970,151)   AND (c2=728 OR 806=c2 OR 141=c2)
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x1e2c6d3]
goroutine 1 [running]:
github.com/dolthub/go-mysql-server/sql/expression.(*GetField).Name(...)
/Users/vinairachakonda/go/pkg/mod/github.com/dolthub/[email protected]/sql/expression/get_field.go:82
github.com/dolthub/go-mysql-server/sql/analyzer.comparisonSatisfiesJoinCondition({0x2d03fbe8, 0xc0013df5e0}, 0xc0013df580)
/Users/vinairachakonda/go/pkg/mod/github.com/dolthub/[email protected]/sql/analyzer/replace_cross_joins.go:50 +0x2d3
github.com/dolthub/go-mysql-server/sql/analyzer.expressionCoversJoin.func1({0x29e5070, 0xc0013df5e0})
/Users/vinairachakonda/go/pkg/mod/github.com/dolthub/[email protected]/sql/analyzer/replace_cross_joins.go:61 +0x5b
github.com/dolthub/go-mysql-server/sql/expression.InspectUp.func1({0x29e5070, 0xc0013df5e0})
/Users/vinairachakonda/go/pkg/mod/github.com/dolthub/[email protected]/sql/expression/transform.go:61 +0x39
github.com/dolthub/go-mysql-server/sql/expression.InspectUp.func2({0x29e5070, 0xc0013df5e0})
/Users/vinairachakonda/go/pkg/mod/github.com/dolthub/[email protected]/sql/expression/transform.go:68 +0x27
github.com/dolthub/go-mysql-server/sql/expression.TransformUp({0x29e5070, 0xc0013defc0}, 0xc00016dfc8)
/Users/vinairachakonda/go/pkg/mod/github.com/dolthub/[email protected]/sql/expression/transform.go:53 +0x13d
github.com/dolthub/go-mysql-server/sql/expression.InspectUp({0x29e5070, 0xc0013defc0}, 0xc00016e020)
/Users/vinairachakonda/go/pkg/mod/github.com/dolthub/[email protected]/sql/expression/transform.go:67 +0xad
github.com/dolthub/go-mysql-server/sql/analyzer.expressionCoversJoin({0x29e5070, 0xc0013defc0}, 0x3cd6108)
/Users/vinairachakonda/go/pkg/mod/github.com/dolthub/[email protected]/sql/analyzer/replace_cross_joins.go:58 +0x3f
github.com/dolthub/go-mysql-server/sql/analyzer.replaceCrossJoins.func1.1({0x29d3b00, 0xc0013df580})
/Users/vinairachakonda/go/pkg/mod/github.com/dolthub/[email protected]/sql/analyzer/replace_cross_joins.go:93 +0x318
github.com/dolthub/go-mysql-server/sql/plan.TransformUp.func1({{0x29d3b00, 0xc0013df580}, {0x29d3b00, 0xc0013df440}, 0x0, {0x37ff6f8, 0x0, 0x0}})
/Users/vinairachakonda/go/pkg/mod/github.com/dolthub/[email protected]/sql/plan/transform.go:98 +0x4f
github.com/dolthub/go-mysql-server/sql/plan.transformUpCtx({{0x29d3b00, 0xc0013df420}, {0x29d3b00, 0xc0013df440}, 0x0, {0x37ff6f8, 0x0, 0x0}}, 0x0, 0xc00016e778)
/Users/vinairachakonda/go/pkg/mod/github.com/dolthub/[email protected]/sql/plan/transform.go:91 +0x6ba
github.com/dolthub/go-mysql-server/sql/plan.transformUpCtx({{0x29d3b00, 0xc0013df440}, {0x29d3b00, 0xc0013df460}, 0x0, {0x37ff6f8, 0x0, 0x0}}, 0x0, 0xc00016e778)
/Users/vinairachakonda/go/pkg/mod/github.com/dolthub/[email protected]/sql/plan/transform.go:72 +0x41f
github.com/dolthub/go-mysql-server/sql/plan.transformUpCtx({{0x29d3b00, 0xc0013df460}, {0x29d3fc8, 0xc0013df480}, 0x0, {0x37ff6f8, 0x0, 0x0}}, 0x0, 0xc00016e778)
/Users/vinairachakonda/go/pkg/mod/github.com/dolthub/[email protected]/sql/plan/transform.go:72 +0x41f
github.com/dolthub/go-mysql-server/sql/plan.transformUpCtx({{0x29d3fc8, 0xc0013df480}, {0x0, 0x0}, 0xffffffffffffffff, {0x37ff6f8, 0x0, 0x0}}, 0x0, 0xc00016e778)
/Users/vinairachakonda/go/pkg/mod/github.com/dolthub/[email protected]/sql/plan/transform.go:72 +0x41f
github.com/dolthub/go-mysql-server/sql/plan.TransformUpCtx({0x29d3fc8, 0xc0013df480}, 0x100c00016e7e8, 0x100e674)
/Users/vinairachakonda/go/pkg/mod/github.com/dolthub/[email protected]/sql/plan/transform.go:53 +0xbd
github.com/dolthub/go-mysql-server/sql/plan.TransformUp({0x29d3fc8, 0xc0013df480}, 0x203000)
/Users/vinairachakonda/go/pkg/mod/github.com/dolthub/[email protected]/sql/plan/transform.go:97 +0x45
github.com/dolthub/go-mysql-server/sql/analyzer.replaceCrossJoins.func1({0x29d3fc8, 0xc0013df480})
/Users/vinairachakonda/go/pkg/mod/github.com/dolthub/[email protected]/sql/analyzer/replace_cross_joins.go:85 +0x159
github.com/dolthub/go-mysql-server/sql/plan.TransformUp.func1({{0x29d3fc8, 0xc0013df480}, {0x29d45f8, 0xc0013cd200}, 0x0, {0x37ff6f8, 0x0, 0x0}})
/Users/vinairachakonda/go/pkg/mod/github.com/dolthub/[email protected]/sql/plan/transform.go:98 +0x4f
github.com/dolthub/go-mysql-server/sql/plan.transformUpCtx({{0x29d3fc8, 0xc0013df2e0}, {0x29d45f8, 0xc0013cd200}, 0x0, {0x37ff6f8, 0x0, 0x0}}, 0x0, 0xc00016eca0)
/Users/vinairachakonda/go/pkg/mod/github.com/dolthub/[email protected]/sql/plan/transform.go:91 +0x6ba
github.com/dolthub/go-mysql-server/sql/plan.transformUpCtx({{0x29d45f8, 0xc0013cd200}, {0x0, 0x0}, 0xffffffffffffffff, {0x37ff6f8, 0x0, 0x0}}, 0x0, 0xc00016eca0)
/Users/vinairachakonda/go/pkg/mod/github.com/dolthub/[email protected]/sql/plan/transform.go:72 +0x41f
github.com/dolthub/go-mysql-server/sql/plan.TransformUpCtx({0x29d45f8, 0xc0013cd200}, 0x1d9c087, 0xc0013b6ec0)
/Users/vinairachakonda/go/pkg/mod/github.com/dolthub/[email protected]/sql/plan/transform.go:53 +0xbd
github.com/dolthub/go-mysql-server/sql/plan.TransformUp({0x29d45f8, 0xc0013cd200}, 0xc0007dbb01)
/Users/vinairachakonda/go/pkg/mod/github.com/dolthub/[email protected]/sql/plan/transform.go:97 +0x45
github.com/dolthub/go-mysql-server/sql/analyzer.replaceCrossJoins(0x26a3cc2, 0x26a0458, {0x29d45f8, 0xc0013cd200}, 0xc0013cd200)
/Users/vinairachakonda/go/pkg/mod/github.com/dolthub/[email protected]/sql/analyzer/replace_cross_joins.go:78 +0x45
github.com/dolthub/go-mysql-server/sql/analyzer.(*Batch).evalOnce(0x29d45f8, 0xc00096d740, 0xc0007dbb60, {0x29d45f8, 0xc0013cc5d0}, 0x104e534)
/Users/vinairachakonda/go/pkg/mod/github.com/dolthub/[email protected]/sql/analyzer/batch.go:93 +0x1e7
github.com/dolthub/go-mysql-server/sql/analyzer.(*Batch).Eval(0xc0006b9350, 0x37ff6f8, 0xc0007dbb60, {0x29d45f8, 0xc0013cc5d0}, 0x1)
/Users/vinairachakonda/go/pkg/mod/github.com/dolthub/[email protected]/sql/analyzer/batch.go:54 +0x145
github.com/dolthub/go-mysql-server/sql/analyzer.(*Analyzer).analyzeWithSelector(0xc0007dbb60, 0xc00054eb40, {0x29d45f8, 0xc00096d740}, 0x0, 0x271def0)
/Users/vinairachakonda/go/pkg/mod/github.com/dolthub/[email protected]/sql/analyzer/analyzer.go:376 +0x314
github.com/dolthub/go-mysql-server/sql/analyzer.(*Analyzer).Analyze(...)
/Users/vinairachakonda/go/pkg/mod/github.com/dolthub/[email protected]/sql/analyzer/analyzer.go:348
...

672: Automatically Generate README File
Doesn't include argument types for functions.
Janky solution to creating Instances of FunctionN.
Might be difficult to edit README now.
668: fix _example/main.go sql.Schema -> sql.PrimaryKeySchema
667: sql/parse: Remove adhoc parsing of certain structures with regexes and custom lexers.
Moves the following to vt/sqlparser:
- SHOW VARIABLES
- SHOW WARNINGS
  Gets rid of unnecessary special handling for SET and SHOW PROCESSLIST.
665: SqlEngine tracks BackgroundThreads
SqlEngine manages the lifecycle of async background threads through a
BackgroundThreads object. Integrators can add goroutines before or
after SqlEngine initialization, but finalizing the engine will terminate
the threads.
Async goroutines are expected to terminate when the context provided by
the thread manager is cancelled. The BackgroundThreads instance uses a
wait group to allow threads to drain gracefully before exiting.
663: Restrict HASH IN conversion to supported operators.
We were converting TupleIn to HashIn for expressions that HashIn could
not evaluate. This change makes explicit which left and right children
we permit for HashIn conversion, and expands HashIn.Eval to evaluate
those expression classes. Added expression unit tests, analyzer unit
tests, query enginetests, and query planner enginetests for more sets of
expressions that we support and do not support.
Additionally, this simplifies the expression evaluation hashing
technique. The new behavior has not been verified with deeply nested
tuple expressions.

vitess

103: go/mysql: conn: Add LoadInfile method which returns a ReadCloser for reading the remote file from a client.
We will migrate the implementation in go-mysql-server from HandleLoadDataLocalQuery to this interface. This keeps the responsibility for writing the file and the likes out of the vitess/conn layer.
102: Added GRANT, REVOKE, and the remainder of CREATE USER
101: go/vt/sqlparser: Parse KILL {CONNECTION,QUERY} statements.
100: go/vt/sqlparser: Extend the parser for correctly parsing some extended SHOW syntax.
- SHOW COUNT(*) {WARNINGS, ERRORS}
- SHOW {WARNINGS,ERRORS} LIMIT ...[, ...]
- SHOW {,SESSION,GLOBAL} VARIABLES {LIKE,WHERE} ...
- SHOW FULL PROCESSLIST
99: Added more statements for user handling
Added RENAME USER, CREATE ROLE, DROP ROLE, SHOW GRANTS, and a portion of CREATE USER. The rest of CREATE USER, along with GRANT and REVOKE will come in the next PR.
97: Added proper syntax parsing for account names, along with DROP USER
This PR primarily implements the changes necessary for account names, which required edits in the parser itself. DROP USER was implemented to test the name variations, so refer to the tests in parse_test.go for how account names work (and to see why the changes were necessary). All parser tests were validated against a MySQL 8.0 instance.
In a nutshell, account names take the form user@host, and both user and host must be separate strings. The parser treated @ as a standard letter, and we used regex matching in GMS and Dolt to catch "invalid" names (@@system_var is valid but what@@ is invalid). My first pass was to treat the entire account name as a string and split it using a regex, however I could not get it to pass all of the parser tests. This was the only way I could make it work with all tests passing.
Notes:
- The other statements will come in a different PR, which will be straightforward in comparison.
- DROP USER ''; looks like weird syntax, but an empty name means that all names match. In this case, you can assign some dbs/tables to be available to everyone.
- Following the above, DROP USER ``; is valid, but failed as we returned an error on empty quoted identifiers. MySQL allows empty quoted identifiers to parse, but rejects them as invalid names depending on the query. This has been fixed.
- Added ast_permissions.go to put all of the statements dealing with users, grants, etc. For me, ast.go operates very slowly and occasionally hangs for a few minutes, so I'm putting my new statements in a new file for now. I doubt you'll have pushback, but if so I can merge the file back into ast.go when I'm done.
96: Better support for MySQL MULTI_STATEMENTS.

Closed Issues

2122: dolt should invoke the pager for long output
2448: Window functions returning incorrect results
2478: Dolt diff misses some rows
2484: Index breaks query when using lower()
2468: dolt sql -q "select dolt_checkout('main')" does not checkout main
664: _example/main.go unable to compile
653: HASH IN fails on arithmetic, function nodes

dolt - 0.34.8

Published by github-actions[bot] almost 3 years ago

Merged PRs

dolt

2467: Fixed several problems in generated docs
2459: fix: userNameRequiredError message in CLI
Running dolt init before configuring user.email and user.name results (correctly) in error; however, the CLI hint itself is incorrect:
```
Author identity unknown
*** Please tell me who you are.
Run
dolt config --global user.email "[email protected]"
dolt config --global user.name "Your Name"
to set your account's default identity.
Omit --global to set the identity only in this repository.
fatal: empty ident name not allowed
```
The proposed commands for identity configuration do not include the --add flag, resulting in a new error message: Exactly one of the -add, -get, -unset, -list flags must be set.
This PR fixes the CLI message by adding the --add flag to the proposed commands.
2449: sort dolt blame result by pk
added sort function in dolt blame, uses first pk to sort if there are multiple pk
2446: default PrimaryKeySchema, maintain primary key ordinals
Use the new GMS sql.PrimaryKeySchema, implement new PrimaryKeyAlterableTable methods, add new TestPkOrdinals enginetests.
Other datails:
- schemaImpl and the serialized schema has a new field for PK ordinals
- support backwards compatibility by using the column definition order for PKs by default
- addPk, dropPk, addColumn, dropColumn, and modifyColumn all track pkOrdinal changes
  Future:
- some methods still return sql.Schema for printing/vizualization
- diffing/merging are not changed (partially because SHOW CREATE TABLE does not print the correct order still)
2443: stop ignoring read_only flag
2439: add force flag to dolt checkout
added -f 'force' flag to dolt checkout. with force flag, the current changes will be wiped

go-mysql-server

663: Restrict HASH IN conversion to supported operators.
We were converting TupleIn to HashIn for expressions that HashIn could
not evaluate. This change makes explicit which left and right children
we permit for HashIn conversion, and expands HashIn.Eval to evaluate
those expression classes. Added expression unit tests, analyzer unit
tests, query enginetests, and query planner enginetests for more sets of
expressions that we support and do not support.
Additionally, this simplifies the expression evaluation hashing
technique. The new behavior has not been verified with deeply nested
tuple expressions.
661: server,sql/parse: ComQuery, Parse updates for better multi statement support.
657: Fix mysql commands in README
656: Removing unnecessary context creation
651: Fixed IS NOT NULL and IN not being recognized with composite indexes
Fixes https://github.com/dolthub/go-mysql-server/issues/650
648: Added check for conflicting column definitions
Checks for attempt to create primary key on nullable column
Added tests in parse_test.go for new error.
642: Track PRIMARY KEY column ordering with a sql.PrimaryKeySchema struct.
The parser exposes PRIMARY KEYs in the TableSpec for CREATE TABLE,
which we pluck before planning. Integrators expected to use the new
sql.PrimaryKeySchema even for keyless tables.
CREATE TABLE LIKE and CREATE TABLE SELECT are handled separately
and need more work to support the same order tracking.
sql.PrimaryKeyAlterableTable now includes PrimaryKeySchema() and Pks()
methods to access the new schema info.

vitess

96: Better support for MySQL MULTI_STATEMENTS.
95: Added support for CONSTRAINT UNIQUE KEY syntax in CREATE TABLE
94: add Null to column definition
create table t (a int primary key) succeeds
create table t (a int not null primary key) succeeds
create table t (a int null primary key) fails
Create new BoolVal field for ColumnDefinition to more easily check for attempts to create nullable columns with primary key.
Also allows people to do create table t(a int null not null ...) if they want.

Closed Issues

2427: [Readme improvement] - we need better migrate explanation.
2324: Merging schemas that look the same in the CLI drops error if NOT NULL constraints were defined slightly differently.
1830: add --force option to dolt checkout
1463: dolt blame columns headers are shifted to the left by one
618: Unhex function returns a string, when it should return []byte
650: IS NOT NULL and IN are causing composite index to be ignored

dolt - 0.34.7

Published by github-actions[bot] almost 3 years ago

Merged PRs

dolt

2454: go/cmd/dolt: commands/sqlserver: Add configuration for disabling MULTI_STATEMENTS support on the server.
Currently sql-server parses some MULTI_STATEMENTS incorrectly, in particular
statements that include substatements get split up incorrectly. This adds
server configuration for the yaml config, so that multi-statement handling can
be disabled server-side. This will allow things like CREATE TRIGGERs submitted
with a client that advertises MULTI_STATEMENTS support to work correctly with
Dolt sql-server. However, if the client actually sends a multi-statement in
this mode, even two SELECTs separated by a ';', for example, while the server
is running in this mode, it will not work.
To enable, place:
```
behavior:
  disable_client_multi_statements: false
```
in the config.yaml for the server.
2435: Adding columns to primary key now adds not null constraint
Also added a test case to ensure adding not null constraint to primary key column does nothing.

go-mysql-server

649: server: Add option to Config to DisableClientMultiStatements.
648: Added check for conflicting column definitions
Checks for attempt to create primary key on nullable column
646: Faster integral type conversions.
We spend up to 10% of sysbench tests in this function.
645: Clean up some unnecessary context creations, which can be expensive.

vitess

94: Add null to supported syntax for column definitions
93: go/mysql/conn.go: Add a way to disable client_multi_statements handling on a per-connection basis.
Conn's handling of CLIENT_MULTI_STATEMENTS is currently wrong for packets that
contain statements which can include other statements (e.g., CREATE TRIGGER).
Add a way for an integrator to disable it for now.
92: Add support for empty tuple in INSERT INTO ... VALUES ()
Now correctly parses
insert into test_tb values ()
insert into test_tb () values ()
insert into test_tb values (), (), ...
insert into test_tb () values (), (), ...

dolt - 0.34.6

Published by github-actions[bot] almost 3 years ago

Merged PRs

dolt

2444: proto: Bump eventsapi proto. Pick up definition for ClientEventType_DUMP.
2438: Export method needed for dolthub fix
2437: parquet reader for import update and replace
Added parquet reader for import update and import replace only, for import create, schema file is required.
2432: go/libraries/doltcore/table/untyped/xlsx: Fix bug where this spammed internal representation of the imported rows to Stdout.
These lines were not visible from the CLI because of color, but it still cost
performance. In an embedded context, these could end up in stdout.
2431: Fix issue with not clearing conflicts correctly
Added change in conflict_source.go to just return ErrNoConflicts when there are no conflicts rather than printing an empty table.
Added extra check in resolve.go to call ClearConflicts() when there are zero rows in conflict after resolving table.
Removed skip statement from bats test that checked for these behaviors.
Edited case-sensitivity test for selecting dolt_conflicts_test, as that now returns a ErrNoConflicts.
2426: Move Import Write Path to Engine
2403: parquet file type for export
added parquet file type for export, write only

go-mysql-server

645: Fix Context Creation
641: Expanded index matches, added many index queries
Besides the obvious addition of a ton of tests (ALL of which make use of indexes, which are enforced by the EXPLAIN statements), this also allows for another scenario to match indexes, although we're still missing a fairly important one. I'll list the scenarios below (assume an index over columns (v1, v2, v3).
The new one we now match: previously expressions had to match the index prefix, but now it has been expanded to require only the first index expression be matched. Therefore, the filter WHERE v1 = 5 AND v3 = 2 is now used by indexes as we can still filter on v1 = 5 using the same principles that partial indexes use. v3 = 2 may not allow for rows to be skipped on the integrator's side when reading from their internal store, but may allow for them to be skipped when returning to the engine (removing the conversion to a sql.Row from the integrator's implementation which should be a fairly substantial speed increase).
This has the added consequence that we can properly combine all ranges that make use of a specific index. That is,
```
(v1 > 1 AND v3 > 1) OR (v1 < 4 AND v2 < 4)
```
will now combine the two ranges (separated by OR). Reducing the prefix limitation has greatly increased the index match possibility. Just to note, the returned indexes are sorted by whether they're an exact match, then by how many expressions are a prefix, then by the index size, and lastly by the index name, so you should always get the "best" index for any given set of expressions.
The above works in the case of a single index existing over the three columns. If (v1, v3, v2) is another index then the above example will match to two different indexes (both being a perfect prefix to an index). There is a way to get around this by taking note of which index has been used previously if multiple indexes share the same columns (or some indexes are subsets/supersets of some other indexes), however this was attempted and abandoned due to the added complexity and time it would take to implement.
Also built off of this, we cannot match
```
(v1 > 1 AND v3 > 1) OR (v2 < 4)
```
As the (v2 < 4) is missing the first index expression (v1), therefore no index is used here (one is used by MySQL, which was tested to confirm that one should be in this case). Again this would be fixed by taking note of which indexes have already been used, as we only need the prefix to be matched one, and then all other sets of expressions just need to be subsets.
Either way, we should now match many more queries than previously.
640: Add utilities needed by Dolt import path
638: Add support for inserting empty value tuples
There was an edge case where if columns were not provided, it required there to be exactly num columns for value tuples, even if there are default values for those columns.
Added a helper function existsNonZeroValues to avoid this.
637: crossjoins converted to inner join if parent filter has suitable join condition
Filter predicates can be pushed into Join nodes under certain conditions:
- Filter is the parent of a CrossJoin
- Predicate expressions cover the two Join node dependency trees
  Filter predicates have to be divided specially to maintain logical equivalence. Only AND conjugates can be split and checked individually for join candidacy. If multiple conjugate trees match the join expression, a new conjugate AND of the subtrees are used as the join condition.
  Nested CrossJoins should be converted to InnerJoins if the parent filter has suitable join comparisons for each.
636: Fixed small issue with used expressions and partial indexes

vitess

92: parser can detect insert into () values <tuple_list>
added new case for insert_data
should be able to correctly parse
insert into test_tb values ()
insert into test_tb () values ()
insert into test_tb values (), (), ...
insert into test_tb () values (), (), ...
91: added FORMAT keyword
added format expression keyword

Closed Issues

2390: Table with new columns not detected as modified
2336: Resolving conflicts without committing does not allow you to continue to modify the working set
2420: Support empty VALUES, column list on INSERT
2381: Dolt dump to parquet
643: pls dont rush the API 1.0 but do think properly and optimize for performance
172: Support for prepared statements

dolt - 0.34.5

Published by github-actions[bot] almost 3 years ago

Merged PRs

dolt

2428: /go/cmd/dolt/commands: Updated cli docs for dolt migrate
2423: More intelligent drop / create for tables
This addresses #2390 and other issues.
Previously we were only reusing dropped tables if they had exactly the same schema upon recreation. This PR loosens that constraint.
2402: Refactor the cli's SQL Engine to its own package

Closed Issues

2329: Should not be able to add primary key when cells have null value

dolt - 0.34.4

Published by github-actions[bot] almost 3 years ago

Merged PRs

dolt

2411: Add support for DROP DATABASE. Improved support for creating and dropping dbs when using the --multi-db-dir flag.
2376: Better printing for []byte fields.
2372: Create release binaries for Apple M1 machines; use them in install.sh.

dolt - 0.34.3

Published by github-actions[bot] almost 3 years ago

Merged PRs

dolt

2413: Prevent creating Primary Key using columns that contain null values
2406: replication with multi-db mode
- Multidb uses global dolt config
- Each database locally sets the url for the named remote
- Error handling and async will work the same as single db
2405: Optimized tuple checks on index reads
2392: Bump GMS and Unskip LOAD DATA sql-server bats test
2384: /go/store/nbs: Optimize NBS rebase

go-mysql-server

635: plan,analyzer: Implement a TopRowsNode, optimizing ORDER BY + LIMIT{,OFFSET} queries like SELECT * FROM table ORDER BY id LIMIT 100 OFFSET 100
634: Fixed bug in string -> integer conversions for large integers (caught by dolt)
633: Implement REGEX_REPLACE
632: Fixed some expressions not using the largest partial index
630: Accept string float values for integer values
628: Fix the global variable usage of load data. Re-add LOAD DATA tests
627: add FORMAT() locale
626: Add ability to specify precision to current_timestamp function
622: implement FORMAT() function
609: Reduced index range complexity, removed range overlapping

Closed Issues

2416: Use of dependency vulnerable to hijacking
2345: Implement REGEXP_REPLACE
2368: unsupported CURRENT_TIMESTAMP(3)
179: Combine index lookups on different columns of multi-column indexes with different operators

dolt - 0.34.2

Published by github-actions[bot] almost 3 years ago

Merged PRs

dolt

2388: go/store/nbs: Fix bug in table_reader where max read size was 128GB instead of 128MB.

Closed Issues

2360: Implement IP SQL Functions
2334: Custom escape characters are not supported?
2375: byte[] prints incorrectly
2348: No support for DROP TABLE CASCADE
1961: Shell completions for bash/zsh/fish

dolt - 0.34.1

Published by github-actions[bot] almost 3 years ago

Merged PRs

dolt

2382: Added support for indexes to use mixed operators, bug fix for returning duplicate rows
Tests covering mixing operators will come in a separate PR.
2379: Command to generate a zsh autocomplete file

go-mysql-server

617: hash in supports tuples
- improve the hash function to handle tuples recursively
- normalize the left side of the IN comparison so GetField is converted to Literal
- analyzer will explicitly fail for HASH IN that should convert doesn't (should it still implicitly fall back?)
  questions:
- what to do about tuple type conversion? should we store the explicit struct of types and validate as we walk the tree while hashing?
- am i missing any edge cases i need to test?
- add max_allowed_packet = 67108864 system variable to limit set size?

vitess

91: added FORMAT keyword
added format expression keyword
90: Noop behavior for DROP TABLE CASCASE/RESTRICT
89: Added Test Cases for LIKE ... ESCAPE ...

Closed Issues

651: Implement FORMAT

dolt - 0.34.0

Published by github-actions[bot] almost 3 years ago

Merged PRs

dolt

2371: dolt push - add missing newline when printing results
2364: Use a fallback username and email when creating databases in SQL.
2357: Stubbed drop DB support and improvements in SQL commands for multiple repos.
2356: Add JSON format to dump command.
2350: Constructing a mysql connection with database/branch syntax or calling use <database>/<branch> in replicated database mode will attempt to fetch the new branch.
2347: dolt fetch - remove empty newline printed when fetching.
2346: DOLT_REPLICATE_HEADs_STRATEGY option, fetch all branches from remote before every read.
2337: dolt_skip_replication_errors config option. Allow erroring in certain cases of misconfigured or unavailable replication, rather than erroring in a way that makes the sql-server unavailable.

go-mysql-server

621: Implement IP Functions
Functions Added:
- inet_aton
- inet_ntoa
- inet6_aton
- inet6_ntoa
- is_ipv4
- is_ipv6
- is_ipv4_compat
- is_ipv4_mapped
619: UNHEX now returns []byte
617: Support tuples when using a hash to evaluation WHERE ... IN (...) in some cases.
615: Use a hash map for IN expressions against literals.
614: Fix for joins on columns that are not a prefix to an index
612: Added ability to specify custom escape characters for LIKE
608: TRIM string functions
605: Added Support for Full MySQL TRIM Syntax
Parser recognizes keywords like: LEADING, TRAILING, and BOTH.
TRIM function can now remove prefixes.
Fixed LTRIM And RTRIM to not remove all whitespace, and just space characters.
Also added functionality for RIGHT function.
604: Fix Panic in unique key error generation

vitess

90: Noop behavior for DROP TABLE CASCASE/RESTRICT
88: Add TRIM functionality
85: can use status w/o quotes for insert statement

Closed Issues

2366: Regression in indexed joins, zero length key and panic
2370: dolt push does not make a new line if it pushes something
1768: dolt fetch prints extra new lines
2202: dolt export command
2353: Feature Request: Make git log a table so that we can perform SQL queries on it
2295: The database fields are case-sensitive?
562: Intermittent DATA RACE error when creating MySQL Servers
603: Panic when using memory implemenation

dolt - 0.33.1

Published by github-actions[bot] almost 3 years ago

Merged PRs

dolt

2349: /go/cmd/dolt/dolt.go: fix dolt version format

Closed Issues

dolt - 0.33.0

Published by github-actions[bot] almost 3 years ago

Merged PRs

dolt

2343: doltdump - add csv format
added csv file format for dolt dump command
2342: go/libraries/doltcore/table/editor: Thread some TupleFactories to places that allocate tuples.
Makes types.TupleFactory thread-safe. Allocates a new TupleFactory when we construct a SessionTableEditor, and threads that TupleFactory through to individual table editors that can use it to do their editing. Makes that TupleFactory available through an accessor that sqlTableEditor uses for a few tuple manipulations.
2341: dolt session methods error when no config (for DoltHub)
2340: dolt_fetch error when working directory != server directory
2339: dolt dump: Add dolt dump command and bats tests.
2331: Add skipped bats for adding a primary key to a column with NULL in it
Linked to #2329
2326: CREATE DATABASE now creates a new dolt repository in the working directory
This is a first implementation of durable CREATE DATABASE support. It always creates a new database in the same directory that the server was run from, as a separate repository. It works in both server and command line mode.
Future extensions:
- Configurable DOLT_DATA_DIR variable
- Create database in the same repository (same commit graph)
2319: First Iteration of Application Server Deploy Script
2317: created 'dolt dump' command
fixed github issue#2202 - created dolt dump command that dumps all tables to doltdump.sql file in current directory.
2316: Jennifer/fix 2202
2307: config --server flag and refactors
- sql-server variables live in ./dolt/config.json with a sqlserver.globals. prefix
- initialize persisted system variables in cli and sql contexts
- standardize variable access through sql.SystemVariables singleton
- disable replication for cli and sql -q commands, unless dolt_engine_mode = 'permissive' is set (used a lot in the bats tests)
2276: Updated the index interface
2270: default session implements persistable interface, can save SQL variables

go-mysql-server

606: Changed some error names
602: Add case insensitivity to insert
601: Fix panic in json_unquote for null values
This supercedes https://github.com/dolthub/go-mysql-server/pull/511 and adds tests
595: persistent session interface
585: Changed index interface, support partial keys

Closed Issues

2297: Implement TRIM and other string functions
584: Proxy support?

dolt - 0.32.2

Published by github-actions[bot] almost 3 years ago

Merged PRs

dolt

2314: Fixed docs markdown generation and fixed a number of related formatting errors
2303: Expand dolt client tests to include Dolt-specific SQL functions
2302: checkout,sql-server: Make dolt checkout and sql-connect-to-branch functionality available even when the configured default branch for the sql-server or CLI do not exist.
2301: dolt_branches INSERT/UPDATE commit replication
Execute commit hooks on dolt_branch INSERT and UPDATE, allowing for replication outside of atomic dolt commits.
2298: Make sql shell context uses the session stored database.
2288: Get DoltMerge to ignore ErrIsAhead instead of error
2155: support oldgen table files in roots command

go-mysql-server

601: Fix panic in json_unquote for null values
599: Remove the Index Editor Sorter and add alias support for Update Join
597: sql/analyzer: pushdown_filters: Avoid pushing down filters below Window nodes.
596: Add Right/left/Cross/Indexed Joins and Implement Table Edit Accumulator

Closed Issues

2262: Dolt Add Commit Statements not working in the MySQL Connector .Net
2291: System table "dolt_status" returns different results
2190: CREATE DATABASE on shell is broken
2189: Allow combining table alias with AS OF clause in JOIN clause
1623: Dolt: Allow partial commit hashes in commands
2284: select dolt_merge('other'); returns error code for "up to date" status

dolt - 0.32.1

Published by github-actions[bot] almost 3 years ago

A bug fix release which includes fixes for UPDATE queries in BEGIN blocks of a trigger and filter clauses on subqueries that include Window functions.

Merged PRs

go-mysql-server

598: sql/analyzer: pushdown.go: Avoid pushing down projects across BEGIN blocks.
597: sql/analyzer: pushdown_filters: Avoid pushing down filters below Window nodes.
594: sql/plan: Fix join iterators to always Close() their secondary RowIter. Fix Subquery to Dispose its subquery.

dolt - 0.32.0

Published by github-actions[bot] almost 3 years ago

Merged PRs

dolt

2278: go/store/types/edits: Changes to ensure AsyncSortedEdits always returned edits in stable order based on how they were added. Fixes UPDATE statements which dropped rows under certain conditions.

go-mysql-server

594: sql/plan: Fix join iterators to always Close() their secondary RowIter. Fix Subquery to Dispose its subquery. Fixes some memory leaks in sql-server mode.
593: Make some update queries determinate.
591: Introduced a ViewProvider extension.
As part of this, got rid of the IndexRegistry and ViewRegistry on the context, put them on the Session instead. These changes significantly simplify the process of constructing an engine and running queries.

Closed Issues

2282: How to get schema diff between two commits using dolt system table
2277: [feature-request] dolt dump

dolt - 0.31.0

Published by github-actions[bot] about 3 years ago

Merged PRs

dolt

2268: Simpler engine construction
No manual management of views and index registries.
2255: Added scripted SQL tests that use DOLT_COMMIT
2254: Prevent pull from erroring on config with no email or name:
2252: go/store/data: Use new parents_closure field for FindCommonAncestor.
2251: Reworked skipped fk test, display unresolved foreign keys
2247: Added test of working set behavior for checkout
Skipped test for #2246
2245: /go/store/types: Extracted sequenceSplitter interface for rollingValueHasher
2235: go/store/datas: Materialize a parents closure map at commit time.
2224: Allow MySQL dumps to delay foreign key resolution

go-mysql-server

591: Introduced a ViewProvider extension
As part of this, got rid of the IndexRegistry and ViewRegistry on the context, put them on the Session instead. These changes significantly simplify the process of constructing an engine and running queries.
590: Add dolt discord to readme
588: Made it possible to use variables in AS OF expressions
As part of this, also pulled resolving variables out of the resolve_columns step into its own rule.
583: Update INNER JOIN Alpha
582: sql/plan: exchange.go: Make Exchange's RowIter wait for all goroutines to shutdown cleanly.
If we do not block on shutting down the goroutines that are handling the
partitions, we cannot guarantee that we will not race with later uses of the
sql.Session, for example.
This converts the implementation to x/sync/errgroup and restructures things
quite a bit.
581: Skip fk validation on CREATE TABLE when fks are disabled

vitess

86: Add transaction characteristic to begin transaction
85: can use status w/o quotes for insert statement
re: https://github.com/dolthub/dolt/issues/2195

Closed Issues

2195: Allow using unquoted status column identifier in INSERT statements
2249: dolt pull is trying to do a commit for a ff merge
2237: dolt table import -u businesses WA.csv fails with fatal error: out of memory
1669: foreign_key_checks = 0 should allow out-of-order creation of foreign key constraints
586: memory/provider.go doesn't use a pointer receiver causing the struct to be copied