Bot releases are visible (Hide)
Published by github-actions[bot] about 3 years ago
AS OF
clauses.value
and status
keywordsPublished by github-actions[bot] about 3 years ago
io/ioutil
package has been deprecated in Go 1.16 (See https://golang.org/doc/go1.16#ioutil).dolt backup
cli
RootSync
command, which backups are based off ofdsqle.ReadReplicaDatabase
which implements dsqle.SqlDatabase
dsqle.Database
to accommodate all dsqle.SqlDatabase
implementationsdb.StartTransaction
to execute a fetch on DOLT_READ_REPLICA_REMOTE
ReadReplicaDatabase
is -not- read only right now, need another decorator or fielddolt diff -r sql
render minimal UPDATE
statementsEXISTS
predicates and expressionsfunction: 'convert_tz' not found
Published by github-actions[bot] about 3 years ago
TransformUpCtx
function that passes along a SchemaPrefix
in the TransformContext
if the schema prefix is scrutable from resolved children at that point in the analysis. apply_hash_lookup
makes use of this to make the transformed expressions in the HashLookup
lookup node refer to the right place, even when JoinNode.Left().Schema()
doesn't have the whole prefix.ctx
parameter might have been added a few months ago as part of some optimization work which never made it across the line. Instead of threading a *sql.Context
everywhere, if we have need of a *sql.Context
during analysis or to precompute or materialize a certain result in the future, I think I'm going to advocate for a specific optional interface that the analyzer is aware of. It could then pass the context through at a specific analyzer phase and the function/expression node would have the opportunity to get ready to do what it needs to.Published by github-actions[bot] about 3 years ago
main
, add dolt init --branch
flagmain
, adds a --branch
flag to dolt init
and changes the clone checkout behavior to prioritize main
, then master
, then the first sorted branch. https://www.dolthub.com/blog/2021-09-29-dolt-and-dolthub-init-main/
cli
, the target dataset in datas
, and logic for manipulating datasets is constrained to doltdb
, this is what fell out without rearranging dependencies or exposing private fieldstype CommitHook func(ctx context.Context, ds Dataset, db Database) error
isn't super flexiblectx
parameter might have been added a few months ago as part of some optimization work which never made it across the line. Instead of threading a *sql.Context
everywhere, if we have need of a *sql.Context
during analysis or to precompute or materialize a certain result in the future, I think I'm going to advocate for a specific optional interface that the analyzer is aware of. It could then pass the context through at a specific analyzer phase and the function/expression node would have the opportunity to get ready to do what it needs to.Published by github-actions[bot] about 3 years ago
dolt table import -u
when import data only represents a subset of the columnsdolt sql <1pk5col-ints-sch.sql
echo -e 'pk,c1\n1,1\n2,6'|dolt table import -u test
echo -e 'pk,c2\n1,2\n2,7'|dolt table import -u test
echo -e 'pk,c3,c4,c5\n1,3,4,5\n2,8,9,10'|dolt table import -u test
dolt sql -r csv -q 'SELECT * FROM test'
would result in:
pk,c1,c2,c3,c4,c5
1,,,3,4,5
2,,,8,9,10
Now it results in:
pk,c1,c2,c3,c4,c5
1,1,2,3,4,5
2,6,7,8,9,10
cast
library, which had a few issues with how we expect SQL numbers to function. One such issue was how number strings were handled (such as interpreting "01000"
as a binary number rather than in decimal). There have been more issues in the past that have gone undocumented, but this should fix them (or allow for an easy fix since it's all custom logic now).Published by github-actions[bot] about 3 years ago
getRefSpecs
functions from cli/envinit.defaultBranch
to Dolt Configgit init
hint: Using 'master' as the name for the initial branch. This default branch name
hint: is subject to change. To configure the initial branch name to use in all
hint: of your new repositories, which will suppress this warning, call:
hint:
hint: git config --global init.defaultBranch <name>
CREATE DATABASE
to integratorsCreateDatabase
, which is now an integrator function.Distinct
for example.AVG
, FIRST
, LAST
, MAX
, MIN
and SUM
to do this. COUNT(DISTINCT ...)
is handled by a special expression node instead, and nothing has been changed in Count
or CountDistinct
. group_concat
also seems to handle DISTINCT itself, and so I have not changed anything there. Json aggregation did not look immediately amenable to combining with DISTINCT
, because the Update
functions seemed to error when the child expression returned nil
, so I have not changed them either.value
and status
keywordsalter table t modify value float(53) not null
)alter table a drop check status
)Published by github-actions[bot] about 3 years ago
corrupted database
error:select dolt_push('origin', 'master')
and select dolt_fetch('origin')
dolt sql-client
commandA bad row was encountered while moving data.
Bad Row:
"03AGDBQ24TPRIFU0LRPRHLHU8KBOK5SQSMCT8BMVFJNANULFUG4RDHNZPOV4PJNMN0DOD1PM2V5AQQTFP1HZXZG5J5BJTP7BN0A8WLBZYMBJIRU7BWTYCNXTCQUD1UYWITEJIZR5EWWP1NM4OR40P6FTQL4NHKIA870UAFXSPRF0AIK_45YNRU98ZQOZQNJFKNF89FW9" is not valid for "VarString(utf8mb4_0900_ai_ci, 180, SQL: VarChar)"
These can be ignored using the '--continue'
After:
A bad row was encountered while moving data.
Bad Row:
"03AGDBQ24TPRIFU0LRPRHLHU8KBOK5SQSMCT8BMVFJNANULFUG4RDHNZPOV4PJNMN0DOD1PM2V5AQQTFP1HZXZG5J5BJTP7BN0A8WLBZYMBJIRU7BWTYCNXTCQUD1UYWITEJIZR5EWWP1NM4OR40P6FTQL4NHKIA870UAFXSPRF0AIK_45YNRU98ZQOZQNJFKNF89FW9" is not valid for column "street_physical" (type "VarString(utf8mb4_0900_ai_ci, 180, SQL: VarChar)")
These can be ignored using the '--continue'
WIP: BATS test still needs to be writtenFetchRemoteBranch
and FetchTag
. Multiple server connections calling dolt_pull
will overwrite each others tempfiles, but not disrupt the workingsetdolt_pull
bats tests are tightly coupled to push
and fetch
. I'll clean those up when I add dolt_push
and dolt_fetch
dolt_pull
, underlying fetch and merge steps are responsible for themselvesv
against itself2021-08-31T12:04:36-07:00 DEBUG [conn 1] Starting query {connectTime=2021-08-31T12:04:36-07:00, query=SHOW VARIABLES LIKE 't%_isolation'}
dolt sql
shell has static query timePublished by github-actions[bot] about 3 years ago
Published by github-actions[bot] about 3 years ago
dolt sql -q "CREATE table parent (pk int, val int, primary key (pk, val), key `backup` (val))"
dolt sql -q "CREATE table child (pk int, val int, foreign key (val) references parent (val))"
dolt sql -q "alter table parent drop primary key"
dolt_remotes
system table.SessionStateAdaptor
refactor removed write path support, so a lot of the code is currently dead. The Insert/Update/Delete code and bats tests still work if we implement SessionStateAdaptor.AddRemote
and SessionStateAdaptor.DeleteRemote
, so I think the code is worth keeping. I refactored the CLI functions in the process of adding Insert/Delete/Update to avoid duplication, though it is unnecessary until dolt_remote
writes are fixed.Published by github-actions[bot] about 3 years ago
@@GLOBAL.dolt_sql_server_branch_ref
overwrites the branch for new sql-server connections. Setting the variable does not change the current connection's branch. If the variable is set to an invalid ref (ex: branch does not exist), the behavior is undefined for new connections.
> set GLOBAL dolt_sql_server_branch_ref = 'refs/heads/new'
Query OK, 1 row affected (0.00 sec)
mysql> select @@GLOBAL.dolt_sql_server_branch_ref;
+-------------------------------------+
| @@GLOBAL.dolt_sql_server_branch_ref |
+-------------------------------------+
| refs/heads/new |
+-------------------------------------+
1 row in set (0.00 sec)
mysql> select active_branch();
+-----------------+
| active_branch() |
+-----------------+
| master |
+-----------------+
mysql> exit
Bye
$ mysql --user root --host=0.0.0.0 -p tmp1
mysql> select active_branch();
+-----------------+
| active_branch() |
+-----------------+
| new |
+-----------------+
1 row in set (0.00 sec)
types.String
at the noms layer to handle our TEXT
types. This works for relatively small strings, but causes issues when using larger strings (such as a 2GB string). I'm not sure of the point at which performance tanks/crashes occur using types.String
, but types.Blob
can handle a string of arbitrary length (up to our allowed maximum of 4GB for sure). This also matches how our BLOB
types are implemented.StringDefaultType
variable all over the codebase, which was a LONGTEXT
field. This has been changed to a VARCHAR(16383)
field to preserve the expected behavior (passing in a types.String
to the default string type for example). This does mean that our length limit is no longer 4GB, however I don't think we were ever reaching the VARCHAR
limit to begin with, so this switch should be fine for all of our pre-existing logic. Additionally, the original string implementation remains (along with a LegacyStringDefaultType
for system tables), so old repositories will continue to reference the old type as expected. This will only affect new tables. It was bad practice that we always used LONGTEXT
to begin with.LONGTEXT
already (it was our only supported string type for a while), so minimal testing needed to be written.types.Blob
to properly sort, as previously it sorted by hash (which is the default for types that are an extension of sequence
). Adding a proper Less
function has broken some expected behaviors in noms for blobs, but they're not being used at all in Dolt, therefore it seems safe? Fixing these behaviors would entail a lower-level change, one which I didn't deem worth.TEXT
type (of which there will probably be numerous) will be unable to be read by the previous version of Dolt (as they're missing the new blobStringType
), and will throw an unknown
error.str_to_date
go test -cover github.com/dolthub/go-mysql-server/sql/parse/dateparse
ok github.com/dolthub/go-mysql-server/sql/parse/dateparse 0.163s coverage: 89.3% of statements
cc @zachmuPublished by github-actions[bot] about 3 years ago
dolt conflicts cat
when there are no conflicts.str_to_date
go test -cover github.com/dolthub/go-mysql-server/sql/parse/dateparse
ok github.com/dolthub/go-mysql-server/sql/parse/dateparse 0.163s coverage: 89.3% of statements
cc @zachmuPublished by github-actions[bot] about 3 years ago
go-mysql-server
is not compatible with insert query like:
INSERT INTO `users` (`id`, `deleted`) VALUES (1, DEFAULT)
Whereas DEFAULT
is just to specify deleted
columns should use column default value explicitly.package main
import (
dbsql "database/sql"
sqle "github.com/dolthub/go-mysql-server"
"github.com/dolthub/go-mysql-server/auth"
"github.com/dolthub/go-mysql-server/memory"
"github.com/dolthub/go-mysql-server/server"
_ "github.com/go-sql-driver/mysql"
)
func main() {
db, _ := dbsql.Open("mysql", "root:@tcp(127.0.0.1:3307)/test")
defer db.Close()
query := `CREATE TABLE users (
id bigint(20) unsigned NOT NULL AUTO_INCREMENT,
deleted tinyint(1) unsigned NOT NULL DEFAULT '0',
PRIMARY KEY (id)
) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8mb4`
stmt, _ := db.Prepare(query)
stmt.Exec()
_, err := db.Exec("INSERT INTO `users` (`id`, `deleted`) VALUES (1, DEFAULT)")
if err != nil {
panic(err.Error())
}
stmtOut, _ := db.Prepare("SELECT `deleted` FROM `users` WHERE `id` = 1")
defer stmtOut.Close()
deleted := true
err = stmtOut.QueryRow().Scan(&deleted)
if err != nil {
panic(err.Error())
}
if deleted == true {
panic("Wrong deleted value")
}
}
var engine *sqle.Engine
func init() {
engine = sqle.NewDefault()
db := memory.NewDatabase("test")
engine.AddDatabase(db)
config := server.Config{
Protocol: "tcp",
Address: "localhost:3307",
Auth: auth.NewNativeSingle("root", "", auth.AllPermissions),
}
s, _ := server.NewDefaultServer(config, engine)
go s.Start()
}
`db.Exec("INSERT INTO `users` (`id`, `deleted`) VALUES (1, DEFAULT)")` will result in error:
Error 1105: plan is not resolved because of node '*plan.Values'
This pull request should avoid such issue by turning explicit DEFAULT
in insert query to implicit.UNIQUE KEY
sPublished by github-actions[bot] about 3 years ago
git revert --abort
command and all). Additionally, if multiple commits are specified then they're all reverted in a single commit (versus Git reversing each commit in a separate commit).% time dolt --prof cpu merge add-ipaddress
Updating uaimen5cq6jucq3abr85klgj0g594qc1..d3922cbd8fop4ast7ohl9mm4d3dn3632
extras_objectchange | 1 +
ipam_ipaddress | 1 +
2 tables changed, 2 rows added(+), 0 rows modified(*), 0 rows deleted(-)
dolt --prof cpu merge add-ipaddress 83.78s user 3.75s system 567% cpu 15.433 total
after:
% time dolt --prof cpu merge add-ipaddress
Updating uaimen5cq6jucq3abr85klgj0g594qc1..d3922cbd8fop4ast7ohl9mm4d3dn3632
extras_objectchange | 1 +
ipam_ipaddress | 1 +
2 tables changed, 2 rows added(+), 0 rows modified(*), 0 rows deleted(-)
dolt --prof cpu merge add-ipaddress 2.27s user 0.17s system 158% cpu 1.545 total
go-mysql-server
is not compatible with insert query like:
INSERT INTO `users` (`id`, `deleted`) VALUES (1, DEFAULT)
Whereas DEFAULT
is just to specify deleted
columns should use column default value explicitly.package main
import (
dbsql "database/sql"
sqle "github.com/dolthub/go-mysql-server"
"github.com/dolthub/go-mysql-server/auth"
"github.com/dolthub/go-mysql-server/memory"
"github.com/dolthub/go-mysql-server/server"
_ "github.com/go-sql-driver/mysql"
)
func main() {
db, _ := dbsql.Open("mysql", "root:@tcp(127.0.0.1:3307)/test")
defer db.Close()
query := `CREATE TABLE users (
id bigint(20) unsigned NOT NULL AUTO_INCREMENT,
deleted tinyint(1) unsigned NOT NULL DEFAULT '0',
PRIMARY KEY (id)
) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8mb4`
stmt, _ := db.Prepare(query)
stmt.Exec()
_, err := db.Exec("INSERT INTO `users` (`id`, `deleted`) VALUES (1, DEFAULT)")
if err != nil {
panic(err.Error())
}
stmtOut, _ := db.Prepare("SELECT `deleted` FROM `users` WHERE `id` = 1")
defer stmtOut.Close()
deleted := true
err = stmtOut.QueryRow().Scan(&deleted)
if err != nil {
panic(err.Error())
}
if deleted == true {
panic("Wrong deleted value")
}
}
var engine *sqle.Engine
func init() {
engine = sqle.NewDefault()
db := memory.NewDatabase("test")
engine.AddDatabase(db)
config := server.Config{
Protocol: "tcp",
Address: "localhost:3307",
Auth: auth.NewNativeSingle("root", "", auth.AllPermissions),
}
s, _ := server.NewDefaultServer(config, engine)
go s.Start()
}
`db.Exec("INSERT INTO `users` (`id`, `deleted`) VALUES (1, DEFAULT)")` will result in error:
Error 1105: plan is not resolved because of node '*plan.Values'
This pull request should avoid such issue by turning explicit DEFAULT
in insert query to implicit.dolt fetch
prints many empty lines to terminaldolt clone
errors due to timeout on poor connectionsdolt sql --multi-db-dir
does not ignore broken dolt repositoriesALTER TABLE
errordolt commit revert
dolt add
in Powershelldolt sql-server --help
should mention the way databases are specifieddolt sql-sever
dolt schema import
assigns sequential tags starting at 0. Sure to cause tag collisions if importing multiple tables.dolt sql
dolt push --set-upstream
.sql
files requires significant memorydolt login
will create new credentials even when I'm already logged indolt merge --abort
and DOLT_MERGE("--abort");
do not reset staging indexPublished by github-actions[bot] about 3 years ago
dolt merge --abort
> dolt push origin ''
panic: runtime error: index out of range [0] with length 0
goroutine 1 [running]:
github.com/dolthub/dolt/go/libraries/doltcore/ref.ParseRefSpecForRemote(0x0, 0x0, 0x7fff638f2e23, 0x0, 0x0, 0x7fff638f2e23, 0x0, 0x0)
/src/libraries/doltcore/ref/ref_spec.go:64 +0x8ec
github.com/dolthub/dolt/go/libraries/doltcore/ref.ParseRefSpec(...)
/src/libraries/doltcore/ref/ref_spec.go:55
github.com/dolthub/dolt/go/cmd/dolt/commands.parsePushArgs(0x1f4d800, 0xc00009f500, 0xc00009fef0, 0xc000232630, 0xc0000f8880, 0xc00009fef0, 0x2afb310)
/src/cmd/dolt/commands/push.go:164 +0x2ff
github.com/dolthub/dolt/go/cmd/dolt/commands.PushCmd.Exec(0x1f4d800, 0xc00009f500, 0xc0003e7e00, 0x9, 0xc00003a0a0, 0x2, 0x2, 0xc000232630, 0x9)
/src/cmd/dolt/commands/push.go:108 +0x21a
github.com/dolthub/dolt/go/cmd/dolt/cli.SubCommandHandler.Exec(0x1c67904, 0x4, 0x1c7e4c4, 0x11, 0xc00042ac80, 0x26, 0x26, 0x0, 0x1f4d800, 0xc00009f500, ...)
/src/cmd/dolt/cli/command.go:165 +0x4c2
main.runMain(0x0)
/src/cmd/dolt/dolt.go:296 +0x123e
main.main()
/src/cmd/dolt/dolt.go:131 +0x2a
Published by github-actions[bot] about 3 years ago
--force
only bypassed invalid foreign keys. With the addition of constraint violations, this check has been rendered redundant, and --force
seems like it should bypass all blockers on commit anyway.Published by github-actions[bot] over 3 years ago
Transactions Enabled by default. Dolt now generates Foreign Key constraint conflicts.
dolt verify-constraints
so that it, by default, writes to the constraint violations map on the table struct, while also only verifying the diff from the working set and HEAD
. Due to these changes, I also added parameters that disable table struct updates (--output-only
) and also allow you to verify all of the rows (--all
, which is a diff against an empty root value to simplify the logic). Lastly, table names are now optional, and omitting all table names will verify the constraints of every table.verify_constraints([tables...])
and verify_constraints_all([tables...])
, which are equivalent to the CLI command with and without the --all
parameter respectively except in output, as they return a SQL bool (1
for no violations, 0
for violations). The user has to manually query the system tables.verify-constraints
, we do not clear the previous map. This way we retain previous violations such as unique key
violations, which cannot be caught through these functions.ACCESS DENIED
error that would occur on table import when GC was running and the manifest was being renamed. That PR, however, did not touch any other rename within the codebase, nor any deletions (which can also trigger this bug). The above mentioned issue points to deletions, therefore this PR attempts to fix those plus any others by implementing the same strategy employed by the manifest renames and applying it to every deletion and rename in the project.map {
"IP_OP": #mt0robnd5t9nr7gj5gch2eec60u9a8k0,
}
New output
Map<String,Ref<Map<Tuple<Union<String,Uint>>,Tuple<Union<>>>>> - map {
"IP_OP": #mt0robnd5t9nr7gj5gch2eec60u9a8k0,
}
sqlalchemy.exc.OperationalError: (pymysql.err.OperationalError) (1105, "syntax error at position 205 near 'PRIMARY'")
[SQL:
CREATE TABLE experiments (
experiment_id INTEGER NOT NULL AUTO_INCREMENT,
name VARCHAR(256) NOT NULL,
artifact_location VARCHAR(256),
lifecycle_stage VARCHAR(32),
CONSTRAINT experiment_pk PRIMARY KEY (experiment_id),
CONSTRAINT experiments_lifecycle_stage CHECK (lifecycle_stage IN ('active', 'deleted')),
UNIQUE (name)
)
ALTER TABLE restaurants AUTO_INCREMENT = 2;
dolt_commit_diff_$tablename
breaks if filter clause is not pushed downdolt clone file://...
of an invalid file remote does not error.SELECT TIMEDIFF(NOW(), UTC_TIMESTAMP())
Fails*
dolt sql-server
in batsIN
clauses against indexed columns should be performant.dolt status
panics on an empty repodolt clone
dies with ugly error when cloning a remote just created on DoltHubdolt push
of hospital-price-transparency-v2 segfaultsPublished by github-actions[bot] over 3 years ago
SqlColToStr
to correctly parse values to strings.DatabaseProvider
abstraction, allowing integrators to add their own logic to database resolutionPublished by github-actions[bot] over 3 years ago
99% SQL Correctness by SQL Logic Test and major write performance improvement to 7X MySQL on writes as measured by sysbench
.
Published by github-actions[bot] over 3 years ago
indexed_col = ... OR different_indexed_col = ...
.Published by github-actions[bot] over 3 years ago