Merged PRs

dolt

• 7421: Add support for @@dolt_transaction_commit_message
  Allows customers to control the messages in commits created by @@dolt_transaction_commit.
  Related to https://github.com/dolthub/dolt/issues/7417
  Documentation updates: https://github.com/dolthub/docs/pull/1996
• 7412: When aborting a merge/cherry-pick, avoid resetting ignored tables.
  Tables that match dolt_ignore in the working set should be ignored by most operations.
  (CAVEAT: If we want to match git, these tables should only be ignored if there are not staged/HEAD versions of those tables. But we appear to not match git in that regard right now.)
  This PR changes the logic for aborting merges to avoid modifying any tables match dolt_ignore
• 7410: Prevent panic when sorting by commit_hash in various dolt system tables
  The analyzer rule replacePkSort does not work over certain types of indexes as some indexes are unordered.
  Certain dolt system tables use a MockIndex because they are not backed by a datastore.
  Unfortunately, we don't have a proper nil check for the idxSch, leading to a panic when attempting to sort by columns in those mock indexes
  This PR adds a nil check, so we don't try to use these indexes.
  fixes https://github.com/dolthub/dolt/issues/7404
• 7408: Adds support for dolt log --stat
  Adds support for dolt log --stat which prints the diffstat for each commit in the log.
• 7401: fix bats tests to include backticks for column names in default and generated expressions
  companion pr: https://github.com/dolthub/go-mysql-server/pull/2291
  fixes https://github.com/dolthub/dolt/issues/7388
• 7390: Concurrent table deletion and schema modification should be reported as a schema conflict.
  Previously during a merge, if one branch altered a table schema and the other branch deleted the table, this would be an error that would automatically abort the merge. Now it's a schema conflict.

go-mysql-server

• 2295: allow tables with self referential foreign keys to be dropped
  Our logic for preventing DROPs on a table if that table was referenced in a FOREIGN KEY did not account for when the table was self referencing. This PR fixes that and closes the gap in our testing for this area.
  fixes https://github.com/dolthub/dolt/issues/7418
• 2294: match range key with range type
  Creating lookups when comparing columns of different types were causing problems.
  Especially, if one of the types is a string.
  This is not a perfect fix as we don't do truncation yet, but it will stop panics.
  Reused solution from this PR:
  https://github.com/dolthub/go-mysql-server/pull/2177
  It fixes the test case in this issue, but a skipped test is added for missing truncation functionality.
  https://github.com/dolthub/dolt/issues/7371
• 2291: backquote column names in default and generated column expressions
  To prevent parsing errors for special column names, we need to wrap column identifiers with backquotes.
  MySQL prints column identifiers in default and generated expressions when doing show create table ... with backquotes anyways.
  fixes https://github.com/dolthub/dolt/issues/7388
• 2288: Add more join statistic tests, some bug fixes
• 2285: Avoid generating SemiJoins when the subquery contains an OFFSET expression, as doing say may generate incorrect results.

Closed Issues

• 7418: Unable to drop table with self referential foreign key
• 7384: Can't add multiple foreign keys in CREATE TABLE
• 7388: Cannot use column names containing spaces or uppercase characters as a generated primary key
• 7404: Panic when querying dolt_log when you order by commit_hash

Latency

Merged PRs

dolt

• 7407: Bug fix: Unresolved FK fields were not included in the FK's hash
  A bug in ForeignKey.HashOf() was silently ignoring the UnresolvedFKDetails fields, due to using the wrong API to write the string values to the bytes.Buffer. Using binary.Write for a string (instead of bytes.Buffer.Write) was resulting in an error, but the error was not captured, so it was silently ignored. This led to FK name collisions when @@FOREIGN_KEY_CHECKS was disabled and the resolved column information wasn't populated to ensure a unique hash.
  Fixing this bug required several tests to be updated with new hash values. I also turned back on the unit tests in foreign_key_test.go, which had been skipped since the storage engine migration. I tried unsetting the UnresolvedFKDetails fields when the FK is resolved, but that broke many tests – it seems that the code still relies on the UnresolvedFKDetails fields being populated even when the FK is resolved.
  Related to: https://github.com/dolthub/dolt/issues/7384
• 7405: /go/performance/utils/{sysbench_runner,tpcc_runner}: use temp dir in tpcc
• 7400: dolt backup restore --force: Add a --force option to dolt backup restore, which allows restoring into an already existing database.

go-mysql-server

• 2285: Avoid generating SemiJoins when the subquery contains an OFFSET expression, as doing say may generate incorrect results.
• 2281: groupby and having getfield index fixes
  Implementing more groupby and having strangeness from MySQL, and fixing some getfield indexing errors.
  Aliases in the projection scope were always added as new columns in the having scope; this led to getfields with indexes that were invalid in aggregation functions. Now, we don't add the alias to the scope if the alias overlaps with a column that is used in an aggregation.
  There is also a small fix to rewriting planbuilder tests to have proper tabbing, write skipped queries, and removed some random new lines.
  There are still some issues, left as skipped tests:
  • We incorrectly disallow aliases from the projection scope in aggregations
  • Columns that are aliased and used in group by should not be visible to having clauses; so, we should throw an error.
    Correctness: https://github.com/dolthub/dolt/pull/7382

Closed Issues

• 7388: Cannot use column names containing spaces or uppercase characters as a generated primary key
• 7313: sql server panic when uploading table
• 6903: allow multiple keys on same column
• 7395: id serial primary key SQL created by Drizzle in MySQL mode. Dolt can't execute.

Latency

Merged PRs

dolt

• 7373: add dolt rebase command
  Adds the dolt rebase command which is supported by the dolt_rebase() procedure.
  Related: https://github.com/dolthub/ld/pull/16323

go-mysql-server

• 2286: Earlier LOAD DATA and insert validation
  re: https://github.com/dolthub/go-mysql-server/pull/2283
  fixes: https://github.com/dolthub/dolt/issues/7313
• 2207: Fix partial left join update when columns have default values or join table is in where clause
  I ran into an issue where I was getting a panic when in update queries using left joins with at least some rows being null. The cases where I could confirm this happens is when the query planner wraps the JoinNode in Project or Filter or both. The logic that is supposed to skip null rows for update fails to skip when the node is not directly a JoinNode. I don't know if there are other node types that should be handled the same way in the toJoinNode function I added. This fix resolved the issues I was having.
  Side note:
  Another related issue I ran into while working on the tests for this was the same type of error happening when the join table is backed by IndexedTableAccess. It seems to set the root join filter to true, with a nested node that has the actual filter. The true filter causes the update skip logic to allow updates on every attempted join, even when null. This was happening for a bit but it stopped while I was messing with the test data so I couldn't keep diagnosing it.

vitess

• 305: Allow UNIQUE and PRIMARY KEY to be specified on the same column
  Related to: https://github.com/dolthub/dolt/issues/7395
• 302: Detect if a system variable scope was explicitly specified or not
  Changes the VarScope function so that it returns whether a scope was explicitly specified, or if one has been inferred.
  This is needed because some tooling (e.g. the official MySQL .NET Connector library) will query system variables (e.g. SELECT @@max_allowed_packet) and then will look up the returned value in the result set using the expected column name (@@max_allowed_packet). Currently, from the way we parse the system variable, this was always returned with the scope present, but to match MySQL's behavior, the column name needs to match the requested name.
  Related GMS PR: https://github.com/dolthub/go-mysql-server/pull/2266/

Closed Issues

• 7313: sql server panic when uploading table

Merged PRs

dolt

• 7386: Cleaner clone CLI
  Fixes https://github.com/dolthub/dolt/issues/7043 for the CLI.
  dolt_clone() procedure has not been touched by this PR.
• 7377: Push tags on remote replication
  Fixes https://github.com/dolthub/dolt/issues/7375
• 7243: Write stats to disk and dolt_statistics table
  New flatbuffer type for statistics, which contains an address reference to a prolly.Map with a hardcoded statistics schema. Statistics for a table are rewritten every ANALYZE TABLE call.
  Because the statistics schema is client-dependent, backwards compatibility will be fairly brittle right now. We could use a version identifier to invalidate a set of statistics if the client code creating a sql engine has a different version than the statistics read from disk.

go-mysql-server

• 2279: fix count distinct with decimals
  Swapped out the library used in the CountDistinct aggregation function, as it would hash decimals to the same value.
  Correctness: https://github.com/dolthub/dolt/pull/7374
• 2278: RangeHeapJoin should consistently sort NULL values before non-NULL values while managing its heap.
  Fixes https://github.com/dolthub/dolt/issues/7260
  This was ultimately caused by https://github.com/dolthub/go-mysql-server/issues/1903. I didn't think it was possible for that issue to cause user-facing problems, but I was wrong. Because of that issue, RangeHeapJoins considered all NULL values in its children iterators to come after all non-NULL values. However, if the child node was an index, then the child iterator would order its rows with the NULL values first. This causes the RangeHeapIterator to mismanage the heap and skip rows that should have been in the results.
  I updated the range heap code to manually check for NULL values when manipulating the heap. I also updated the plan tests to include NULL values in the test tables, which should now catch this issue.
• 2274: Fixup index selection when prefix not complete
  Consider a query SELECT * from t where b = 1 and c = 1 and two indexes, (a,b,c) and (b,c). We want to use the (b,c) index as a lookup, because (a,b,c) will be disjoint on a (b,c) key. This PR fixes index costing to record and prefer non-zero prefix matches. We only differentiate zero and non-zero cases here because it is easier and I think pretty reliable.
• 2247: Merge joins populate join stats
  When merge joins are a join operator for a memo group, use the two indexes in the merge to estimate the join cardinality. Small updates so that join cardinality estimates work in the coster. A few tests that make use of join statistics. The tests are affected both by stat estimates and costing methodology. It's a bit hard to separate the two, since more accurate stat estimates so often identify issues with costing. The join statistic tests are subject to shifting based on whether the smallest table is estimated to be smaller than the smallest join cardinality estimate. Better tests would be less subject to noise. Tests for avoiding anti-patterns for specific join operators would also be useful.

Closed Issues

• 7043: dolt clone can include extraneous remote refs
• 7375: new tags not automatically being pushed to remote for remote primary
• 7260: Unexpected Results when Using BETWEEN AND after CREATE INDEX
• 7348: Add "Alter User" and "Set Password" SQL Statements

Latency

This release contains backwards incompatible changes:

• The dolt_merge() and dolt_pull() procedure output schemas have changed (documentation). Each procedure has an additional column named message which is an optional field intended for human readable information.
• Merge behavior for changes which are already in the reachable history from HEAD previously would result in new merge commits. (issue). This is unexpected behavior, and deviates from Git. Now when this occurs dolt_merge() and dolt_pull() will return without any updates to the commit history. As this is not an error, we opted to use the message column to communicate to users what occurred.

Per Dolt’s versioning policy, this is a minor version bump because these changes may impact existing applications. Please reach out to us on GitHub or Discord if you have questions or need help with any of these changes.

Merged PRs

dolt

• 7363: dolt merge noop fix
  dolt_merge() operation now appropriately ends as a no-op when an attempt is made to merge in a commit which is already reachable from HEAD. This matches git behavior, and is a non-error scenario.
  In order to communicate to users what actually happened, the message column was added to the output of dolt_merge() and dolt_pull() stored procedures. Given the change in schema and that some user may actually depend on this broken merge behavior, this is a breaking change.
  Fixes: https://github.com/dolthub/dolt/issues/7344

go-mysql-server

• 2274: Fixup index selection when prefix not complete
  Consider a query SELECT * from t where b = 1 and c = 1 and two indexes, (a,b,c) and (b,c). We want to use the (b,c) index as a lookup, because (a,b,c) will be disjoint on an (b,c) key. This PR fixes index costing to record and prefer non-zero prefix matches. We only differentiate zero and non-zero cases here because it is easier and I think pretty reliable.
• 2273: Add support for the ALTER USER statement to change account passwords
  Adds basic support for the ALTER USER statement, so that users can change passwords. All users are allowed to change their own password; changing another user's password requires the CREATE USER privilege or the UPDATE privilege on the mysql database.
  MySQL reference docs for ALTER USER
  Related to: https://github.com/dolthub/dolt/issues/7348
• 2248: Use DESCRIBE ANALYZE / EXPLAIN ANALYZE to display stats data about joins.
  (In MySQL, DESCRIBE and EXPLAIN are aliases. GMS pretty consistently uses DESCRIBE internally, so I prefer that here.)
  This PR has a couple different pieces:

  sql.DescribeStats

  The most important part of this PR is the mixin struct sql.DescribeStats, defined in sql/describe.go. By embedding this struct within a Node or Expression, the node will store information about the cost estimate and row count estimate of the plan that was used to generate the node. This information can then be included in the Node/Expressions's string representation by calling the method DescribeStats.GetDescribeStatsString.
  Currently, only Join nodes embed this mixin.
  There's two ways to surface this information during a query.
  1. DESCRIBE format=estimates SELECT ... will include the row estimates and cost estimates of every node that supports this.
  2. DESCRIBE ANALYZE SELECT ... will execute the plan, and then display the actual row counts alongside the estimates.
     Note that DESCRIBE ANALYZE is only permitted for queries that don't have side effects. For instance, DESCRIBE ANALYZE INSERT ... will fail with an error.

  sql.Describable

  Previously, each Node or Expression had two different string representations, implemented as String() and DebugString(). DESCRIBE could emit either by using format=tree (the default) or format=debug. This isn't really a scalable solution, and the number of levers we want to control plan formatting may increase further over time.
  So this PR also adds a new interface for controlling how plans trees are displayed: sql.Describable. It has one method: Describe(options sql.DescribeOptions) string. DescribeOptions is a struct that contains all options that control how plans are displayed, and more options can be added as needed.
  These options are set via the value of the format= parameter in the describe query, which now accepts an underscore-separated list of options. Eg, format=debug_estimates.
  When writing the string representation of a node with children, you should call sql.Describe(node, options) instead of calling node.String() or sql.DebugString(node). This function calls Describe on the node if it exists, and otherwise falls back on DebugString or String. This allows for incremental support of the sql.Describable interface as needed instead of needing to add support for every node type right out of the gate.

  How to add support for additional node types

  While only Join nodes are currently supported, it's very easy to add support for other types. Literally all you have to do is embed the sql.DescribeStats struct in the node, and implement the Describe method, like so:

  type DemoNode struct {
  sql.DescribeStats,
  ...
  }
  func (d DemoNode) Describe(options sql.DescribeOptions) string {
  p := sql.NewTreePrinter()
  _ = p.WriteNode("Demo")
  _ = p.WriteChildren(sql.Describe(d.Child, options))
  return p.String()
  }
  

  Testing

  In order to test this functionality, every plan test generated by plangen.go now has three different expected plans: One for debug strings, one for debug + estimates, and one for debug + estimates + analyze.
  This may turn out to be too noisy when we make changes to the coster, but the plan tests are already noisy wrt to costing changes. If this turns out to be too much of a hassle we can limit this to just query_plans.go instead of all the plan tests.

Closed Issues

• 7344: dolt_pull creates unnecessary merge commits if the local branch is only ahead
• 2184: GolangCI does not work

Merged PRs

dolt

• 7336: Fixed spurious error message for event scheduling when the dolt_schem…
  …as table doesn't exist.

go-mysql-server

• 2267: Fix HASH IN <tuple> when comparing against collated string types
  This PR changes the HASH IN to always use a collation to hash the values in the case that we compare a numerical type against a collated string type.
  fixes https://github.com/dolthub/dolt/issues/7338
• 2266: Field response packets MySQL consistency improvements
  While debugging a customer issue, I noticed that Dolt/GMS has several details in the field metadata packets that differ from MySQL's behavior. This PR fixes the following differences:
  • flags are now populated for nullability, unsigned types, blobs
  • column names for projected system variables now only include the variable scope when it was explicitly specified in the query
  • @@max_allowed_packet should be an unsigned integer type
    Vitess PR: https://github.com/dolthub/vitess/pull/302
• 2264: implement octet_length() function
  This PR implements OCTET_LENGTH(), which is just a synonym for LENGTH()
  MySQL Docs: https://dev.mysql.com/doc/refman/8.0/en/string-functions.html#function_octet-length
• 2258: implementing quarter() function
  This PR adds support for the QUARTER() function, which tells the quarter from the provided date.
  MySQL Docs: https://dev.mysql.com/doc/refman/8.0/en/date-and-time-functions.html#function_quarter
• 2256: implementing ORD() function
  This PR adds support for the ORD() function in MySQL, which converts a unicode character into its numerical representation.
  In the case of STRINGS, only the leftmost unicode character is evaluated.
  Additionally, this PR fixes the bool -> string coversion to match MySQL.
  MySQL Docs: https://dev.mysql.com/doc/refman/8.0/en/string-functions.html#function_ord
  Also fixes: https://github.com/dolthub/dolt/issues/7323
• 2255: implementing CHAR() function
  This PR implements most of CHAR() functionality.
  Additionally, this fixes an overflow issue with UINT32 and UINT64
  By default, we display Bit Strings in utf8mb4, while you need to use USING in MySQL.
  Without USING the bit strings are supposed to show up as hexadecimal values.
  MySQL Docs: https://dev.mysql.com/doc/refman/8.0/en/string-functions.html#function_char
• 2212: Join estimation algos
  Code and tests related to estimating output distributions for joins. Limited to numeric types.

vitess

• 303: Add parser support for the ALTER USER statement
  Adds support for parsing ALTER USER statements (MySQL reference). First step towards allowing users to change passwords.
  Related to: https://github.com/dolthub/dolt/issues/7348
• 302: Detect if a system variable scope was explicitly specified or not
  Changes the VarScope function so that it returns whether a scope was explicitly specified, or if one has been inferred.
  This is needed because some tooling (e.g. the official MySQL .NET Connector library) will query system variables (e.g. SELECT @@max_allowed_packet) and then will look up the returned value in the result set using the expected column name (@@max_allowed_packet). Currently, from the way we parse the system variable, this was always returned with the scope present, but to match MySQL's behavior, the column name needs to match the requested name.
  Related GMS PR: https://github.com/dolthub/go-mysql-server/pull/2266/
• 301: support special syntax for position()
  This PR adds support for POSITION(<expr1> in <expr2>), which is a synoynm for LOCATE(<expr1>, <expr2>);
  MySQL Docs: https://dev.mysql.com/doc/refman/8.0/en/string-functions.html#function_position
• 300: custom char() syntax handling
  In order to support the charset argument for CHAR(... using <charset>), there has to be special syntax support.
  Companion PR: https://github.com/dolthub/go-mysql-server/pull/2255
• 299: Define an ExtendedHandler interface to handle postgres specific interactions
  These are pretty narrowly tailored for postgres prepared statements and long term should probably be placed in either doltgresql or another package. Defining them here is just expedient for experimenting with the new interfaces while they change.

Closed Issues

• 7343: Increasing Code Cov and integration zero-code tool for tests
• 7277: Implement SUBDATE
• 7338: Unexpected result when using Boolean in IN
• 7275: Deleting from dolt_constraint_violations_ causes a panic when unique column constraint violation exists.
• 7322: Subquery in an INSERT statement return NULL

Merged PRs

dolt

• 7327: fix tests from string to bool conversion change
  A GMS PR changed the conversion of boolean values to string from true = 'true' and false = 'false' to true = '1' and false = '0'.
• 7312: /.github/{scripts,workflows}: working through some pr regressions labeling
• 7311: go/store/nbs: Flush a journal writer to fsync and write index records when it builds up substantial unflushed data, even if there is no commit yet.
• 7285: Bug fix: keyless table constraint violations
  Keyless tables didn't expose a unique row identifier in the dolt_constraint_violations_<table> system tables. This manifested as a panic when trying to delete the violation artifacts from the system table, but it also meant that we couldn't uniquely identify rows in the system table by the system table's PK for keyless tables. To enable keyless tables to use constraint violation artifacts, we now return the row's identifying hash from the source keyless table in the system table. Note that the dolt_conflicts_<table> system table did not have a similar issue because its primary key is a unique ID generated in the dolt_conflict_id field.
  Related to: https://github.com/dolthub/dolt/issues/7275

go-mysql-server

• 2263: Insert VALUES statements with subqueries index correctly
  fixes: https://github.com/dolthub/dolt/issues/7322
  A couple things I noticed in the process of debugging this:
  • We appear to inline inject an empty row when evaluating VALUES statements. Code comments suggest TRIGGERS are the reason for this. Most of the time this doesn't impact other value statements, and I think we've adapted VALUE expressions to use this pattern, but it shifts subquery expression indexes in a way that we were not accounting for. The change here updates subqueries in INSERT VALUES to expect this offset.
  • We can't fix this during expression indexing at the INSERT level, because a second analysis of VALUES statements is necessary, for unclear reasons.
  • We differentiate between INSERT VALUES and INSERT SELECTs, because INSERT SELECTs do not inject an offset row. In theory we could probably normalize these two cases and inject an empty row into insert select sources, but I ran into problems trying to implement the fix this way.
• 2262: implement elt() and field() function
  This PR implements the ELT() function, which just returns the nth argument as a string.
  This PR also implements the FIELD() function, which returns the first case-insensitive match to the first argument.
  MySQL Docs:
  • https://dev.mysql.com/doc/refman/8.0/en/string-functions.html#function_elt
  • https://dev.mysql.com/doc/refman/8.0/en/string-functions.html#function_field
• 2261: implement exp() function
  This PR adds support for the EXP() function in MySQL.
  MySQL Docs: https://dev.mysql.com/doc/refman/8.0/en/mathematical-functions.html#function_exp
• 2260: implementing bit_count() function
  This PR implements the BIT_COUNT() function, which counts the number of 1s in the binary representation of the integer argument.
  MySQL Docs: https://dev.mysql.com/doc/refman/8.0/en/bit-functions.html#function_bit-count
• 2259: implement atan2() function and support second argument for atan
  This PR adds support for ATAN() and ATAN2(), which just computes atan
  MySQL Docs: https://dev.mysql.com/doc/refman/8.0/en/mathematical-functions.html#function_atan2
• 2257: preserve original col name for subqueries
  There are likely other places where we don't copy over originalCol name when creating scope columns, but I can't tell which should be left as empty and which should be filled out.
  fixes https://github.com/dolthub/dolt/issues/7316
• 2256: implementing ORD() function
  This PR adds support for the ORD() function in MySQL, which converts a unicode character into its numerical representation.
  In the case of STRINGS, only the leftmost unicode character is evaluated.
  Additionally, this PR fixes the bool -> string coversion to match MySQL.
  MySQL Docs: https://dev.mysql.com/doc/refman/8.0/en/string-functions.html#function_ord
  Also fixes: https://github.com/dolthub/dolt/issues/7323
• 2254: implementing pi() function
  Implements the PI() function, which just returns the value of PI
  MySQL docs: https://dev.mysql.com/doc/refman/8.0/en/mathematical-functions.html#function_pi
• 2253: implement SPACE() function
  This function outputs a string containing the number of space characters specified by its argument.
  MySQL Docs: https://dev.mysql.com/doc/refman/8.0/en/string-functions.html#function_space
• 2251: fix order by optimization
  This hopefully fixes 24 sqllogictests.
  We introduced an optimization where we would drop Sort nodes for queries that included ORDER BYs if there were IndexLookups that were a matching prefix over the sortfields. The idea was that since indexes are already in order there was no need to sort.
  However, there is a case when that isn't necessarily true. If the index is created from a filter over multiple columns, specifically when that filter contains an OR expression, it is possible to iterate over columns in a non-sorted order. During analysis, the filters are converted into several non-overlapping ranges; it is possible for a range expression in the range to overlap with another range's range expression, but for the two ranges to not overlap. It's clearer to look at the test.
  We didn't catch this bug earlier, as it only affects queries in dolt. We iterate over rows in the in-memory tables in such a way that the rows will still appear in order.
• 2250: Add support for SUBDATE()
  Implements the SUBDATE() built-in function and cleans up a few other spots.
  Related to: https://github.com/dolthub/dolt/issues/7277

Closed Issues

• 7323: Unexpected results when comparing with empty string
• 7322: Subquery in an INSERT statement return NULL
• 7316: Subqueries lowercase the column names

Latency

Merged PRs

dolt

• 7297: dolt push: Only assert on a clean working set when doing a push if we are running against a server which requests it.
• 7284: dolt table import --all-text correctly infers pk if none given
  dolt table import will use the first column in the import file as the primary key if none is explicitly defined. This change updates --all-text to also correctly do this inference.
• 7283: add comments to help devs when a failure occurs
• 7278: default to text option for dolt table import
  Adds the --all-text option to dolt table import which will default all columns to TEXT type.
  Resolves: https://github.com/dolthub/dolt/issues/6471
• 7264: Bh/minver improvements

go-mysql-server

• 2251: fix order by optimization
  This hopefully fixes 24 sqllogictests.
  We introduced an optimization where we would drop Sort nodes for queries that included ORDER BYs if there were IndexLookups that were a matching prefix over the sortfields. The idea was that since indexes are already in order there was no need to sort.
  However, there is a case when that isn't necessarily true. If the index is created from a filter over multiple columns, specifically when that filter contains an OR expression, it is possible to iterate over columns in a non-sorted order. During analysis, the filters are converted into several non-overlapping ranges; it is possible for a range expression in the range to overlap with another range's range expression, but for the two ranges to not overlap. It's clearer to look at the test.
  We didn't catch this bug earlier, as it only affects queries in dolt. We iterate over rows in the in-memory tables in such a way that the rows will still appear in order.
• 2249: fix decimal out of bounds in case statement
  We are failing 42 sqllogictest with the same error: Out of bounds value for decimal type.
  This is caused by Case performing a convert to a decimal type that had a bad exclusiveUpperBound.
  MySQL has specific rules about the precision , scale, and conversion of decimals, and it is difficult to match their behavior exactly.
  Fortunately, the result is correct before the conversion, so the fix here is to just use the result, and not convert.
  This is somewhat of a bandaid fix; decimal type is an area that needs more improvement in general.
  Fixes https://github.com/dolthub/dolt/issues/7079
• 2246: round IndexAccess for numerical strings over int columns
  fixes https://github.com/dolthub/dolt/issues/7261
• 2244: Feature: SYSDATE() function
  Adds support for the SYSDATE() function.
  Related to: https://github.com/dolthub/dolt/issues/7270
• 2239: Support for doltgres prepared statements
  See https://github.com/dolthub/vitess/pull/299 for new handler interface
  This includes three main changes:
  1. Implements new ExtendedHandler methods for doltgres prepared statement support
  2. Refactors the handler and engine to pull out functionality common to MySQL and Postgres prepared statement execution paths
  3. Adds type information to BindVars where they can be implicitly recognized, as required for the Describe message in Doltgres. This tells the client the postgres types required for parameters in query strings before binding. Doltgres inspects the plan node to find bindvars and their types to return to the client.
     I'm most interested in feedback on 3) before I go deeper on it. It's implemented only for insert values and comparison expressions (used in filters). A complete solution also requires support for UPDATE and probably some other expression types. If this approach is too hacky or is likely to break or interfere with other things, I want to know before continuing. If the latter, it would be pretty easy to introduce a bind context specific to this use case and not assign types unless it's set.
     See https://github.com/dolthub/doltgresql/pull/87 for how doltgres uses this new bindvar type information.

vitess

• 299: Define an ExtendedHandler interface to handle postgres specific interactions
  These are pretty narrowly tailored for postgres prepared statements and long term should probably be placed in either doltgresql or another package. Defining them here is just expedient for experimenting with the new interfaces while they change.
• 298: parse empty begin end block in trigger
  14 SQL correctness tests are failing because we throw syntax errors on queries like this:

  CREATE TRIGGER t1r1 AFTER UPDATE ON t1 FOR EACH ROW BEGIN END;
  

Closed Issues

• 7276: Implement DATEDIFF
• 7079: Error in evaluating expression: "Out of range value for column of Decimal type"
• 6471: feature request: dolt table import add option to default to text/varchar type
• 7261: Unexpected Result when Querying with CONCAT
• 7270: Please implement SYSDATE for Mediawiki install

This release contains backwards incompatible changes:

• Values in JSON fields that were edited on both sides of a merge are now automatically merged, when possible, using a new JSON merging algorithm. The current JSON merging algorithm supports merging object keys in edited JSON docs, but other changes, such as edits to arrays, will still result in data conflicts in the merge that must be manually resolved. If you want to opt-out of this JSON merging behavior, you can set the @@dolt_dont_merge_json session variable.

Per Dolt’s versioning policy, this is a minor version bump because these changes may impact existing applications. Please reach out to us on GitHub or Discord if you have questions or need help with any of these changes.

Merged PRs

dolt

• 7267: recursively walk the fields of structs
• 7242: sql shell prompt includes branch name
  Three changes to the dolt sql shell:
  • Show the branch you are on: mydb/main>
  • Show the workspace is dirty with a "*" in the prompt
  • Add color to the DB name, branch, and dirty status.
• 7230: Allow for three way merges to automatically resolve concurrent changes to a JSON object when the changes affect different keys.
  I recommend looking at one commit at a time.
  This PR implemented the "Primary Goal" outlined in this doc.
  Basically, when two branches in a merge both modify the same JSON value, but touch different keys, dolt should be able to automatically resolve the merge by doing a Three Way Merge that compares the value on both branches to the value at a common ancestor.
  This implementation deviates from the strategy outlined in the doc in one major way: when a branch contains an edit to a nested object, we don't run the diff or merge algorithms recursively. Instead, we generate a diff that contains a JSON path to the modified object. Doing this allows us to generate all the diffs in a single linear pass, and is still easy to reason about what changes are conflicts.
  We discussed only performing this merge on JSON objects below a certain size for performance reasons, but the size of the object isn't actually visible to the merge algorithm. Given that this algorithm only runs for rows that would otherwise have a merge conflict, all dolt merges that succeed before this PR should see zero performance impact from this change, so I'm not worried about performance.

go-mysql-server

• 2244: Feature: SYSDATE() function
  Adds support for the SYSDATE() function.
  Related to: https://github.com/dolthub/dolt/issues/7270
• 2243: Fix context warn panics
  fixes: https://github.com/dolthub/dolt/issues/7262
• 2238: [memo] RequiredIndex interface
  Some table nodes need to be executed as IndexScans with mandatory filters. The new interface makes this transparent to join planning.
  re: https://github.com/dolthub/dolt/pull/7256
• 2216: Use indexes and lookups for cardinality estimates
  Cardinality estimation happens once after join reordering, indexing, and lookup join planning. The indexes and lookups help with result count estimates.

Closed Issues

• 7262: Panic when Using INNER JOIN with String in IN

Latency

Merged PRs

dolt

• 7259: support importing files with UTF-8, UTF-16LE, UTF16-BE BOM headers on sql path

Closed Issues

• 7247: Problem with selecting fully qualified column names

Latency

Merged PRs

dolt

• 7257: Update MySQL Connector/J integration tests to cover table-qualified field lookups
  When using the MySQL Connector/J library to pull out a field value from a row, you can specify the column name, or you can specify the table-qualified column name (e.g. myTable.col1). https://github.com/dolthub/go-mysql-server/pull/2235 fixed the missing metadata, and this PR adds an integration test for the MySQL Connector/J library to ensure we don't regress.
  Related to https://github.com/dolthub/dolt/issues/7247
• 7252: dolt_pull performance improvement
  dolt_pull currently performs unnecessary FastForwards for remote branches. There is a more optimal mechanism to fetch all remotes, actions.FetchRefSpecs, and by using that we greatly increase the speed of dolt_pull when there are lot of remote branches.
• 7249: fix minver
• 7248: omit empty on new yaml items

go-mysql-server

• 2238: [memo] RequiredIndex interface
  Some table nodes need to be executed as IndexScans with mandatory filters. The new interface makes this transparent to join planning.
  re: https://github.com/dolthub/dolt/pull/7256
• 2236: trim floats when hashing
  fmt.Sprintf("%v", x) writes floats with a x.0 which causes it to never equal hashed strings.
  Initially, I wanted to always convert the LHS to the type in the RHS, but this is difficult when there are multiple types in the RHS
  fixes https://github.com/dolthub/dolt/issues/7246
• 2235: Add source table and database to field metadata in wire responses
  The field metadata messages we were sending back for a result set did not include the source table and source database for fields. This caused a behavior difference from MySQL where the table-qualified column name (e.g. table1.ID) would not work with Dolt when using the MySQL Java Connector library. See https://github.com/dolthub/dolt/issues/7247 for more details.
  Updates to unit tests for the schemaToFields code are in this PR, and I'll follow up with a PR in the Dolt repo that updates our MySQL Connector library integration tests to add coverage for the table-qualified column name.
• 2234: use cached decimal type for casts
  Fixes a variety (hopefully 49) of SQLLogicTests involving CASTS(... AS DECIMAL)
• 2233: Add logging for COM_RESET_CONNECTION command
  Adding debug logging to see when clients are sending this command, even though we don't implement it yet. For example, ORMs may send this command when returning a connection to a connection pool, so it may be helpful to implement this command and clear out session state in that case.
• 2230: guard ctx and session with nil
  fixes https://github.com/dolthub/dolt/issues/7235

vitess

• 298: parse empty begin end block in trigger
  14 SQL correctness tests are failing because we throw syntax errors on queries like this:

  CREATE TRIGGER t1r1 AFTER UPDATE ON t1 FOR EACH ROW BEGIN END;
  

• 297: Fixing the version keyword to not require identifier quotes
  The version keyword still required identifier quoting in some usages, such as SELECT * FROM base.version;. See https://github.com/dolthub/dolt/issues/7237 for more details.
  This change moves the version keyword into the main list of non-reserved keywords. There was one conflict from use of the version keyword in the function_call_keyword rule, but it turns out that use of version there is not required. We have an existing test for using the version() function, so I didn't add a new one.

Closed Issues

• 7246: Unexpected Result when Using -'' in IN
• 3467: Support dolt rebase
• 7237: Unable to query tables named version in another database without escaping

Merged PRs

dolt

• 7241: Mark rebase_order as the primary key for the dolt_rebase system table
• 7233: Delete branch message as a push result
  Currently every push results in the message "[new branch] ..." which is misleading. Unfortunately fixing this for all updates is way harder than I want to dig into at the moment, so I'm going to make the only case we can detect easily (delete) print a better message.
• 7183: Feature: Interactive Rebase
  Adds support for an interactive rebase workflow with Dolt. This allows users to edit their commit history, including rewording commit messages, reordering commits, dropping commits, and squashing multiple commits together. At the end of an interactive rebase, the current branch points to a new commit history created by executing the rebase plan the user specified.

  -- we start on a dev branch named myDevBranch
  select commit_hash, message from dolt_log;
  +----------------------------------+----------------------------+
  | commit_hash                      | message                    |
  +----------------------------------+----------------------------+
  | fjq1sd7t8nbqn0ddvcor1i455i0i5ken | inserting row 3            |
  | v8g3ic9j4euf68t7tfnj7je8e1j6vt9m | inserting row 2            |
  | q909vda7hc7ithb2fcgcfghuvt8epsrj | inserting row 1            |
  | 0djr0smhr9ucdjivsnabu7ls419p0f37 | creating table t           |
  | d0lte08fpl5hpcmqj90d6jqrb6i17lfi | Іnitiаlіzе dаtа repоsіtory |
  +----------------------------------+----------------------------+
  -- the interactive rebase is started by calling the dolt_rebase stored procedure
  call dolt_rebase('-i', 'main');
  -- once an interactive rebase is started, the session is placed on a temporary working
  -- branch and a dolt_rebase table is populated with the default rebase plan
  select * from dolt_rebase;
  +--------------+--------+----------------------------------+-----------------+
  | rebase_order | action | commit_hash                      | commit_message  |
  +--------------+--------+----------------------------------+-----------------+
  | 1.00         | pick   | q909vda7hc7ithb2fcgcfghuvt8epsrj | inserting row 1 |
  | 2.00         | pick   | v8g3ic9j4euf68t7tfnj7je8e1j6vt9m | inserting row 2 |
  | 3.00         | pick   | fjq1sd7t8nbqn0ddvcor1i455i0i5ken | inserting row 3 |
  +--------------+--------+----------------------------------+-----------------+
  -- users can adjust the plan to reorder commits, drop commits, change commit messages or squash commits together
  update dolt_rebase set action='reword', commit_message='inserting rows 1, 2, 3' where rebase_order=1;
  update dolt_rebase set action='fixup' where rebase_order > 1;
  -- execute the adjusted plan
  call dolt_rebase('--continue');
  -- see that the Dolt commit history has been rewritten
  select commit_hash, message from dolt_log;
  +----------------------------------+----------------------------+
  | commit_hash                      | message                    |
  +----------------------------------+----------------------------+
  | 2mggja4n903gdo5fmbvcbj9vsvgdjd8q | inserting rows 1, 2, 3     |
  | 0djr0smhr9ucdjivsnabu7ls419p0f37 | creating table t           |
  | d0lte08fpl5hpcmqj90d6jqrb6i17lfi | Іnitiаlіzе dаtа repоsіtory |
  +----------------------------------+----------------------------+
  

  Resolves https://github.com/dolthub/dolt/issues/3467

go-mysql-server

• 2231: fix precision for utc_timestamp
  The UTC_TIMESTAMP() function should take in an argument and round the milliseconds. For now, we stick to always returning the full precision (6 places)
• 2230: guard ctx and session with nil
  fixes https://github.com/dolthub/dolt/issues/7235
• 2228: fix type conversion in Between expressions
  Replace the logic in Between.Eval() with a logically equivalent AND statement to reuse the type conversion logic in comparison.go
  fixes https://github.com/dolthub/dolt/issues/7229
• 2227: Add JsonIter class for iterating over the key-value pairs of a JSON object.
  This is the GMS side of automating JSON merging in Dolt: just some type aliases and a simple iterator for getting the keys in a JSON object in a deterministic order.
  It's worth pointing out that currently Dolt stores JSON in a normalized form by sorting keys by length, but the iterator here uses a simple lexicographic order instead. This difference doesn't really matter at the moment because we unmarshall the entire object into a go map no matter what. But Dolt needs to be aware of the ordering used in order to correctly compute three-way diffs.
• 2226: Error on NOW() eval with nil context
• 2218: implement NOW() siblings
  This PR has our behavior surrounding NOW() functions more closely match MySQL.
  Changes:
  • Added NOW() synonyms to registry
  • Have CURRENT_TIMESTAMP(), LOCALTIME(), LOCALTIMESTAMP() all just call NOW()
  • Support parsing synonyms in DEFAULT and ON UPDATE expressions
  • Fixed SHOW CREATE TABLE to print CURRENT_TIMESTAMP for NOW() and synonyms
    Companion PR: https://github.com/dolthub/vitess/pull/296
    Fixes:
  • https://github.com/dolthub/dolt/issues/7129
  • https://github.com/dolthub/dolt/issues/6058
• 2214: Fix wrongly written 'aribtrary' -> 'arbitrary'
  Fix wrongly written word 'aribtrary' -> 'arbitrary'

vitess

• 297: Fixing the version keyword to not require identifier quotes
  The version keyword still required identifier quoting in some usages, such as SELECT * FROM base.version;. See https://github.com/dolthub/dolt/issues/7237 for more details.
  This change moves the version keyword into the main list of non-reserved keywords. There was one conflict from use of the version keyword in the function_call_keyword rule, but it turns out that use of version there is not required. We have an existing test for using the version() function, so I didn't add a new one.
• 296: refactoring default and on update expressions
  This PR changes the grammar to more closely match MySQL's behavior, specifically around the NOW() function and its synonyms.
  Changes:
  • Throw syntax errors for ON UPDATE expressions against functinos that aren't NOW() or a synonym.
  • Only accept integer for argument to NOW() and synonyms; syntax error for anything else
  • Simplified grammar rules
  • Removed CurTimeFuncExpr from AST in favor of plain FuncExpr
    Companion PR: https://github.com/dolthub/go-mysql-server/pull/2218
• 295: Allow inline column check constraint definitions to appear in any order
  Previously, an inline column check constraint could only appear as the very last option for a column definition. This change allows it to appear in other positions in the column definition. For example, this query now works:

  create table t123 (c1 varchar(5) check (c1 in ('v1', 'v2')) NOT NULL);
  

  Resolves: https://github.com/dolthub/dolt/issues/7195

Closed Issues

• 6058: current_timestamp() and now() do not return the same values
• 7129: support current_timestamp synonyms
• 7235: Panic when Using INNER JOIN
• 7229: Unexpected Results when Using BETWEEN and LEFT JOIN
• 3467: Support dolt rebase

Merged PRs

dolt

go-mysql-server

• 2224: empty string is 0.0 decimal
  fix https://github.com/dolthub/dolt/issues/7222

Closed Issues

Merged PRs

dolt

• 7217: Nonsense config warning prints to stderr
  Updates the config warning to print to stderr instead of stdout so as to not clutter up output

go-mysql-server

• 2224: empty string is 0.0 decimal
  fix https://github.com/dolthub/dolt/issues/7222
• 2223: Fix multi db views with overlapping name
  fixes https://github.com/dolthub/dolt/issues/7223
• 2221: fix decimal max precision check
• 2220: guard ctx functions with nil check
  fixes https://github.com/dolthub/dolt/issues/7216
• 2219: Implement fmt.Stringer for JsonDocument.
  GMS side of an upcoming Dolt PR.
  I'm improving error handling in schema merge tests by having the tests print the de-serialized values in incorrect tables instead of just their bytes. Since table values are passed around as an interface{}, it seems reasonable to have them implement Stringer.
• 2204: fix foreign key panic on update
  When resolving ForeignKeys, ForeignKeyReferenceHandler.CheckTable() calls Partition() and iterates over all rows in the referenced table to ensure that the key is valid (doesn't contain any values that are not in the referenced table).
  WriteableIndexedDoltTable prevents us from reading every value through Partition() as it is only supposed to return rows that match the index range. Additionally, there are some differences between the way MySQL and dolt handle certain ddl and dml operations during delayed resolution of Foreign Keys.
  To fix this, we selectively perform table scans (only when we are creating/altering the table to resolve foreign keys).
  Fixes https://github.com/dolthub/dolt/issues/6963
  Companion PR: https://github.com/dolthub/dolt/pull/7203

Closed Issues

• 7222: Unexpected Results when Using ROUND()
• 7223: Views, referencing another database crash with: 1105 - exceeded max analysis iterations (8)
• 7221: dolt can be deleted oddly

Merged PRs

dolt

• 7214: Additional dolt schema import types
  This change adds support for JSON and TEXT columns in dolt schema import.
  Fixes: https://github.com/dolthub/dolt/issues/2401

Closed Issues

• 7216: Panic when Using BETWEEN AND
• 2401: dolt schema import does not support long columns

Merged PRs

dolt

• 7158: /{go,integration-tests}: add oci blobstore implementation
  This PR adds an Oracle Cloud Infrastructure (OCI) blobstore implementation.
  OCIBlobstore.Concatenate is unimplemented as OCI doesn't currently support building composite objects.
  OCIBlobstore.Put will use the objectstorage.PutObject method for uploads smaller than the minimum multipart upload part size of 10MB, but will use the Multipart Upload process for all uploads larger than 10MB.
  Objects have a VersionId as well as an ETag, however the ETag value is used for the conditional match header, not the VersionId, see https://github.com/oracle/oci-go-sdk/blob/v65.54.0/objectstorage/get_object_request_response.go?s=719:5196#L39
  UPDATE: this PR also contains changes so that an Oracle bucket can be used as a remote and backup

go-mysql-server

• 2217: Fix decimal parsing bug
  Decimals with capitalized exponential 'E' were incorrectly bound to literals, losing precision in some cases.

  select 5.0E-5; // -> 0.0001
  

• 2210: sql/planbuilder: Update join defaults are not prefixed, need individual table schemas to resolve
  Updates need to resolve default expressions. The input definitions for update joins include the total join output columns. If two of those columns have the same name, the target table's unqualified default expression will throw an "ambiguous column" error. We partition the update join schemas/column definitions to sidestep the error.
  The case where this is problematic is ALTER COLUMN expressions, where only the columns being modified are scoped for some reason. In those cases, I create a new scope with the source table schema to provide the full set of underlying column definitions. There is probably a whole class of virtual columns/column default bugs related to the way we resolve alter statements.
• 2208: fix type evaluating for mult and div arithmetic operations
  • For Mult expression, the .Type() method is fixed that it returns correct type for decimal type results, which uses the sum of precision and scale values of its right and left values.
  • For Div expression, the .Type() method is fixed that it returns the correct precision and scale for the final result.
• 2204: fix foreign key panic on update
  When resolving ForeignKeys, ForeignKeyReferenceHandler.CheckTable() calls Partition() and iterates over all rows in the referenced table to ensure that the key is valid (doesn't contain any values that are not in the referenced table).
  WriteableIndexedDoltTable prevents us from reading every value through Partition() as it is only supposed to return rows that match the index range. Additionally, there are some differences between the way MySQL and dolt handle certain ddl and dml operations during delayed resolution of Foreign Keys.
  To fix this, we selectively perform table scans (only when we are creating/altering the table to resolve foreign keys).
  Fixes https://github.com/dolthub/dolt/issues/6963
  Companion PR: https://github.com/dolthub/dolt/pull/7203

Closed Issues

• 6963: Update FK panic

Latency

This release contains backwards incompatible changes:

• The -u short option has been removed from the dolt_fetch and dolt_pull stored procedures and from the CLI options for fetch and pull. You must specify the long argument flag as --user now. This change was made to sync up with the push command, which has a -u option used for updating the remote tracking branch.
• The --old-format argument is no longer supported by the dolt init command. New databases will only support the current storage format.

Per Dolt’s versioning policy, this is a minor version bump because these changes may impact existing applications. Please reach out to us on GitHub or Discord if you have questions or need help with any of these changes.

Merged PRs

dolt

• 7206: upgrade crypto
  Dependabot sent two proposed changes to update crypto dependency, but they don't build unless they are combined.
  https://github.com/dolthub/dolt/pull/7190
  https://github.com/dolthub/dolt/pull/7188
• 7152: Support pushing to a sql-server instance
  This change alters the remoteapi server behavior running in a sql-server to allow people to push to it, with some caveats:
  • User authenticates using the DOLT_REMOTE_PASSWORD environment vairable, and --user passed as a stored procedure or CLI flag
  • User has SuperUser privledges, due to the nature of the root update
  • The update of the server will be performed only if there are no working set changes and the commit being pushed is a fast forward.
  • The --force option will overwrite any changes on the target, including the working set changes
• 7135: Removing --old-format option from dolt init
  Removes the --old-format flag from dolt init. I decided to keep the flag for --new-format for a little longer (even though it's the default, so that's a no-op) just in case people are still using it in scripts.
  This one should technically be released as a minor version bump, so we can hold off on merging it in if we want to coordinate it with another minor version bump change so they go out together.

Closed Issues

• 7155: Unexpected Results when Using IN after Creating Index

Merged PRs

dolt

• 7206: upgrade crypto
  Dependabot sent two proposed changes to update crypto dependency, but they don't build unless they are combined.
  https://github.com/dolthub/dolt/pull/7190
  https://github.com/dolthub/dolt/pull/7188
• 7152: Support pushing to a sql-server instance
  This change alters the remoteapi server behavior running in a sql-server to allow people to push to it, with some caveats:
  • User authenticates using the DOLT_REMOTE_PASSWORD environment vairable, and --user passed as a stored procedure or CLI flag
  • User has SuperUser privledges, due to the nature of the root update
  • The update of the server will be performed only if there are no working set changes and the commit being pushed is a fast forward.
  • The --force option will overwrite any changes on the target, including the working set changes
• 7135: Removing --old-format option from dolt init
  Removes the --old-format flag from dolt init. I decided to keep the flag for --new-format for a little longer (even though it's the default, so that's a no-op) just in case people are still using it in scripts.
  This one should technically be released as a minor version bump, so we can hold off on merging it in if we want to coordinate it with another minor version bump change so they go out together.

Closed Issues

• 7155: Unexpected Results when Using IN after Creating Index

Merged PRs

dolt

• 7191: Correctly resolve merge conflicts for types where two values with different representations can compare equal.
  Prior to this PR, we used the column's comparison operator when attempting to detect if concurrent changes to a column could be resolved during a merge operation.
  However, this can cause a problem when a type has two different representations that compare equal.
  • Some types have different representations that compare equal but can exhibit different behavior. For example, strings in a case-insensitive text column compare equal but behave differently when used in a context with a different collation and when displayed to the user. In this case, the merge algorithm should not consider them equal.
  • Other types have different representations that compare equal and exhibit identical behavior. For example, we store Decimal types as a (integer, exponent) pair, but we don't normalize this pair. So pairs like (45, 1), (450, 0), and (4500, -1) are all indistinguishable, including comparing equal. Since there's no way to distinguish these and the user should not have to know or care which representation is used, there is no danger in resolving potential merge conflicts by simply choosing a value. However, care must be taken that the same value is used regardless of the direction of the merge, in order to guarentee that the final table hash is the same.
    In order to accomplish both of these, this PR does two things:
  • Forces the three-way merge algorithm to use a default comparator instead of the table's comparator. This defaults to a binary collation for strings.
  • In the event that a merge conflict is auto-resolvable because the two values being assigned are equal to each other, sort them and use the higher one instead of choosing one branch arbitrarily.
• 7181: Warning for nonsense variables in config
  Prints a warning if any nonsense variables are found in global or local config files whenever any CLI command is run. Also restricts users from adding any nonsense variables with dolt config --add or dolt config --set.
  Resolves: https://github.com/dolthub/dolt/issues/7165

go-mysql-server

• 2210: sql/planbuilder: Update join defaults are not prefixed, need individual table schemas to resolve
  Updates need to resolve default expressions. The input definitions for update joins include the total join output columns. If two of those columns have the same name, the target table's unqualified default expression will throw an "ambiguous column" error. We partition the update join schemas/column definitions to sidestep the error.
  The case where this is problematic is ALTER COLUMN expressions, where only the columns being modified are scoped for some reason. In those cases, I create a new scope with the source table schema to provide the full set of underlying column definitions. There is probably a whole class of virtual columns/column default bugs related to the way we resolve alter statements.
• 2206: guard ctx.Span again
  https://github.com/dolthub/go-mysql-server/pull/2203 again, because it was overwritten
  fixes https://github.com/dolthub/dolt/issues/7182 again
• 2205: use entire lateral scope when indexing RangeHeapJoin
  RangeHeapJoins looked at the lateral join scope when assigning indexes.
  However, we never tested nested joins for this case, leading to https://github.com/dolthub/dolt/issues/7177
  What made the error more apparent was the string in the left scope that would result in empty results when doing lookups into the right tables. The fix was to look at the whole lateral scope when indexing RangeHeapJoins
  fixes https://github.com/dolthub/dolt/issues/7177
• 2188: support on update current_timestamp
  This PR adds support for handling and executing ON UPDATE column expressions, which are triggered when an update is applied to any column in the row.
  TODO:
  • we support now() for default, but not the syntax for on update
    fixes https://github.com/dolthub/dolt/issues/6030

vitess

• 295: Allow inline column check constraint definitions to appear in any order
  Previously, an inline column check constraint could only appear as the very last option for a column definition. This change allows it to appear in other positions in the column definition. For example, this query now works:

  create table t123 (c1 varchar(5) check (c1 in ('v1', 'v2')) NOT NULL);
  

  Resolves: https://github.com/dolthub/dolt/issues/7195
• 294: Allow SqlType to parse "CHARACTER" and add tests for every other possible type that could be passed in.
  SqlType is a function in Vitess for normalizing every type name. It was missing an entry for the "CHARACTER" keyword.
  I added tests that should verify every single valid type keyword in the grammar, so this shouldn't happen again.

Closed Issues

• 7195: Error parsing CREATE TABLE with both CHECK and NOT NULL
• 7080: Joins against columns with case insensitive collations sometimes compare the join condition as if they were case sensitive.
• 7177: Query results are not as expected
• 7165: Fail when there is nonsense in config_golbal.json
• 7186: Support SET PERSIST GLOBAL for Dolt system variables

Merged PRs

dolt

• 7179: go: sqle: DoltDatabaseProvider: If we encounter an error while creating a database, try to clean up after ourselves so we do not leave partially initialized database state around.
• 7178: go: sqle: cluster: Create missing remotes for cluster replication on startup if they do not already exist.
  Previously dolt sql-server would error if there were existing databases and they were missing any of the remotes listed as cluster replication remotes in the config.yaml. This changes it so that those remotes are automatically created at startup instead.
• 7175: /go/store/{blobstore,datas,nbs}: working, needs cleanup
• 7173: go: sqle: cluster: Fix MySQLDb and dolt_branch_control replication for the cases where you have more than one standby server in the cluster.
  Previously these changes would only get replicated to the first database in the cluster.
• 7171: Db/oc persister
• 7169: More detailed warning message for import schema not matching
  Updates the warning message for dolt table import to print when the import schema does not match the destination table's schema, not just when there are fewer columns in the import file. Also updates the warning message to list the column names that differ between the import file and the destination table's schema.
  Resolves: https://github.com/dolthub/dolt/issues/6113
• 7161: Return a different error message for tx commit conflicts depending on @autocommit
  When a transaction commit conflict occurs, the next steps to take are slightly different depending on whether @autocommit is enabled or not. This PR changes the error message we return to describe those steps in more detail when @autocommit is enabled.
• 7153: proto/third_party/flatbuffers,go/gen/fb/serial: Bump flatc to pick up the version that only has Try accessors.
• 7137: Adding tests for column name metadata through MySQL C++ and Rust Connector
  Added assertions that result set column name metadata is populated for the C++ and Rust MySQL Connector libraries.
• 7115: serialize on update expressions
  This PR add serialization for ON UPDATE expressions on columns.
  This change requires changes to the flatbuffer schema as we have to store an additonal field.
  Older versions are able to read and modify databases in the newer version without any error, but any ON UPDATE functionality is missing (which makes sense). Tables with ON UPDATE ... defined on them won't be displayed and performing an UPDATE won't trigger any of the defined expressions to run. So, this should be a Feature Verison bump.
  companion pr: https://github.com/dolthub/go-mysql-server/pull/2188
  fixes https://github.com/dolthub/dolt/issues/6030
• 7114: .github: Iterate on getting bats tests running with lambdabats.

go-mysql-server

• 2203: guard ctx.Span with nil check
  We have places where we call expression.Eval(nil, nil).
  The null context causes a panic when we call Span on it.
  This PR just adds a nil check inside the *context.Span() receiver
  fixes https://github.com/dolthub/dolt/issues/7154
• 2200: Replace carder with stats
  I wanted to separate 1) moving cardinality into stats objects, and 2) changing costing to start at join relations, but I kept finding small bugs and hard a time separating the two. So i did the move, fixed bugs along the way, and then regenerated plans with the changes.
  Our current join plans are less useful after we start doing costed join reordering, which makes it kind of annoying that these refactors are dominated by mostly irrelevant plan changes. Still I think the current refactors were necessary, because we need the costing changes and users will still use stats-less tables. Soon I will start adding more unit-esque plan tests that are equally comprehensive but test fewer things, like whether certain plans' cardinalities are accurate, and how closely cost estimates reflect execution runtime.
  Dolt side is passing: https://github.com/dolthub/dolt/pull/7156
• 2199: remove delimiter before caching the prepared stmt
  When using prepared statement, some create statements were being stored in the cache with delimiter at the end of it, which does not trimmed when executing the prepared statement. This causes consistency issue where in cases of storing a create statement, it will have the delimiter whereas running the same query without it being prepared statement trims before storing.
• 2197: prevent RangeJoinHeap plans over Filter nodes
  Currently, we don't have proper handling of nodes that aren't RangeHeaps in the RangeJoinIter.
  This PR makes it so that we don't create plans that would have these nodes.
  Additionally, we error instead of panicking when we see an invalid node in this area.
  fixes https://github.com/dolthub/dolt/issues/7139
• 2194: fix decimal result scale difference on server
  This PR fixes:
  • converting the decimal type values to given decimal column type before inserting.
  • getting precision and scale of final decimal type correctly.
• 2191: Move index costing into join planning phase
  Put index costing inside join planning, so that in the future join planning will have better cardinalities (statistics) for join ordering. Most of the changes will look like refactoring the way we expression index lookups in the memo. I attempted to do this in a way that makes as few changes as possible to join planning; the goal here is to set me up for rewriting cardinality checks with stats objects. It didn't go as cleanly as I wanted, I ended up shifting a lot of join plans back to lookup plans because HASH_JOIN was beating LOOKUP_JOIN in several key places.
  One downside of the current PR is that it converts a sysbench MERGE_JOIN into a LOOKUP_JOIN. I would prefer fixing this in the next PR when I do a bigger costing overhaul.
  Variety of fixes for join hinting, correctness, etc.
  At some point we appeared to fix this:
  https://github.com/dolthub/go-mysql-server/issues/1893
• 2188: support on update current_timestamp
  This PR adds support for handling and executing ON UPDATE column expressions, which are triggered when an update is applied to any column in the row.
  TODO:
  • we support now() for default, but not the syntax for on update
    fixes https://github.com/dolthub/dolt/issues/6030

Closed Issues

• 7182: Previous panic concerning CASE WHEN
• 6030: ON UPDATE CURRENT_TIMESTAMP in column definition does not update TIMESTAMP
• 6113: Warning during csv import
• 7154: Panic when Using BETWEEN and CASE WHEN
• 7148: UPDATE counters SET value = value + 1 WHERE key = 'foo'; should be concurrency-safe
• 7164: How to connect a dolt sql-server from a remote server

Latency