This is a patch release on the Q2 release to remove the NewDefaultServer method, which was was panicking at query time when used.

go-mysql-server's sql interfaces are not guaranteed to settle until 1.0.

Merged PRs

go-mysql-server

• 2437: lowercase when looking up self referential foreign key columns fixes: https://github.com/dolthub/dolt/issues/7700
• 2436: Making @@server_id default value match MySQL
• 2434: Fixing the default value for the binlog_checksum system variable Small change to make the default value of the global binlog_checksum system variable match MySQL's default value (i.e. "CRC32").
• 2433: NULL to nil The SHOW FIELDS/COLUMNS FROM <table> query would return the string "NULL" for Default column rather than nil. This mattered for Knex, which relied on it being NULL and not "NULL". fixes: https://github.com/dolthub/dolt/issues/7692
• 2432: support Threads_connected and Threads_running status variables This PR adds support for Threads_connected and Threads_running status variables. Additionally, the local enginetest are flaking consistently in dolt ci, so those have been removed; we have handler tests for com_delete, com_insert, and com_update anyway. Related: https://github.com/dolthub/dolt/issues/7646
• 2431: Setting Innodb_buffer_pool_pages_total to 1, to avoid an issue with Datadog's collector Datadog's metric collector errors out with a divide by zero error if the Innodb_buffer_pool_pages_total status variable is 0; changing it to 1 avoids this and allows the agent to collect metrics from Dolt.
• 2430: have status variables use go routines This PR changes Status Variables to update through go routines, to avoid slowing down query execution due to the mutexes present.
• 2429: server trace time includes parsing
• 2427: support Com_delete, Com_insert, Com_update status variables related: https://github.com/dolthub/dolt/issues/7646
• 2426: use @@session.collation_server during create database ... This PR makes it so create database ... actually reads the @@session.collation_server variable. Additionally, this ensures that settings @@character_set_server sets @@collation_server and vice versa. Interestingly, it seems like MySQL actually ignores the global scope of these system variables, and reads the session scope instead. fixes https://github.com/dolthub/dolt/issues/7651
• 2423: Adding test for preparing time.Time types This PR adds tests for using time.Time, some tests have to be skipped because we don't support Timespan correctly. companion pr: https://github.com/dolthub/vitess/pull/327 https://github.com/dolthub/vitess/pull/328 test for https://github.com/dolthub/dolt/issues/7665
• 2422: Support Questions status variable This PR adds logic to update status variable Questions. This only works in the server context, probably doesn't through dolt sql cli. https://github.com/dolthub/dolt/issues/7646
• 2421: [stats] costed index scan perf Histogram copying is expensive. Instead pass and mutate references. We have to use a different struct type to load stats from JSON in order to support histogram interface generalization. related Dolt-side: https://github.com/dolthub/dolt/pull/7666
• 2420: support case-insensitive LIKE for show status/variables MySQL stores session and global variables in a performance_schema database, and these tables have a case-insensitive collation on the variable names. This PR emulates that behavior by hard coding the collation the schemas for ShowStatus and ShowVariables nodes.
• 2419: Bug fix: Allow JSON scalar comparison between int64 and float64 When comparing JSON values, numbers may be represented internally as an int64 or float64, but our comparison code wasn't casting an int64 to a float64 in order to compare it with a float64 value. Fixes https://github.com/dolthub/dolt/issues/7656
• 2418: fix show create database to actually show charset/collation This PR fixes the SHOW CREATE DATABASE ... statement to actually show the charset/collation that the db is under instead of always default. Additionally, this PR parses the charset database option, instead of ignoring it like before. partially fixes: https://github.com/dolthub/dolt/issues/7651
• 2416: /{.github,go.mod,go.sum}: bump go version
• 2414: stubbing out status variables This PR adds the initial implementation of Status Variables. There are 682 status variables, and are very similar to System Variables. Every variable is read-only (and can only be updated by the server itself), and there are session-specific variables. MySQL Docs: https://dev.mysql.com/doc/refman/8.0/en/server-status-variable-reference.html Related: https://github.com/dolthub/dolt/issues/7646
• 2412: New interface for binlog primary callbacks First pass on connecting the GMS layer with the Dolt layer for handling callbacks when the SQL server is acting in binlog primary mode, through the new BinlogPrimaryController interface. This new interface pretty closely mirrors the existing callback interface for replica callbacks, the BinlogReplicaController interface. Related to https://github.com/dolthub/dolt/issues/7512
• 2411: implement json_search() MySQL Docs: https://dev.mysql.com/doc/refman/8.3/en/json-search-functions.html#function_json-search
• 2410: Adding system variable innodb_autoinc_lock_mode We currently only support innodb_autoinc_lock_mode = 2, not 0 or 1. MySQL Docs: https://dev.mysql.com/doc/refman/8.0/en/innodb-auto-increment-handling.html related: https://github.com/dolthub/dolt/issues/7634
• 2404: Improve handling of unsigned and decimal types in JSON Fixes https://github.com/dolthub/go-mysql-server/issues/2391 MySQL's JSON type differs from standard JSON in some important ways. It supports types not supported in standard JSON, such as separate types for integers and floats, an unsigned int type, and a decimal type. Prior to this PR, we would convert values to JSON by using the encodings/json package to marshall the value to a JSON string and then unmarshall it to a go map. This is not only slow, but it's incorrect for these additional types. The main purpose of this PR is to add special handling for these types that allow them to be stored in JSON documents. We also avoid generating and parsing JSON in places where it's not actually necessary, and fix bugs where decimals get incorrectly converted into strings, or unsigned ints get converted into signed ints. Finally, this fixes an issue where we send incorrect bytes for JSON-wrapped decimal values along the wire.
• 2403: fix dbName not being used in the example This PR makes sure that dbName in the example is actually being used, instead of having a hardcoded "mydb" in createTestDatabase. fixes [#2402]
• 2401: refactor and parse table options, support auto_increment table option Table Options are now parsed as structs, so we can read/use some of the variables. Character Sets, Collations, Create Table, TableSpec, etc. have been refactored. Additionally, this PR adds support to parse and use the auto_increment table option. TODO:
• CREATE TABLE ... LIKE ... needs to preserve table opts, like comments
• alter table add column ... auto_increment does not work when there are already rows Companion PR: https://github.com/dolthub/vitess/pull/322
• 2399: fix custom insert ordering for pk fixes https://github.com/dolthub/go-mysql-server/issues/2397
• 2398: fix in-memory implementation of RenameTable to read from session The in-memory implementation of RenameTable uses data from the BaseDatabase, instead of reading it from the session. This is problematic when there are multiple alter statements. Additonally, includes some small refactor so all functions are pointer receiver instead of a mix. fixes https://github.com/dolthub/go-mysql-server/issues/2396
• 2394: Bug fix: Set non-boolean system variable enum values to 'ON' or 'OFF' We were automatically converting ON and OFF values to to true and false when setting a system variable, which made it impossible to set system variables to those enum values. For example: sql SET @@GLOBAL.gtid_mode='ON'; Variable 'gtid_mode' can't be set to the value of 'true'
• 2393: Restored and refactored missing functionality required by Dolt
• 2392: implement sql_mode = 'NO_AUTO_VALUE_ON_ZERO' This PR implements the sql_mode NO_AUTO_VALUE_ON_ZERO. This makes it so that 0 values (not NULL) do not increment the auto_increment counter. MySQL Docs: https://dev.mysql.com/doc/refman/8.3/en/example-auto-increment.html fixes https://github.com/dolthub/dolt/issues/7600
• 2390: Remove the NewDefaultServer, which doesn't work for any included session implementation Fixes https://github.com/dolthub/go-mysql-server/issues/2364 Added a memory.NewSessionBuilder method, now used in the example code.
• 2387: exit early when IF NOT EXISTS and table exists This PR addresses various issues related to CREATE TABLE IF NOT EXISTS ... queries. Before, we simply ignored the table exists error, and continued creating indexes, foreign keys, and checks. This led to errors when attempting to create indexes/foreign keys/checks that already exists. Additionally, it would errorneously create indexes/foreng keys/checks that did exist. The correct behavior is to do nothing if IF NOT EXISTS is specified and the table exists. Also this contains some refactors and simplifications. fixes https://github.com/dolthub/dolt/issues/7602
• 2386: ignore large tokens in fulltext indexes We were panicking when attempting to insert/delete tokens that exceed the column type length. It appears as though MySQL simple ignores these tokens. fixes: https://github.com/dolthub/dolt/issues/7593
• 2385: optimize sql.HashOf
• pool *xxhash.Digest objects
• use fmt.Fprintf to write to hash benchmark stats oos: linux goarch: amd64 pkg: github.com/dolthub/go-mysql-server/sql cpu: AMD Ryzen 9 7900 12-Core Processor │ b1 │ b2 │ │ sec/op │ sec/op vs base │ HashOf-24 79.65n ± 4% 70.86n ± 7% -11.03% (p=0.002 n=6) ParallelHashOf-24 10.47n ± 4% 11.85n ± 19% ~ (p=0.368 n=6) geomean 28.88n 28.98n +0.32% │ b1 │ b2 │ │ B/op │ B/op vs base │ HashOf-24 4.000 ± 0% 0.000 ± 0% -100.00% (p=0.002 n=6) ParallelHashOf-24 4.000 ± 0% 0.000 ± 0% -100.00% (p=0.002 n=6) geomean 4.000 ? ¹ ² ¹ summaries must be >0 to compute geomean ² ratios must be >0 to compute geomean │ b1 │ b2 │ │ allocs/op │ allocs/op vs base │ HashOf-24 2.000 ± 0% 0.000 ± 0% -100.00% (p=0.002 n=6) ParallelHashOf-24 2.000 ± 0% 0.000 ± 0% -100.00% (p=0.002 n=6) geomean 2.000 ? ¹ ² ¹ summaries must be >0 to compute geomean ² ratios must be >0 to compute geomean
• 2383: promote string lookup range types When performing range lookups, we convert the key to the type of the column. The conversion throws an error when the key doesn't fit within the type for the index. The fix is to promote these (only for StringType) so the ranges fit. There were issues with type.Promote() for all types. Additionally, there are some inconsistencies with MySQL when performing these checks with NUL characters (\0). They are skipped tests for now. related https://github.com/dolthub/dolt/issues/7588
• 2382: add support for json_pretty MySQL Docs: https://dev.mysql.com/doc/refman/8.0/en/json-utility-functions.html#function_json-pretty
• 2381: [memory] force mutating the editAcc AutoInc because tableEditor is unreliable I can't figure a clean way to get the insert editor's edit accumulator and table editor data in sync when a self-referential foreign key initializes the session editor during analysis. So I just forced us to mutate the edit accumulator's auto increment id, which should prevent bugs of the kind we've been seeing. Zach might have a better understanding of how this should work. fixes: https://github.com/dolthub/go-mysql-server/issues/2369
• 2380: Stubbing out support for COM_BINLOG_DUMP_GTID command Stubbing out support for two new mysql.Handler methods to support streaming binlog events from a server to a client. Depends on Vitess PR: https://github.com/dolthub/vitess/pull/317 Related to https://github.com/dolthub/dolt/issues/7512
• 2378: map straight join to inner join This PR temporarily remaps STRAIGHT_JOIN operator to INNER_JOIN operator. fixes https://github.com/dolthub/dolt/issues/7580
• 2377: fix binary cast to maintain length We were improperly dropping CAST node during comparison. It might be fine in many cases, but it is definitely not for comparing BINARY column types.
• 2376: return errors for charset/collation mismatch
• 2375: create system variable interface This PR creates SystemVariable and SystemVariableScope interfaces. This allows doltgres to use these interfaces for defining and handling all the configuration parameters. The SystemVariable struct is renamed to MysqlSystemVariable. The SystemVariableScope byte is renamed to MysqlSVScopeType.
• 2373: charset table option tests In GMS, we reparse table options with a regular expression, but we only cover CHARACTER SET and not its synonym CHARSET. As a result, we just ignore table options for CHARSET. The fix is in https://github.com/dolthub/vitess/pull/315 TODO: maybe should just address this TODO instead...
• 2372: implement JSON_QUOTE() MySQL Docs: https://dev.mysql.com/doc/refman/8.0/en/json-creation-functions.html#function_json-quote
• 2371: don't use all caps for engines info table Some clients rely on the correct casing of column names For some reason, the casing is different when performing SHOW ENGINES; vs select * from information_schema.engines; Fortunately, the way we do SHOW ENGINES; is actually a SELECT ... under the hood. So the hacky fix is to just use an alias :) related https://github.com/dolthub/dolt/issues/7574
• 2368: support JSON_MERGE() and JSON_MERGE_PATCH() MySQL Docs:
• https://dev.mysql.com/doc/refman/8.0/en/json-modification-functions.html#function_json-merge
• https://dev.mysql.com/doc/refman/8.0/en/json-modification-functions.html#function_json-merge-patch
• 2367: implement JSON_OVERLAPS() MySQL Docs: https://dev.mysql.com/doc/refman/8.0/en/json-search-functions.html#function_json-overlaps
• 2366: Update README with latest sample code from _example/main.go The latest release of go-mysql-server requires a different way of instantiating the database server. Notably, a function that provides sessions must now be provided. This change updates our README with the latest working example code from _example/main.go. Related to https://github.com/dolthub/go-mysql-server/issues/2364
• 2365: implement random_bytes() MySQL Docs: https://dev.mysql.com/doc/refman/8.0/en/encryption-functions.html#function_random-bytes
• 2362: Feature: last_insert_uuid() function UUIDs are often used in place of auto_increment IDs, but MySQL doesn't provide an easy way to get the last generated UUID that was used in an insert. This change introduces a new function, last_insert_uuid() that operates similarly to last_insert_id(). For a column identified as a UUID column, callers can use last_insert_uuid() to retrieve the last generated UUID value that was inserted into that column. In order to be considered a UUID column, a column must be part of the primary key and it must meet one of the following type signatures:
• VARCHAR(36) or CHAR(36) with a default value expression of UUID()
• VARBINARY(16) or BINARY(16) with a default value expression of UUID_to_bin(UUID()) (optionally, the swap_flag for UUID_to_bin may also be specified) #### Example usage: sql create table t (pk binary(16) primary key default (UUID_to_bin(UUID())), c1 varchar(100)); insert into t (c1) values ("one"), ("two"); select last_insert_uuid(); select c1 from t where pk = uuid_to_bin(last_insert_id()); Related to https://github.com/dolthub/dolt/issues/7547
• 2361: implement ADDDATE() MySQL Docs: https://dev.mysql.com/doc/refman/8.0/en/date-and-time-functions.html#function_adddate
• 2360: fix special case for now() string The BATS tests import-mysqldump.bats was failing due to a recent string replace case added by https://github.com/dolthub/go-mysql-server/pull/2357. I didn't account for when NOW() would be used with other functions inside a DEFAULT column expression, and was replacing the "NOW(" with "CURRENT_TIMESTAMP(". This was causing problems as we reparse ColumnDefaultExprs, resulting in unparenthesized default expressions, which threw an error.
• 2359: unskip fixed tests There are some tests that are fixed, but we haven't unskipped them. Additionally, this cleans up the way we skip certain tests so that it is more consistent.
• 2357: fix ON UPDATE CURRENT_TIMESTAMP precision I had mistaken assumed that MySQL did not allow any precision arguments for NOW() (and synonyms) for column DEFAULT and ON UPDATE expressions. It turns out MySQL only requires that the column type precision match the expression. Additionally, we did not perform these error checks for DEFAULT expression, so now we do. fixes https://github.com/dolthub/dolt/issues/7555
• 2351: implement JSON_TYPE() MySQL Docs: https://dev.mysql.com/doc/refman/8.0/en/json-attribute-functions.html#function_json-type Companion PR: https://github.com/dolthub/vitess/pull/314

vitess

• 329: Changes to binlog event creation functions
• Exposing the Length() function in the BinlogEvent interface so calling code can access the event size present in a binlog event's header. Needed for calculating binlog file position.
• Renaming FakeBinlogStream → BinlogStream so calling code can use it when serializing binlog events.
• 328: revert decimals issue: https://github.com/dolthub/vitess/pull/328
• 327: add case for time.Time and decimal in bindvars We were unable to use time.Time and decimal type variables bind vars, so they couldn't be used as arguments to prepare statements. This PR addresses that issue. fixes: https://github.com/dolthub/dolt/issues/7665
• 325: /{.github,go.mod,go.sum}: bump go version
• 324: Ensure that float values parsed into expressions will always be parsed back into floats, not decimals.
• 323: Parser support for SOURCE_AUTO_POSITION When acting as a replica, Dolt implicitly assumes SOURCE_AUTO_POSITION is set to 1. This adds parser support so that it can be explicitly specified. Adding this so that when we're configuring a MySQL replica, we can use the same setup code without having to special case this parameter.
• 322: parse table options into struct Instead of just making a large string that is reparsed in GMS, parse table_options into a list of structs similar to how we handle index_options and constraint_options.
• 321: [ast] walk tableFuncExpr for bind variables
• 320: Updates for binlog primary protocol
• Porting over more code from vitess.io/vitess – support for the COM_REGISTER_REPLICA command.
• Fixing a bug in Vitess' deserialization of the COM_BINLOG_DUMP_GTID command.
• Adding some String functions to help with debugging.
• Exposing the ability to flush a connection's buffer, as a short-term workaround for needing to flush the buffer more frequently than at the end of the connection in order to support `COM_BINLOG_DUMP_GTID'.
• 319: make constraint name optional for primary key fixes https://github.com/dolthub/dolt/issues/7601
• 318: Support for more binlog statements Adding parsing support for:
• show replicas
• show binary logs
• show binary log status
• 317: Port: Support for Vitess server to send binlog events Porting over support from the main Vitess repo for a server to send back binlog events over a connection. Also includes support for handling the COM_BINLOG_DUMP_GTID command. Related to https://github.com/dolthub/dolt/issues/7512
• 315: support binary charset in table option This PR adds binary as a valid charset option. Additionally, this changes CHARSET to expand to synonym CHARACTER SET for parsing in GMS. fixes https://github.com/dolthub/dolt/issues/7576
• 314: support casting to year
• 313: supports FROM s.account, in which account is non-reserved keyword

Closed Issues

• 2402: Consistent usage of variables in the example
• 2391: Potential regression: number cast to JSON no longer read as float
• 2396: Running multiple migrations in a transaction
• 2397: Primary key column order changes column order on insert
• 2369: Self-referencing foreign key constraint breaks auto-incrementing ids in memory mode
• 2341: Does it support Functional Key Parts index ?
• 2349: Foreign key constraints break auto-incrementing ids in memory mode
• 1157: show index from table_name doesn't include primary key
• 1340: Hangs on TPC-H Query 1
• 399: sql.FilteredTable usage and implementation
• 514: benchmark memory db
• 807: Documentation of some interfaces and some important concepts
• 1567: navicat connect DB but can not edit
• 2213: UPDATE statements are Noop
• 2215: any plan to build new release?