MySQL Community Server 8.0.2-dmr has been released (part 1/2)

Lars Tangvald <[email protected]> Tue, 18 Jul 2017 08:36:15 +0200

Newsgroups	gmane.comp.db.mysql.general,gmane.comp.db.mysql.packagers
Message-ID	<[email protected]>

Dear MySQL users,

[Due to file size limitations, the announcement is split in 2
parts. This is part 1.]

MySQL Server 8.0.2-dmr (Milestone Release) is a new version of the world's
most popular open source database. This is the first public milestone
release of MySQL 8.0.

http://dev.mysql.com/doc/mysql-development-cycle/en/development-milestone-releases.html

As with any other pre-production release, caution should be taken when
installing on production level systems or systems with critical data.

For information on installing MySQL 8.0.2-dmr on new servers, please see
the MySQL installation documentation at

http://dev.mysql.com/doc/refman/8.0/en/installing.html

MySQL Server 8.0.2-dmr is available in source and binary form for a
number of
platforms from the "Development Releases" selection of our download
pages at

http://dev.mysql.com/downloads/mysql/

MySQL Server 8.0.2-dmr is also available from our repository for Linux
platforms, go here for details:

http://dev.mysql.com/downloads/repo/

Windows packages are available via the Installer for Windows:

http://dev.mysql.com/downloads/installer/

along with .ZIP (no-install) packages for more advanced needs.

8.0.2-dmr also comes with a web installer as an alternative to the full
installer.

The web installer doesn't come bundled with any actual products
and instead relies on download-on-demand to fetch only the
products you choose to install. This makes the initial download
much smaller but increases install time as the individual products
will need to be downloaded.

We welcome and appreciate your feedback, bug reports, bug fixes,
patches, etc.:

http://bugs.mysql.com/report.php

The following link lists the changes in the MySQL 8.0.2-dmr since
the release of MySQL 8.0.1.

https://dev.mysql.com/doc/relnotes/mysql/8.0/en/news-8-0-2.html

Enjoy!

Changes in MySQL 8.0.2 (2017-07-17, Development Milestone)

Note

This is a milestone release, for use at your own risk.
Significant development changes take place in milestone
releases and you may encounter compatibility issues, such as
data format changes that require attention in addition to the
usual procedure of running mysql_upgrade. For example, you
may find it necessary to dump your data with mysqldump before
the upgrade and reload it afterward.

Account Management Notes

* During data directory initialization or upgrade, MySQL
now creates a 'mysql.session'@'localhost' reserved
account. This account is used internally by plugins to
access the server. It is locked so that it cannot be used
for client conections. (Bug #25642343)

* These system variables now are available to permit
automatic assignment of roles and to permit granted roles
to be automatically activated at client connection time:

+ mandatory_roles takes a value listing roles that the
server should treat as automatically granted to all
users.

+ activate_all_roles_on_login, if enabled, causes all
roles granted to users to be made active at client
connection time.

Character Set Support

* For Unicode data that uses NO PAD collations, sorting of
multibyte and variable-length values has been improved:

+ NO PAD collations are those based on UCA 9.0.0 and
higher, such as utf8mb4_0900_ai_ci or
utf8mb4_ja_0900_as_cs.

+ The performance improvement is greatest for sparse
key values; that is, strings that do not fill their
entire permitted length. For a VARCHAR(10) column
that uses the utf8mb4_0900_ai_ci collation, values
may take up to 40 bytes. The string 'a' is more
sparse than 'abcdefghij'. But even 'abcdefghij' uses
only 10 bytes of a possible 40 and is more sparse
than a string of 10 emojis. The emoji string is
dense because each character uses 4 bytes for a
resulting string that requires the entire 40 bytes
available.
(Bug #25750527, Bug #85546)

* MySQL now supports a new collation, utf8mb4_0900_as_ci,
for the utf8mb4 Unicode character set. This collation is
accent sensitive and case insensitive. It is similar to
the default utf8mb4 collation (utf8mb4_0900_ai_ci) except
that the default collation is accent insensitive.
MySQL also now supports a new Japanese collation,
utf8mb4_ja_0900_as_cs_ks, for the utf8mb4 Unicode
character set. This collation is like
utf8mb4_ja_0900_as_cs in that it is accent sensitive and
case sensitive, but utf8mb4_ja_0900_as_cs_ks is also kana
sensitive and distinguishes Katakana characters from
Hiragana characters. utf8mb4_ja_0900_as_cs treats
Katakana and Hiragana characters as equal for sorting.
Applications that require a Japanese collation but not
kana sensitivity may use utf8mb4_ja_0900_as_cs for better
sort performance. utf8mb4_ja_0900_as_cs uses three weight
levels for sorting; utf8mb4_ja_0900_as_cs_ks uses four.
utf8mb4_ja_0900_as_cs_ks is the first collation to use
the _ks collation suffix. Japanese collations without
this suffix are not kana sensitive.
For more information, see Unicode Character Sets
(http://dev.mysql.com/doc/refman/8.0/en/charset-unicode-sets.html).

* These character set changes were made to MySQL client
support:

+ These clients use a default character set of utf8mb4
rather than latin1: mysql, mysql_upgrade,
mysqladmin, mysqlcheck, mysqldump, mysqlimport,
mysqlpump, mysqlslap, mysqlshow, and mysqltest.
(This change does not affect character set
autodetection performed by mysql, mysqladmin,
mysqlcheck, mysqlimport, and mysqlshow, as described
at Connection Character Sets and Collations
(http://dev.mysql.com/doc/refman/8.0/en/charset-connection.html)).

+ Client connections created using the libmysqlclient
library use a default character set of utf8mb4
rather than latin1.

+ The mysqltest program supports a
--default-character-set option for specifying the
character set explicitly.

Compilation Notes

* Performance: The strict aliasing optimization for GCC is
no longer disabled, which results in a minor improvement
for MySQL single-threaded performance.

* Linux: MySQL now compiles on Alpine Linux. (Bug
#25945568, Bug #80322)

* Solaris: On Solaris, server builds now use std::atomic,
so they must be linked against libstatomic. (Bug
#25957991)

* MySQL now compiles using Visual Studio 2017. (Bug
#25788406)

* The required version of the Boost library for server
builds has been raised to 1.64.0. (Bug #25772329, Bug
#85593)

* These changes were made with respect to client program
development:

+ Client programs should only need to #include the
<mysql.h> header file. In particular, <my_config.h>
should not be needed, and is no longer installed.

+ The my_init() function is no longer included in the
list of symbols exported from libmysqlclient. It
need not be called explicitly by client programs
because it is called implicitly by other C API
initialization functions.
(Bug #25732787)

* For the Xcode IDE, header files are added to MySQL
project sources so they can be searched. (Bug #25636986)

* Work was done to clean up the source code base,
including: Removing unneeded CMake checks; removing
unused macros from source files; reorganizing header
files to reduce the number of dependencies and make them
more modular, removing function declarations without
definitions, replacing locally written functions with
equivalent functions from industry-standard libraries.

Component Notes

* Two new services are available to enable server
components and plugins to register and unregister
user-defined-functions (UDFs):
mysql_service_udf_registration and
mysql_service_udf_registration_aggregate provide
registration services for scalar and aggregate UDFs,
respectively. These services enable components and
plugins to manage UDFs for themselves, without the need
for CREATE FUNCTION and DROP FUNCTION statements.
UDFs registered using either these services or CREATE
FUNCTION now are listed in the new Performance Schema
user_defined_functions table. See The
user_defined_functions Table
(http://dev.mysql.com/doc/refman/8.0/en/user-defined-functions-table.html).

* The mysql_string string-manipulation service has been
converted from a plugin service to a component service: a
set of string service APIs for use by server components.
The password validation component has been updated to use
the revised service implementation. For information about
services, see MySQL Services for Components and Plugins
(http://dev.mysql.com/doc/refman/8.0/en/component-plugin-services.html).

Configuration Notes

* Previously, LOCAL capability for LOAD DATA operations was
enabled by default in some contexts: The client library
in MySQL binary distributions was compiled with
client-side LOCAL capability enabled, and the
local_infile system variable was enabled on the server
side. LOCAL capability is now disabled by default in all
contexts. Client programs must be configured explicitly
to enable LOCAL, and the server must be run with
local_infile enabled. See Security Issues with LOAD DATA
LOCAL
(http://dev.mysql.com/doc/refman/8.0/en/load-data-local.html).
(Bug #24511108)

* mysqld startup behavior has been modified as follows. In
general, these changes enable more consistent or easier
server startup when not using mysqld_safe.

+ The server executable determines its own full path
name at startup and uses the parent of the directory
in which it is located as the default basedir value.
This in turn enables the server to use that basedir
when searching for server-related information such
as the share directory containing error messages.

+ If error log output cannot be redirected to the
--log-error option value, error output could be
lost. This is now less likely.

+ If the server is started using the --daemonize
option and is not connected to a tty device, a
default error logging option of --log-error="" is
used in the absence of an explicit logging option,
to direct error output to the default log file.
Previously, an error could occur under these
circumstances.

+ The -D and -I options now are synonyms for
--daemonize and --initialize, respectively.
(Bug #20398088, Bug #75343)

* MySQL now supports a SET PERSIST_ONLY variant of SET
statement syntax, for making configuration changes at
runtime that also persist across server restarts. Like
SET PERSIST, SET PERSIST_ONLY writes the variable setting
to an option file named mysqld-auto.cnf in the data
directory. However, unlike PERSIST, PERSIST_ONLY does not
modify the runtime global system variable value. This
makes PERSIST_ONLY suitable for configuring read-only
system variables that only be set can at server startup.
For more information, see Using Option Files
(http://dev.mysql.com/doc/refman/8.0/en/option-files.html),
and SET Syntax for Variable Assignment
(http://dev.mysql.com/doc/refman/8.0/en/set-variable.html).
Use of SET PERSIST_ONLY requires the new
PERSIST_RO_VARIABLES_ADMIN privilege.

* The explicit_defaults_for_timestamp system variable is
now enabled by default (previously disabled by default),
and a warning occurs if you disable it. This means that
the nonstandard (and deprecated) behaviors for default
values and NULL-value handling in TIMESTAMP columns are
now disabled by default.

* Symbolic link support as described at Using Symbolic
Links for MyISAM Tables on Unix
(http://dev.mysql.com/doc/refman/8.0/en/symbolic-links-to-tables.html),
along with the --symbolic-links option
that controls it, is now deprecated and will be removed
in a future version of MySQL. In addition, the option is
now disabled by default. The related have_symlink system
variable also is deprecated and will be removed in a
future version of MySQL.

Data Dictionary Notes

* The column_stats system table has been removed and
replaced by the column_statistics data dictionary table.

* The version data dictionary table was renamed to
dd_properties.
Metadata for INFORMATION_SCHEMA tables created by dynamic
plugins now is recorded in the data dictionary and
visible through the INFORMATION_SCHEMA TABLES table.
These tables are no longer visible in INFORMATION_SCHEMA:
SHOW_STATISTICS, SHOW_STATISTICS_DYNAMIC,
STATISTICS_BASE, STATISTICS_DYNAMIC, TABLES_DYNAMIC. User
impact is minimal because these tables are for internal
use by server components.

* The InnoDB storage engine now uses the global MySQL data
dictionary rather than its own storage engine-specific
data dictionary. For information about the data
dictionary, see MySQL Data Dictionary
(http://dev.mysql.com/doc/refman/8.0/en/data-dictionary.html).
The following list briefly describes the main
implications of this change:

+ Upgrade and downgrade implications:
o To upgrade from MySQL 5.7 to MySQL 8.0, you
must perform the upgrade procedure described at
Upgrading MySQL
(http://dev.mysql.com/doc/refman/8.0/en/upgrading.html).
o Downgrading from MySQL 8.0 to MySQL 5.7 is only
supported using the logical downgrade method (a
mysqldump downgrade). In-place downgrades are
not supported.

+ Metadata updates associated with exporting and
importing tablespaces using the transportable
tablespace feature are now performed on global data
dictionary tables instead of InnoDB data dictionary
tables.

+ InnoDB in-memory metadata is now instantiated from
global data dictionary objects. This metadata was
previously read from InnoDB system tables.

+ Table options that signify tablespace encryption and
transparent page compression are now retrieved from
the global data dictionary.

+ Data dictionary support was added for InnoDB
FULLTEXT indexes. Auxiliary index table names were
changed to lowercase.

+ InnoDB metadata created or modified during DDL
operations is now written to the global data
dictionary.

* mysql system tables and data dictionary tables are now
created in a single InnoDB tablespace file named
mysql.ibd in the MySQL data directory. Previously, these
tables were created in individual InnoDB tablespace files
in the mysql database directory. Associated changes
include:

+ The InnoDB data dictionary buffer table, which
stores fast changing InnoDB metadata, was renamed to
innodb_dynamic_metadata and moved from the InnoDB
system tablespace to the data dictionary tablespace.

+ Undo tablespace metadata now resides in the data
dictionary tablespace, which allows InnoDB to locate
undo tablespaces at startup.

+ Temporary tablespace metadata now resides in the
data dictionary tablespace.

+ Undo tablespace and temporary tablespace metadata is
found the mysql.tablespaces data dictionary table,
which makes the data visible to other data
dictionary tables.

+ Table definitions for tables created in the mysql
tablespace, the InnoDB system tablespace
(innodb_system), or general tablespaces now include
a TABLESPACE attribute. This change has the
following implications:
o You cannot use CREATE TEMPORY TABLE ... LIKE to
create an empty table based on the definition
of a table that resides in any of the
aforementioned tablespaces, as these
tablespaces do not support temporary tables.
o CREATE TABLE ... LIKE preserves the TABLESPACE
attribute of the original table and creates a
new table in the defined tablespace regardless
of the innodb_file_per_table setting. This is a
temporary regression.
For more information and workarounds, see CREATE
TABLE ... LIKE Syntax
(http://dev.mysql.com/doc/refman/8.0/en/create-table-like.html),
and CREATE TEMPORARY TABLE Syntax
(http://dev.mysql.com/doc/refman/8.0/en/create-temporary-table.html).

Optimizer Notes

* MySQL now enables management of histogram statistics for
table column values:

+ The ANALYZE TABLE statement supports UPDATE
HISTOGRAM and DROP HISTOGRAM clauses for generating
and removing column histogram statistics.

+ The server stores histogram information in the
column_statistics data dictionary table. Histograms
are viewable using the
INFORMATION_SCHEMA.COLUMN_STATISTICS table.

+ The histogram_generation_max_mem_size system
variable controls the amount of memory available for
histogram generation.

+ The Performance Schema has a memory/sql/histograms
instrument for monitoring memory allocations
performed for histogram generation.

+ mysqldump and mysqlpump have a --column-statistics
option to add ANALYZE TABLE statements to the output
to generate histogram statistics for dumped tables
when the dump file is reloaded.
For more information, see ANALYZE TABLE Syntax
(http://dev.mysql.com/doc/refman/8.0/en/analyze-table.html),
Optimizer Statistics
(http://dev.mysql.com/doc/refman/8.0/en/optimizer-statistics.html),
and The INFORMATION_SCHEMA COLUMN_STATISTICS
Table
(http://dev.mysql.com/doc/refman/8.0/en/column-statistics-table.html).

* MySQL now supports window functions that, for each row
from a query, perform a calculation using rows related to
that row. These include functions such as RANK(), LAG(),
and NTILE(). In addition, several existing aggregate
functions now can be used as window functions; for
example, SUM() and AVG(). For more information, see
Window Functions
(http://dev.mysql.com/doc/refman/8.0/en/window-functions.html).
Note
Each of the following words now is a reserved word and
cannot be used as an identifier without identifier
quoting: CUME_DIST, DENSE_RANK, FIRST_VALUE, GROUPS, LAG,
LAST_VALUE, LEAD, NTH_VALUE, NTILE, OVER, PERCENT_RANK,
RANK, ROW_NUMBER, WINDOW.

Packaging Notes

* mysqladmin was added to Docker/Minimal packages because
it is needed by InnoDB Cluster. (Bug #25998285)

* Debian/Ubuntu packages now support multiple MySQL
instances with systemd. See Managing MySQL Server with
systemd
(http://dev.mysql.com/doc/refman/8.0/en/using-systemd.html).
(Bug #24559588, Bug #82785)

Parser Notes

* The parser rules for ALTER TABLE were refactored to be
context independent and improve maintainability and
extensibility. A resulting effect is that some previously
accepted undocumented syntax variants are no longer
accepted. For example, CREATE TABLE statements were
permitted with column names qualified by the table name,
or by the current database and table name. Such
statements now produce an error.

* Support for these deprecated syntax constructs for table
and column references has been removed and their use now
results in an error. Instances of these constructs should
be changed to remove the leading period.

+ .col_name

+ .tbl_name

+ .tbl_name.col_name

Performance Schema Notes

* The Performance Schema now supports versioning, and
maintains the current definitions for its tables
internally. At startup, the server compares its supported
Performance Schema version with the Performance Schema
version stored in the data dictionary. If the versions
differ, the server drops any old Performance Schema
tables and recreates them using the current definitions.
In consequence of this change:

+ For MySQL upgrades, it is no longer necessary to run
mysql_upgrade to incorporate changes to Performance
Schema tables because they are recreated
automatically as necessary at server startup.

+ The mysql_system_tables.sql support script no longer
includes SQL statements for Performance Schema table
creation because these table definitions are
maintained internally.
To support dynamic Performance Schema table manipulation,
a new component service named pfs_table_service is now
available.

* The Performance Schema default instrumentation settings
have changed:

+ The transaction instrument and the
events_transactions_current and
events_transactions_history consumers are now
enabled by default. See Performance Schema
Transaction Tables
(http://dev.mysql.com/doc/refman/8.0/en/performance-schema-transaction-tables.html).

+ All memory/% memory instruments are now enabled by
default. See Memory Summary Tables
(http://dev.mysql.com/doc/refman/8.0/en/memory-summary-tables.html).

+ The wait/lock/metadata/sql/mdl metadata lock
instrument is now enabled by default. See The
metadata_locks Table
(http://dev.mysql.com/doc/refman/8.0/en/metadata-locks-table.html).

Security Notes

* Security Fix: The linked OpenSSL library for the MySQL
Commercial Server has been updated to version 1.0.2l.
Issues fixed in the new OpenSSL version are described at
http://www.openssl.org/news/vulnerabilities.html.
This change does not affect the Oracle-produced MySQL
Community build of MySQL Server, which uses the yaSSL
library instead. (Bug #26160962)

* Previously, any user could execute the XA RECOVER
statement to discover the XID values for outstanding
prepared XA transactions, possibly leading to commit or
rollback of an XA transaction by a user other than the
one who started it. Now XA RECOVER is permitted only to
users who have the new XA_RECOVER_ADMIN privilege, which
is expected to be granted only to administrative users
who have need for it. This might be the case, for
example, for administrators of an XA application if it
has crashed and it is necessary to find outstanding
transactions started by the application so they can be
rolled back. This privilege requirement does not affect
normal commit or rollback of an XA transaction because
the user who started it knows its XID.
For MySQL uprades, mysql_upgrade grants XA_RECOVER_ADMIN
to users who have the SUPER privilege, unless some
account is found that already has XA_RECOVER_ADMIN. (Bug
#17188129)

Spatial Data Support

* Argument suitability checking was improved for these
spatial functions: ST_Contains(), ST_Crosses(),
ST_Disjoint(), ST_Equals(), ST_Intersects(),
ST_Overlaps(), ST_Touches(), ST_Within(), MBRContains(),
MBRCoveredBy(), MBRCovers(), MBRDisjoint(), MBREquals(),
MBRIntersects(), MBROverlaps(), MBRTouches(),
MBRWithin(). For more information, see Spatial Relation
Functions That Use Object Shapes
(http://dev.mysql.com/doc/refman/8.0/en/spatial-relation-functions-object-shapes.html),
and MySQL-Specific Spatial
Relation Functions That Use Minimum Bounding Rectangles
(MBRs)
(http://dev.mysql.com/doc/refman/8.0/en/spatial-relation-functions-mysql-specific.html).

Test Suite Notes

* The --do-test-list option for mysql-test-run.pl accepts
an argument for a file containing tests one per line.
Lines in the file are now accepted in any of the
following formats:
test_name
test_name.test
suite_name.test_name
suite_name.test_name.test
path/to/test/file

(Bug #25700362)

* mysql-test-run.pl and mysqltest are now more restrictive
about permitted test case and result file names. Names
must consist only of alphanumeric characters (A-Z, a-z,
0-9), dash (-), or underscore (_), and cannot start with
dash or underscore. An error occurs for nonconforming
file names. (Bug #25487471)

* These changes were made to the --xml-report option for
mysql-test-run.pl:

+ A <failure> tag identifies any test failing on a
retry attempt.

+ Separate statistics and fields are included for
skipped and disabled tests.

+ The XML report is created in the build directory if
no absolute path is given for out-of-source builds.

+ For combination runs, a field named variation is
included in the <testcase> tag.

+ Results for all tests belonging to a suite are
aggregated within a single <testsuite> tag.

+ Information regarding failures is reported with a
brief reason in an attribute named message, along
with other details within the tag.
(Bug #25349924)

* mysql-test-run.pl now supports a --report-unstable-tests
option that has these effects:

+ Reports any test that has passed using at least one
retry attempt in a separate category called
"Unstable tests" in the summary.

+ If all failures are due to unstable tests,
mysql-test-run.pl produces a warning but exits
successfully.

+ Adds a new XML tag to report unstable tests, if the
--xml-report option is also specified.
(Bug #24473420, Bug #25984429)

* mysqltest now supports a replace_numeric_round command
that takes an argument value from 0 to 16 indicating the
number of decimals to round numeric values to. This can
be used to help prevent result content mismatch errors
for tests in which slightly different results are
obtained across platforms due to precision differences.
Thanks to Daniel Black for the patch. (Bug #23280117, Bug
#81399)

* mysql-test-run.pl now looks for a testname-client.opt
file, which is analogous to the testname-master.opt file
but for specifying test-specific client options. (Bug
#17084918)

Platform-Specific Notes

* Linux: The generic Linux build for MySQL 8.0 now supports
Non-Uniform Memory Access (NUMA) on its host system.
Systems that use the build need to have libnuma installed
on them. See Installing MySQL on Unix/Linux Using Generic
Binaries
(http://dev.mysql.com/doc/refman/8.0/en/binary-installation.html)
for more details. (Bug #26005558)

Functionality Added or Changed

* Incompatible Change; InnoDB: ADD PARTITION, DROP
PARTITION, COALESCE PARTITION, REORGANIZE PARTITION, and
REBUILD PARTITION ALTER TABLE options are now supported
by native partitioning in-place APIs and may be used with
ALGORITHM={COPY|INPLACE} and LOCK clauses.
EXCHANGE PARTITION, which previously supported ALGORITHM
and LOCK clauses, is performed by a new server layer API
for compatibility with the MySQL data dictionary.
Partition and table exchange is performed internally by
the native partition handler.
DROP PARTITION with ALGORITHM=INPLACE deletes data stored
in the partition and drops the partition. However, DROP
PARTITION with ALGORITHM=COPY or old_alter_table=ON
rebuilds the partitioned table and attempts to move data
from the dropped partition to another partition with a
compatible PARTITION ... VALUES definition. Data that
cannot be moved to another partition is deleted.

* InnoDB: InnoDB now uses tablespace map files during
recovery to identify tablespaces that require redo log
application. This is a change from redo log tablespace
discovery that was used previously.
If tablespace map files are lost or corrupted, the
innodb_scan_directories startup option may be used to
specify tablespace directories when starting MySQL after
a server outage.
For more information, see Tablespace Discovery During
Crash Recovery
(http://dev.mysql.com/doc/refman/8.0/en/innodb-recovery.html#innodb-recovery-tablespace-discovery).
(Bug #24793413)

* Replication: In previous versions issuing STOP
GROUP_REPLICATION stopped the plugin but there was a
period during this process when the server still accepted
transactions. This meant the transactions were not
transmitted to the group, and if the plugin failed to
stop correctly further transactions were accepted. To
make STOP GROUP_REPLICATION safer, now super_read_only is
set to ON immediately upon issuing STOP
GROUP_REPLICATION, which ensures no transactions are
accepted. (Bug #25495067, Bug #84795)

* Replication: When there was a network partition and a
member was in a minority all queries to that member
blocked. To improve this situation, the
group_replication_unreachable_majority_timeout variable
has been added which enables you to configure how long
members in a minority wait to regain contact with a
member in the majority before leaving the group. (Bug
#25473794)

* Replication: The group_replication_transaction_size_limit
variable was added to enable you to protect a group
against large transactions causing a failure. (Bug
#84785, Bug #25510757)

* Replication: Support for binary log files created by
versions earlier than MySQL 5.0 has been removed, and
binary log Version 1 and Version 3 formats are no longer
supported by slaves or mysqlbinlog.

* Replication: The replication_group_members and
replication_group_member_stats Performance Schema tables
have been extended to provide more monitoring information
for Group Replication. The replication_group_members
table now includes information about member's roles, and
the versions of MySQL running on a member. The
replication_group_member_stats table now includes
information about all members in the group, the applier
and local queues, and rolled back transactions.

* Replication: The receiver thread has been improved to no
longer block other thread's activities when waiting for
disk space. This improves the monitoring of replication
ensuring that it reports correctly when the receiver
thread is waiting for disk space. If you are not able to
free disk space to allow the receiver thread to continue
its activity, it can be forcefully stopped without side
effects in most cases.

* Replication: A new transaction length field has been
added to the Gtid_log_event which stores the transaction
length in bytes.

* Replication: The following variables have had their
defaults changed to ensure replication is as robust and
efficient as possible by default:

+ master_info_repository=TABLE and
relay_log_info_repository=TABLE ensure that binary
log information is stored in InnoDB tables, making
replication resilient to unexpected halts.

+ transaction_write_set_extraction=XXHASH64 enables
the optimal method for generating write set hashes.

+ slave_rows_search_algorithms='INDEX_SCAN,HASH_SCAN'
enables the use of a hash table to avoid repeated
table scans when no Primary Key or Primary Key
Equivalent (non-null unique secondary index) exists
for a table

+ expire_logs_days=30 causes unused binary logs that
are older than 30 days to be purged, ensuring
storage space is used efficiently.

* Replication: The Group Replication plugin now notifies
other components in the server that some relevant events
have happened. Upon view changes, recovery state updates,
network partitioning and primary election, the plugin
informs listeners registered in the service registry and
notifies them that an event has occurred. These
listeners, which could even be other plugins, can then
react to these events.

* Replication: The group_replication_member_weight variable
has been added which enables you to control the election
of new primaries in single-primary mode. In previous
versions primary election was based on the member's UUID,
with the lowest UUID elected as the new primary in the
event of fail over. Use this variable to assign numeric
weights to members to ensure that specific members are
elected, for example during scheduled maintenance of the
primary or to ensure certain hardware is prioritised.

* Replication: The following new system variables have been
added to configure Group Replication:

+ group_replication_flow_control_hold_percent

+ group_replication_flow_control_max_commit_quota

+ group_replication_flow_control_member_quota_percent

+ group_replication_flow_control_min_quota

+ group_replication_flow_control_min_recovery_quota

+ group_replication_flow_control_period

+ group_replication_flow_control_release_percent
These variables enable you to fine tune flow control of
individual group members, adjusting the quota depending
on the task the member performs in the group.

* JSON: Added support for ranges in the XPath expressions
used with many MySQL JSON functions, including
JSON_EXTRACT() and JSON_REMOVE(). Such a range is
specified using the syntax start to end, where start and
end are, respectively, the first and last indexes of a
range of elements from a JSON array (always numbered
starting with 0). For example, $[1 to 3] includes the
second, third, and fourth elements, as shown here:
mysql> SELECT JSON_EXTRACT('[1, 2, 3, 4, 5]', '$[1 to 3]');
+----------------------------------------------+
| JSON_EXTRACT('[1, 2, 3, 4, 5]', '$[1 to 3]') |
+----------------------------------------------+
| [2, 3, 4] |
+----------------------------------------------+
1 row in set (0.00 sec)

This work also provides support in such expressions for
the last keyword, which you can use to represent the
index of the last (rightmost) element in the current
array, like this:
mysql> SELECT JSON_EXTRACT('[1, 2, 3, 4, 5]', '$[last]');
+--------------------------------------------+
| JSON_EXTRACT('[1, 2, 3, 4, 5]', '$[last]') |
+--------------------------------------------+
| 5 |
+--------------------------------------------+
1 row in set (0.00 sec)

Indexes relative to the end of the array are also
supported, as shown here:
mysql> SELECT JSON_EXTRACT('[1, 2, 3, 4, 5]', '$[last-4 to last-2]');
+--------------------------------------------------------+
| JSON_EXTRACT('[1, 2, 3, 4, 5]', '$[last-4 to last-2]') |
+--------------------------------------------------------+
| [1, 2, 3] |
+--------------------------------------------------------+
1 row in set (0.00 sec)

For further information and examples, see Searching and
Modifying JSON Values
(http://dev.mysql.com/doc/refman/8.0/en/json.html#json-paths).
(Bug #79052, Bug #22285926)

* JSON: The optimizer now supports partial (in-place)
updates of JSON documents when using JSON_SET(),
JSON_REPLACE(), or JSON_REMOVE(). (Previously, all
updates of JSON column values were done by completely
removing the previous document and writing the new one in
its place.) In general, this optimization can be
performed when the replacement value is less than or
equal in size to the column's original value, and no new
elements are added to the original value. It cannot be
performed for a simple replacement of the column value of
the form UPDATE tbl SET json_col = json_val.
The JSON Data Type
(http://dev.mysql.com/doc/refman/8.0/en/json.html)
provides more information about the requirements for an
update to be optimized in this way.
This work includes the addition of two JSON utility
functions, JSON_STORAGE_SIZE() and JSON_STORAGE_FREE(),
in the MySQL Server. JSON_STORAGE_SIZE() returns the
number of bytes used to store the binary representation
of a JSON document, whether the document is presented as
a column value in a table, as the value of a user
variable, or as a JSON literal. In the case of a JSON
column, this is the space used to store the JSON document
as it was inserted into the column, prior to any partial
updates that may have been performed on it since then.
JSON_STORAGE_FREE() shows the number of bytes in the
binary representation of a JSON column value that were
freed by the most recent partial update of the column.
For a user variable storing a JSON document,
JSON_STORAGE_FREE() always returns 0; it also returns 0
if the argument is a JSON literal.
Each of these functions, like many other MySQL functions
that act on JSON values, also accepts a string that can
be successfully parsed as a JSON document. For more
information and examples, see JSON Utility Functions
(http://dev.mysql.com/doc/refman/8.0/en/json-utility-functions.html).

* RPM .spec files now include support for running unit
tests. (Bug #25814143, Bug #85743)

* For Windows, MSI installer packages now include a check
for the required Visual Studio redistributable package,
and produce a message asking the user to install it if it
is missing. (Bug #25658832)

* The mysql client now supports a --binary-as-hex option
that causes display of binary data using hexadecimal
notation (0xvalue). Thanks to Daniël van Eeden for the
patch. (Bug #25340722, Bug #84391)

* The SHOW TABLES statement now supports an optional
EXTENDED keyword that causes statement output to list
hidden tables created by failed ALTER TABLE statements.
(These temporary tables have names beginning with #sql.)
For more information, see SHOW TABLES Syntax
(http://dev.mysql.com/doc/refman/8.0/en/show-tables.html).
(Bug #24786075, Bug #83241)

* The mysql client by default strips comments in statements
sent to the server, and this behavior is controlled using
--skip-comments (strip comments), and --comments
(preserve comments).
Comment stripping is now deprecated. This feature and the
options to control it will be removed in a future MySQL
release.

* The TempTable storage engine replaces the MEMORY storage
engine as the default engine for in-memory internal
temporary tables. The TempTable storage engine provides
efficient storage for VARCHAR and VARBINARY columns. The
internal_tmp_mem_storage_engine session variable defines
the storage engine for in-memory internal temporary
tables. Permitted values are TempTable (the default) and
MEMORY. The temptable_max_ram configuration option
defines the maximum amount of memory that the TempTable
storage engine can use before data is stored to disk.
The memory/temptable/physical_ram and
memory/temptable/physical_disk Performance Schema
instruments may be used to monitor TempTable memory
allocation and disk storage.

* The MySQL 8.0.2 release introduces a number of undo
related changes:

+ The number of undo tablespaces may be modified at
runtime or when the server is restarted using the
existing innodb_undo_tablespaces configuration
option. Previously, the number of undo tablespaces
could only be configured when initializing the MySQL
instance and could not be changed afterward. This
change permits the addition of undo tablespaces and
rollback segments as a database installation grows.

+ innodb_undo_log_truncate is enabled by default. When
enabled, any undo tablespace that exceeds the
threshold value defined by innodb_max_undo_log_size
is marked for truncation. See Truncating Undo Logs
That Reside in Undo Tablespaces
(http://dev.mysql.com/doc/refman/8.0/en/truncate-undo-tablespace.html).

+ The innodb_undo_tablespaces default value was
changed from 0 to 2, which means that rollback
segments are created in two separate undo
tablespaces instead of the InnoDB system tablespace
by default. A minimum of two undo tablespaces is
required to permit truncation of undo logs.
Setting innodb_undo_tablespaces to 0 is deprecated
and will not be supported in a future release.

+ The naming convention used for undo tablespace files
is changed from undoNNN to undo_NNN, where NNN is
the undo space number.

+ The innodb_rollback_segments configuration option
defines the number of rollback segments per undo
tablespace. Previously, innodb_rollback_segments was
a global setting that specified the total number of
rollback segments for the MySQL instance. This
change increases the number of rollback segments
available for concurrent transactions, and more
rollback segments increases the likelihood that
concurrent transactions use separate rollback
segments for undo logs resulting in less resource
contention.

+ The innodb_undo_logs configuration option is
removed. The innodb_rollback_segments configuration
option performs the same function and should be used
instead.

+ The Innodb_available_undo_logs status variable is
removed. The number of available rollback segments
per tablespace may be retrieved using SHOW VARIABLES
LIKE 'innodb_rollback_segments';

* The SHOW COLUMNS and SHOW INDEX statements now support an
optional EXTENDED keyword that causes statement output to
include information about hidden columns and indexes that
MySQL uses internally and are not accessible by users.
For more information, see SHOW COLUMNS Syntax
(http://dev.mysql.com/doc/refman/8.0/en/show-columns.html),
and SHOW INDEX Syntax
(http://dev.mysql.com/doc/refman/8.0/en/show-index.html).

[ To be continued...]

--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: http://lists.mysql.com/mysql