Author: Sergey Petrunia
Author Date: 2019-08-22 21:57:40 UTC

MDEV-20109: Optimizer ignores distinct key created for materialized...

Sj_materialization_picker::check_qep(): fix error in cost/fanout
calculations:
- for each join prefix, add #prefix_rows / TIME_FOR_COMPARE to the cost,
  like best_extension_by_limited_search does
- Remove the fanout produced by the subquery tables.

optimize_wo_join_buffering() (used by LooseScan and FirstMatch)
- also add #prefix_rows / TIME_FOR_COMPARE to the cost of each prefix.

Author: Sergey Petrunia
Author Date: 2019-08-09 21:53:28 UTC

Optimizer trace: print cost and #rows of the join prefix

The names rows_for_plan and cost_for_plan follow MySQL
Also added post-join-operation selectivity cost

Author: Sachin Setiya
Author Date: 2018-05-17 09:10:55 UTC

MDEV-16239 Many test in rpl suite fails

Fix rpl_skip_error test.
  We cant reset Slave_skipped_errors(even with FLUSH STATUS), So instead
of absolute slave_skipped_errors we look for delta of slave_skipped_errors
Fix rpl.rpl_binlog_errors and binlog_encryption.rpl_binlog_errors
  We create the $load_file and $load_file2 but we never remove them.
Fix rpl_000011.test
  Instead of real value use delta value , Since flush status wont flush
LONGLONG variable.
Fix rpl_row_find_row_debug
  Instead of searching whole log_error_ file we will use search_pattern_in_file
which runs pattern search only on latest test run , instead of full file.
Fix rpl_ip_mix rpl_ip_mix2
  We should call reset slave all because we also want to reset master_host
otherwise show slave status wont be empty and making repeat N a failure.
Fix rpl_rotate_logs
  First we have to remove master.info file (cleanup) and second we have to
call reset slave all because if we do not call reset slave all then we wont
read master.info file beacuse we already have master config in memory.
And this makes start slave to pass , which shoud fail becuase its permision
is 000
Fix circular_serverid0 test
  The reason is that ++dbug_rows_event_count == 2 in queue_event does
not take --repeat into account. So I have reseted the dbug_rows_event_count
in if body.

Author: Nikita Malyavin
Author Date: 2019-08-08 14:56:44 UTC

MDEV-20297 sql_list: add begin() and end() for List<T>

Author: Sergey Petrunia
Author Date: 2019-08-03 20:15:44 UTC

Post-merge fixes for rocksdb.group_min_max test

- Fix the LooseScan code to support storage engines that return
  HA_ERR_END_OF_FILE if the index scan goes out of provided range
  bounds
- Add a DBUG_EXECUTE_IF("force_group_by",...) to allow a test to
  force a LooseScan
- Adjust rocksdb.group_min_max test not to use features not present
  in MariaDB 10.2 (e.g. optimizer_trace. In MariaDB 10.4 it's present
  but it doesn't meet the assumptions that the test makes about it
- Adjust the test result file:
  = MariaDB doesn't support "Enhanced Loose Scan" that FB/MySQL has
  = MariaDB has different cost calculations.

Author: Julius Goryavsky
Author Date: 2019-07-30 15:18:31 UTC

MDEV-20185: Windows: Use of uninitialized value $bpath in string eq

The execution of mtr in the Windows environment fails due to
the fact that the new code from MDEV-18565 does not take into
account the need to add the ".exe" extension to the names of
executable files when searching for pre-requisites that are
needed to run SST scripts (especially when using mariabackup)
and when searching paths to some other Galera utilities.
This patch fixes this flaw.

Also adding paths to the PATH environment variable is now
done with the correct delimiter character.

Author: Julius Goryavsky
Author Date: 2019-07-26 00:55:28 UTC

Fix for MW-336 test

Author: Julius Goryavsky
Author Date: 2019-07-16 09:33:11 UTC

MDEV-18565: Galera mtr-suite fails if galera library is not installed

Currently, running mtr with an incorrect (for example, new or
obsolete) version of wsrep_provider (for example, with the 26
version of libgalera_smm.so) leads to the failure of tests in
several suites with vague error diagnostics.

As for the galera_3nodes suite, the mtr also does not effectively
check all the prerequisites after merge with MDEV-18426 fixes.
For example, tests that using mariabackup do not check for presence
of ss and socat/nc. This is due to improper handling of relative
paths in mtr scripts.

In addition, some tests in different suites can be run without
setting the environment variables such as MTR_GALERA_TFMT, XBSTREAM,
and so on.

To eliminate all these issues, this patch makes the following changes:

1. Added auxiliary wsrep_mtr_check utility (which located in the
mysql-test/lib/My/SafeProcess subdirectory), which compares the
versions of the wsrep API that used by the server and by the wsrep
provider library, and it does this comparison safely, without
accessing the API if the versions do not match.

2. All checks related to the presence of mariabackup and utilities
that necessary for its operation transferred from the local directories
of different mtr suites (from the suite.pm files) to the main suite.pm
file. This not only reduces the amount of code and eliminates duplication
of identical code fragments, but also avoids problems due to the inability
of mtr to consider relative paths to include files when checking skip
combinations.

3. Setting the values of auxiliary environment variables that
are necessary for Galera, SST scripts and mariabackup (to work
properly) is moved to the main mysql-test-run.pl script, so as
not to duplicate this code in different suites, and to avoid
partial corrections of the same errors for different suites
(while other suites remain uncorrected).

4. Fixed duplication of the have_file_key_management.inc and
have_filekeymanagement.inc files between different suites,
these checks are also transferred to the top level.

5. Added garbd presence check and garbd path variable.

https://jira.mariadb.org/browse/MDEV-18565

Author: Julius Goryavsky
Author Date: 2019-07-16 10:48:04 UTC

MDEV-18565: Galera mtr-suite fails if galera library is not installed

Currently, running mtr with an incorrect (for example, new or
obsolete) version of wsrep_provider (for example, with the 26
version of libgalera_smm.so) leads to the failure of tests in
several suites with vague error diagnostics.

As for the galera_3nodes suite, the mtr also does not effectively
check all the prerequisites after merge with MDEV-18426 fixes.
For example, tests that using mariabackup do not check for presence
of ss and socat/nc. This is due to improper handling of relative
paths in mtr scripts.

In addition, some tests in different suites can be run without
setting the environment variables such as MTR_GALERA_TFMT, XBSTREAM,
and so on.

To eliminate all these issues, this patch makes the following changes:

1. Added auxiliary wsrep_mtr_check utility (which located in the
mysql-test/lib/My/SafeProcess subdirectory), which compares the
versions of the wsrep API that used by the server and by the wsrep
provider library, and it does this comparison safely, without
accessing the API if the versions do not match.

2. All checks related to the presence of mariabackup and utilities
that necessary for its operation transferred from the local directories
of different mtr suites (from the suite.pm files) to the main suite.pm
file. This not only reduces the amount of code and eliminates duplication
of identical code fragments, but also avoids problems due to the inability
of mtr to consider relative paths to include files when checking skip
combinations.

3. Setting the values of auxiliary environment variables that
are necessary for Galera, SST scripts and mariabackup (to work
properly) is moved to the main mysql-test-run.pl script, so as
not to duplicate this code in different suites, and to avoid
partial corrections of the same errors for different suites
(while other suites remain uncorrected).

4. Fixed duplication of the have_file_key_management.inc and
have_filekeymanagement.inc files between different suites,
these checks are also transferred to the top level.

5. Added garbd presence check and garbd path variable.

https://jira.mariadb.org/browse/MDEV-18565

Author: Jan Lindström
Author Date: 2019-07-15 04:02:18 UTC

Try #2

Author: Julius Goryavsky
Author Date: 2019-07-09 14:54:08 UTC

Improved error messages and added another path to the version check utility

Author: Marko Mäkelä
Author Date: 2019-07-03 14:31:20 UTC

MDEV-18582: Extend SHOW STATUS LIKE 'Innodb_%'

Many InnoDB internal variables and counters were only exposed
in an unstructured fashion via SHOW ENGINE INNODB STATUS.

Expose more variables via SHOW STATUS. Many of these were
exported in XtraDB.

Also, introduce SHOW_SIZE_T and use the proper size for
exporting the InnoDB variables.

Remove some unnecessary indirection via export_vars, and
bind some variables directly.

dict_sys_t::rough_size(): Replaces dict_sys_get_size()
and includes the hash table sizes.

This is based on a contribution by Tony Liu from ServiceNow.

Author: Marko Mäkelä
Author Date: 2019-06-26 18:57:42 UTC

MDEV-19845: Adaptive spin loops

Starting with the Intel Skylake microarchitecture, the PAUSE
instruction latency is about 140 clock cycles instead of earlier 10.
On AMD processors, the latency could be 10 or 50 clock cycles,
depending on microarchitecture.

Because of this big range of latency, let us scale the loops around
the PAUSE instruction based on timing results at server startup.

my_cpu_relax_multiplier: New variable: How many times to invoke PAUSE
in a loop. Only defined for IA-32 and AMD64.

my_cpu_init(): Determine with RDTSC the time to run 16 PAUSE instructions
in two unrolled loops according, and based on the quicker of the two
runs, initialize my_cpu_relax_multiplier. This form of calibration was
suggested by Mikhail Sinyavin from Intel.

LF_BACKOFF(), ut_delay(): Use my_cpu_relax_multiplier when available.

ut_delay(): Define inline in my_cpu.h.

UT_COMPILER_BARRIER(): Remove. This does not seem to have any effect,
because in our ut_delay() implementation, no computations are being
performed inside the loop. The purpose of UT_COMPILER_BARRIER() was to
prohibit the compiler from reordering computations. It was not
emitting any code.

Author: Sachin Setiya
Author Date: 2019-06-19 05:05:39 UTC

MDEV-19049 Server crashes in check_duplicate_long_entry_key, ASAN stack-buffer-overflow in Field_blob::get_key_image

Long Unique keys should always be last unique key.

Author: Rasmus Johansson
Author Date: 2019-05-18 06:11:01 UTC

MDEV-17592 Create MariaDB named commands/symlinks

Author: Georg Richter
Author Date: 2019-06-14 04:39:27 UTC

Fix for embedded test

Author: Anel Husakovic
Author Date: 2019-06-13 14:55:58 UTC

Do empty job for package/package body - refactor store_schema_params()

Author: Julius Goryavsky
Author Date: 2019-06-13 11:59:54 UTC

MDEV-18863: Galera SST scripts can't read [mysqldN] option groups

Some users and some scripts (for example, mysqld_multi.sh) use special
option groups with names like [mysqld1], [mysqld2], ..., [mysqldN].

But SST scripts can't currently fully support these option groups.
The only option group-related value it gets from the server is
--defaults-group-suffix from the server, if that option was set
for mysqld when the server was started.

However, the SST script does not get told by the server to read
these option groups, so this means that the SST script will fail to
read options like innodb-data-home-dir when it is in a option group
like [mysqld1]...[mysqldN].

Moreover, SST scripts ignore many parameters that can be passed
to them explicitly and cannot transfer them further, for example,
to the input of mariabackup utility. Ideally, we want to transfer
all the parameters of the original mysqld call to utilities such
as mariabackup, however the SST script does not receive these
parameters and therefore cannot transfer them to mariabackup.

To correct these shortcomings, we need to add a transfer to the
script of all the parameters of the original mysqld call, and in
the SST scripts themselves provide for the transfer of these
parameters to utilities such as mariabackup. To prevent these
parameters from mixing with the script's own parameters, they
should be transferred to SST script after the special option
"--mysqld-args", followed by the line of the original parameters,
as received by mysqld call at the time of launch (further all
these parameters will be passed to mariabackup, for example).

In addition, the SST scripts themselves must be refined so that
they can read the parameters from the user-selected group, not just
from the global mysqld configuration group. And also so that they
can receive the parameters (which important for their work) as
command-line arguments.

Author: Georg Richter
Author Date: 2019-06-11 10:44:16 UTC

MDEV-14101 Provide an option to select TLS protocol version

Server and command line tools now support option --tls_version to specify the
TLS version between client and server. Valid values are TLSv1.0, TLSv1.1, TLSv1.2, TLSv1.3
or a combination of them. E.g.

--tls_version=TLSv1.3
--tls_version=TLSv1.2,TLSv1.3

In case there is a gap between versions, the lowest version will be used:
--tls_version=TLSv1.1,TLSv1.3 -> Only TLSv1.1 will be available.

If the used TLS library doesn't support the specified TLS version, it will use
the default configuration.

Limitations:

SSLv3 is not supported. The default configuration doesn't support TLSv1.0 anymore.
TLSv1.3 protocol currently is only supported by OpenSSL 1.1.0 (client and server) and
GnuTLS 3.6.5 (client only).

Overview of TLS implementations and protocols

Server:

+-----------+-----------------------------------------+
| Library | Supported TLS versions |
+-----------+-----------------------------------------+
| WolfSSL | TLSv1.1, TLSv1,2 |
+-----------+-----------------------------------------+
| OpenSSL | (TLSv1.0), TLSv1.1, TLSv1,2, TLSv1.3 |
+-----------+-----------------------------------------+
| LibreSSL | (TLSv1.0), TLSv1.1, TLSv1,2, TLSv1.3 |
+-----------+-----------------------------------------+

Client (MariaDB Connector/C)
+-----------+-----------------------------------------+
| Library | Supported TLS versions |
+-----------+-----------------------------------------+
| GnuTLS | (TLSv1.0), TLSv1.1, TLSv1.2, TLSv1.3 |
+-----------+-----------------------------------------+
| Schannel | (TLSv1.0), TLSv1.1, TLSv1.2 |
+-----------+-----------------------------------------+
| OpenSSL | (TLSv1.0), TLSv1.1, TLSv1,2, TLSv1.3 |
+-----------+-----------------------------------------+
| LibreSSL | (TLSv1.0), TLSv1.1, TLSv1,2, TLSv1.3 |
+-----------+-----------------------------------------+

Author: Sergei Golubchik
Author Date: 2019-06-11 15:51:09 UTC

MDEV-16249 CHECKSUM TABLE for a spider table is not parallel and saves all data in memory in the spider head by default (#1328)

followup for be5c432a42e

ha_partition::calculate_checksum() has to invoke calculate_checksum()
for partitions unconditionally, not under (HA_HAS_OLD_CHECKSUM | HA_HAS_NEW_CHECKSUM).
Because the server uses ::info() to ask for a live checksum, while
calculate_checksum() must, precisely, calculate it the slow way,
also for tables that don't have the live checksum at all.

Also, fix the compilation on Windows (ha_checksum/ulonglong type mix).

Author: Georg Richter
Author Date: 2019-06-10 07:15:49 UTC

Fixed windows build

Author: Sergey Petrunia
Author Date: 2019-05-27 14:43:13 UTC

MDEV-19608: MyRocks should check if it is compiled with CRC32 support

- An attempt at checking that MyRocks ins compiled with fast CRC32

Author: Alexey Botchkov
Author Date: 2019-05-24 16:48:09 UTC

Custom build.

Author: Kentoku SHIBA
Author Date: 2019-05-02 19:10:35 UTC

MDEV-18994 Fix the bug encountered when the size of (v1,v2,v3...) value list in “select * from tb where id in (v1, v2, v3...)” is greater than 1000
Some engines need to avoid in optimization.

Author: Monty
Author Date: 2019-05-17 13:34:35 UTC

Make it trivial to get stack traces from external programs.

To get a stacktrace in a program like aria_chk, one only have to do:

#include <my_stacktrace.h>

call my_init_stacktrace(1) in main().

Author: Kentoku SHIBA
Author Date: 2019-05-19 17:05:22 UTC

MDEV-19002 Partition performance optimization
Add the partition options of the following function for batch call instead o
- store_lock
- external_lock
- start_stmt
- extra
- cond_push
- info_push
- top_table

Author: Sergey Petrunia
Author Date: 2019-05-10 09:47:44 UTC

MDEV-19134: EXISTS() slower if ORDER BY is defined

Step #2: "[ORDER BY ...] LIMIT n" should not prevent EXISTS-to-IN
conversion, as long as
- the LIMIT clause doesn't have OFFSET
- the LIMIT is not "LIMIT 0".

Author: Monty
Author Date: 2019-04-15 15:16:02 UTC

MDEV-17841 S3 storage engine

A read-only storage engine that stores it's data in (aws) S3

To store data in S3 one could use ALTER TABLE:
ALTER TABLE table_name ENGINE=S3

libmarias3 integration done by Sergei Golubchik
libmarias3 created by Andrew Hutchings

Author: Sachin Setiya
Author Date: 2019-05-13 06:29:11 UTC

empty commit 3

Author: Sergey Vojtovich
Author Date: 2019-05-10 19:19:05 UTC

MDEV-19439 - Revert orig_thd hack

This patch is to be squashed with previous one when MDEV-19439 is fixed.
Causes assertion failurs currently.

Author: Kentoku SHIBA
Author Date: 2019-05-01 05:46:23 UTC

MDEV-18990 Wrong result when binary column is used as a condition in hexadecimal mode

Author: Vlad Lesin
Author Date: 2019-04-25 20:36:42 UTC

One more compilation error fix for c++11 in runtime code

Author: Marko Mäkelä
Author Date: 2019-04-26 14:55:35 UTC

WIP: MDEV-17603 REPLACE and INSERT…ON DUPLICATE KEY UPDATE are deadlock-prone

Implement an alternative fix for the bug whose original fix
mysql/mysql-server@c93b0d9a972cb6f98fd445f2b69d924350f9128a
in MySQL 5.7.4 caused problems.

This is based on
mysql/mysql-server@e0e4bacddf421550baca3578bc0db13693874fdb
in MySQL 5.7.26.

When performing a rollback to the start of the current row
operation in REPLACE or INSERT...ON DUPLICATE KEY UPDATE
we were not maintaining serializability, because we would
release implicit locks that could already have been acquired
for some of the indexes.

lock_rec_convert_impl_to_expl_for_trx(): Declare globally.

undo_node_t::convert_impl_to_expl(): Convert an implicit lock
to an explicit one during a partial rollback.

row_insert_for_mysql(): Set trx->duplicates=ULINT_UNDEFINED
for rolling back the current row operation. This will allow
undo_node_t::convert_impl_to_expl() to be effective only
for this use case, not for other scenarios, such as
rolling back to the start of the statement, or
ROLLBACK TO SAVEPOINT.

FIXME: Neither innodb.auto_increment_dup,log-bin nor the
upstream fix (which we did not add) innodb.iodku pass.
While the undo_node_t::convert_impl_to_expl() is working
as intended, what happens in innodb.auto_increment_dup,log-bin
is that the newly created explicit record lock for the record
heap number 6 on the PRIMARY key root page (3) will be released
when that record is deleted moments later, with the following
stack trace:

lock_rec_reset_nth_bit
lock_rec_reset_and_release_wait_low
lock_rec_reset_and_release_wait
lock_update_delete
btr_cur_optimistic_delete_func
row_undo_ins_remove_clust_rec
row_undo_ins
row_undo
row_undo_step
que_thr_step
que_run_threads_low
que_run_threads
trx_rollback_to_savepoint_low
trx_rollback_to_savepoint
row_mysql_handle_errors
row_insert_for_mysql

The idea might work with predicate locks, which we do not have.
This entire scenario could also be fixed by MDEV-16232, which
could allow the entire operation to be protected with page latches.

Author: Jani
Author Date: 2019-04-24 15:09:36 UTC

Merge branch '10.3' of github.com:MariaDB/server into bb-10.3-jani

Author: Kentoku SHIBA
Author Date: 2019-04-11 09:03:22 UTC

MDEV-19003 ignore send <cache> to remote
It's just added regression tests.

Author: Alexey Botchkov
Author Date: 2019-04-08 11:33:28 UTC

MDEV-7974 SA transactions.

Failing tests added.

Author: Kentoku SHIBA
Author Date: 2019-04-04 21:23:57 UTC

MDEV-7795 Merge vertical partitioning storage engine

Author: Marko Mäkelä
Author Date: 2019-03-15 11:49:18 UTC

Merge branch '10.4-MDEV-18726-large-page-overallocate' of https://github.com/grooverdan/mariadb-server into HEAD

Author: Marko Mäkelä
Author Date: 2019-02-25 13:35:00 UTC

MDEV-18719 Assertion (c.prtype ^ o->prtype) & ... failed on ALTER TABLE

The prtype & DATA_LONG_TRUE_VARCHAR flag only plays a role when
converting between InnoDB internal format and the MariaDB SQL layer
row format. Ideally this flag would never have been persisted in the
InnoDB data dictionary.

There were bogus assertion failures when an instant ADD, DROP, or
column reordering was combined with a change of extending a VARCHAR
from less than 256 bytes to more than 255 bytes. Such changes are
allowed starting with MDEV-15563 in MariaDB 10.4.3.

dict_table_t::instant_column(), dict_col_t::same_format(): Ignore
the DATA_LONG_TRUE_VARCHAR flag, because it does not affect the
persistent storage format.

Author: Marko Mäkelä
Author Date: 2019-02-20 20:42:18 UTC

WIP MDEV-17805: Remove InnoDB cache for temporary tables

The intention is to link dict_table_t to TABLE_SHARE.

FIXME: Implement ha_innobase::delete_table() for temporary tables.
FIXME: Implement rollback of ha_innobase::create() for temporary tables.

FIXME: Currently, DBUG_ASSERT(s) fails in ha_innobase::open() because
we fail to pass the InnoDB_share from ha_innobase::create().
This seems to happen for any ALTER TABLE that uses ALGORITHM=COPY

(Old note from December, maybe not true any more:)
An assignment to TABLE_SHARE::ha_share would not work either, because
create() and open() will use different TABLE_SHARE.

Author: Sergey Petrunia
Author Date: 2019-02-18 10:29:01 UTC

MDEV-18608: Defaults for 10.4: histogram size should be set

Followup: update test results

Author: Vlad Lesin
Author Date: 2019-02-13 13:01:38 UTC

MDEV-18573: avoid backup inconsistency in 10.2

--no-lock and --lock-ddl-per-table options deprecared, the corresponing
code is removed. aria_log_control is copied first and aria_log.# files last
under FTWRL.

Author: Vicențiu Ciorbaru
Author Date: 2019-02-09 23:43:15 UTC

Default Bernoulli Sampling implementation

Author: Vicențiu Ciorbaru
Author Date: 2019-02-09 23:43:15 UTC

Default Bernoulli Sampling implementation

Author: Daniel Bartholomew
Author Date: 2019-02-04 21:49:23 UTC

bump the VERSION

Author: Alexey Botchkov
Author Date: 2019-02-03 17:23:35 UTC

MODIFY idx_read_map

Author: Daniel Bartholomew
Author Date: 2019-02-01 15:47:27 UTC

bump the VERSION

Author: Sergei Golubchik
Author Date: 2019-01-02 11:03:15 UTC

fix the test for 2019

Author: Oleksandr "Sanja" Byelkin
Author Date: 2019-01-24 09:43:27 UTC

MDEV-17401: LOAD DATA from very big file into MyISAM table results in EOF error and corrupt index

fix of incorrect 10.3 merge

Author: Alexey Botchkov
Author Date: 2019-01-23 19:26:06 UTC

Support issue 25102.

Author: Sergey Petrunia
Author Date: 2019-01-11 20:57:11 UTC

Apply this patch from Percona Server:

commit cd7201514fee78aaf7d3eb2b28d2573c76f53b84
Author: Laurynas Biveinis <laurynas.biveinis@gmail.com>
Date: Tue Nov 14 06:34:19 2017 +0200

    Fix bug 1704195 / 87065 / TDB-83 (Stop ANALYZE TABLE from flushing table definition cache)

    Make ANALYZE TABLE stop flushing affected tables from the table
    definition cache, which has the effect of not blocking any subsequent
    new queries involving the table if there's a parallel long-running
    query:

    - new table flag HA_ONLINE_ANALYZE, return it for InnoDB and TokuDB
      tables;
    - in mysql_admin_table, if we are performing ANALYZE TABLE, and the
      table flag is set, do not remove the table from the table
      definition cache, do not invalidate query cache;
    - in partitioning handler, refresh the query optimizer statistics
      after ANALYZE if the underlying handler supports HA_ONLINE_ANALYZE;
    - new testcases main.percona_nonflushing_analyze_debug,
      parts.percona_nonflushing_abalyze_debug and a supporting debug sync
      point.

    For TokuDB, this change exposes bug TDB-83 (Index cardinality stats
    updated for handler::info(HA_STATUS_CONST), not often enough for
    tokudb_cardinality_scale_percent). TokuDB may return different
    rec_per_key values depending on dynamic variable
    tokudb_cardinality_scale_percent value. The server does not have a way
    of knowing that changing this variable invalidates the previous
    rec_per_key values in any opened table shares, and so does not call
    info(HA_STATUS_CONST) again. Fix by updating rec_per_key for both
    HA_STATUS_CONST and HA_STATUS_VARIABLE. This also forces a re-record
    of tokudb.bugs.db756_card_part_hash_1_pick, with the new output
    seeming to be more correct.

Author: Marko Mäkelä
Author Date: 2018-12-05 18:14:24 UTC

Try to ensure that rollback of inserting metadata record works

We must use the original page format for inserting metadata records.
This is because in case the instant ALTER TABLE operation needs to
be rolled back (due to crash recovery), all pages must remain in
the original format.

btr_cur_optimistic_insert(): Avoid reorganize for inserting the
metadata record.

btr_cur_optimistic_update(): Initialize rec,offsets correctly
after possible page format conversion.

innobase_instant_try(): Do not store NULL values for those
metadata record columns that were originally declared NOT NULL.

dict_index_t::get_n_nullable(): Refer to dict_col_t::was_not_null()
in order to keep the original format intact.

lock_move_reorganize_page(): Allow the two pages to be in
different formats.

rec_convert_dtuple_to_rec_comp(), rec_convert_dtuple_to_rec_comp():
Also observe the DATA_WAS_NOT_NULL flag.

Author: Marko Mäkelä
Author Date: 2018-12-07 14:32:32 UTC

Merge 10.4 into HEAD

Author: Georg Richter
Author Date: 2018-12-05 16:36:34 UTC

Windows fixes

Author: Georg Richter
Author Date: 2018-12-03 12:55:20 UTC

MDEV-14101: Provide option to specify tls_version for client tools

Author: Kristian Nielsen
Author Date: 2018-10-14 18:41:49 UTC

Move deletion of old GTID rows to slave background thread

This patch changes how old rows in mysql.gtid_slave_pos* tables are deleted.
Instead of doing it as part of every replicated transaction in
record_gtid(), it is done periodically (every @@gtid_cleanup_batch_size
transaction) in the slave background thread.

This removes the deletion step from the replication process in SQL or worker
threads, which could speed up replication with many small transactions. It
also decreases contention on the global mutex LOCK_slave_state. And it
simplifies the logic, eg. when a replicated transaction fails after having
deleted old rows.

With this patch, the deletion of old GTID rows happens asynchroneously and
slightly non-deterministic. Thus the number of old rows in
mysql.gtid_slave_pos can temporarily exceed @@gtid_cleanup_batch_size. But
all old rows will be deleted eventually after sufficiently many new GTIDs
have been replicated.

Author: Oleksandr "Sanja" Byelkin
Author Date: 2018-11-20 09:58:34 UTC

MDEV-15073: Generic UDAF parser code in server for windows functions

Added support for usual agreggate UDF (UDAF)
Added remove() call support for more efficient window function processing
Added example of aggregate UDF with efficient windows function support

Author: Sergey Petrunia
Author Date: 2018-11-01 08:35:28 UTC

MDEV-17518: Range optimization doesn't use ON expressions from nested outer joins

Part#2: take into account that join nest that we are marking as constant
might already have constant tables in it. Don't count these tables twice.

Author: Marko Mäkelä
Author Date: 2018-10-19 16:00:25 UTC

Merge 10.4 into 10.4-mdev-15562

This branch exists just for the record, for preserving the
development history, should it ever be needed.

Author: Larysa Sherepa
Author Date: 2018-10-18 16:46:00 UTC

CTE tests

Author: Vicențiu Ciorbaru
Author Date: 2018-10-14 17:54:52 UTC

Merge branch 'MDEV1732_check_constraint_10.2_v1' of https://github.com/an3l/server into an3l-MDEV1732_check_constraint_10.2_v1

Author: Igor Babaev
Author Date: 2018-10-10 11:40:30 UTC

MDEV-17382 Hash join algorithm should not be used to join materialized
           derived table / view by equality

Now rows of a materialized derived table are always put into a
temporary table before join operation. If BNLH is used to join this
table with the result of a partial join then both operands of the
join are actually put into main memory. In most cases this is not
efficient.
We could avoid this by sending the rows of the derived table directly
to the join operation. However this kind of data flow is not supported
yet.
Fixed by not allowing usage of hash join algorithm to join a materialized
derived table if it's joined by an equality predicate of the form
f=e where f is a field of the derived table.

Adjusted the results of the test case for 10.2-compatibility.

Author: Vicențiu Ciorbaru
Author Date: 2018-09-22 11:01:46 UTC

Small change for feedback debug

Author: Sachin Setiya
Author Date: 2018-09-17 07:16:24 UTC

multiple_user_binlog

Author: Monty
Author Date: 2018-09-05 17:20:16 UTC

Added DBUG_ASSERT_AS_PRINTF to BUILD/..asan-max

Author: Otto Kekäläinen
Author Date: 2018-09-02 19:36:14 UTC

WIP: test 6

Author: Sergey Petrunia
Author Date: 2018-09-01 12:10:50 UTC

Trigger another BB run

Author: Oleksandr "Sanja" Byelkin
Author Date: 2018-08-16 11:55:36 UTC

MDEV-16930: Crash when VALUES in derived table contains expressions

Give names to the value constructor columns as in SELECT-list.

Author: Vicențiu Ciorbaru
Author Date: 2018-07-30 15:34:12 UTC

Fix tmp table patch

Author: Andrei Elkin
Author Date: 2018-07-28 09:15:05 UTC

MDEV-14014 Unittest extension to cover concurrent IO_CACHE read and write by the dump and user threads.

Author: Sergey Petrunia
Author Date: 2018-07-13 13:23:04 UTC

MDEV-16710, Issue#857: MyRocks: Incorrect behavior when miltiple statements fail inside a transaction

rollback_to_stmt_savepoint() calls do_rollback_to_savepoint(). This removes
the changes made by this statement, and also removes the set savepoint.

Before we start processing the next statement, we need to set the new
savepoint, so we will have something to rollback to if the next statement
fails.

Since rollback_to_stmt_savepoint always sets a new savepoint now,
m_n_savepoints is now redundant and is removed.

Author: Jacob Mathew
Author Date: 2018-07-09 21:25:37 UTC

MDEV-16246: insert timestamp into spider table from mysqldump gets wrong time zone.

The problem occurred because the Spider node was incorrectly handling
timestamp values sent to and received from the data nodes.

The problem has been corrected as follows:
- Added logic to set and maintain the UTC time zone on the data nodes.
  To prevent timestamp ambiguity, it is necessary for the data nodes to use
  a time zone such as UTC which does not have daylight savings time.
- Removed the spider_sync_time_zone configuration variable, which did not
  solve the problem and which interfered with the solution.
- Added logic to convert to the UTC time zone all timestamp values sent to
  and received from the data nodes. This is done for both unique and
  non-unique timestamp columns. It is done for WHERE clauses, applying to
  SELECT, UPDATE and DELETE statements, and for UPDATE columns.
- Disabled Spider's use of direct update when any of the columns to update is
  a timestamp column. This is necessary to prevent false duplicate key value
  errors.
- Added a new test spider.timestamp to thoroughly test Spider's handling of
  timestamp values.

Author:
  Jacob Mathew.

Reviewer:
  Kentoku Shiba.

Author: Anel Husaković
Author Date: 2018-05-15 11:17:43 UTC

MDEV-12914: Disable silently call of myisam

Author: Anel Husaković
Author Date: 2018-04-03 12:41:13 UTC

MDEV-14474 information_schema.check_constraints

Implement according to standard SQL specification 2008.
The check_constraints table is used for fetching metadata about
the constraints defined for tables in all databases.

There were some result files which failed after running mtr.
These files are updated with newly create record with mtr --record.

Author: Vladislav Vaintroub
Author Date: 2018-06-29 21:18:47 UTC

MDEV-16596 : Windows - redo log does not work on native 4K sector disks.

Disks with native 4K sectors need 4K alignment and size for unbuffered IO
(i.e files opened with FILE_FLAG_NO_BUFFERING)

Innodb opens redo log with FILE_FLAG_NO_BUFFERING, but it does 512byte IOs
Thus, the IO fails, and Innodb is non-functional.

The fix is to check whether OS_FILE_LOG_BLOCK_SIZE is multiple of logical
sector size, and if it is not, reopen the redo log without
FILE_FLAG_NO_BUFFERING flag.

Author: Marko Mäkelä
Author Date: 2018-02-23 20:26:26 UTC

MDEV-15058: Allocate buf_pool statically

buf_pool_t::create(): Replaces buf_pool_init().

buf_pool_t::close(): Replaces buf_pool_free().

FIXME: Do not add buf_pool_t::is_initialised()

Author: Sachin Setiya
Author Date: 2018-05-14 06:45:38 UTC

MDEV-14472 Assertion `is_current_stmt_binlog_format_row()' failed...
in THD::binlog_write_table_map

Problem:- So the issue is when the test case (mdev_14472) is run with debug
  build , Assert DBUG_ASSERT(is_current_stmt_binlog_format_row()); is fired
  in binlog_write_table_map.

Analysis:- When we lock table t1, and if t1 has some trigger which updates
  another table then all other tables are also locked. So in the case of lock
  t1(in mdev_14472 test case) we will lock t1, t2, t3 and t4. In next insert
  command (t2 insert), we call handler->check_table_row_based from
  write_locked_table_maps which updates the table handler
  check_table_binlog_row_based_done to 1 and check_table_binlog_row_based_result
  to 1 , it is set to one because this particular insert stmt is unsafe(because
  of trigger on t4). The next insert stmt(insert t3) is safe so we will write
  that in stmt format , but since we havent cleared the previous
  check_table_binlog_row_based_result so it is logged in row format while
  thd->is_current_stmt_binlog_format_row() is still 0, and this fires the
  assert. So the question is why check_table_binlog_row_based_result is not
  reseted because mark_used_tables_as_free_for_reuse skips locked tables

Solution:- We make mark_used_tables_as_free_for_reuse to reset
  check_table_binlog_row_based_result even when table is locked

Author: Sergey Vojtovich
Author Date: 2018-06-05 08:54:06 UTC

MDEV-16371 - Fix for memory write order inversion

my_atomic implementation for interruptable waits.

Author: Oleksandr "Sanja" Byelkin
Author Date: 2018-05-17 17:17:28 UTC

MDEV-12465: Server crashes in my_scan_weight_utf8_bin upon collecting stats for RocksDB table

Do not use "only index read" in analyzing indices if there is a field which present in the index only partially.

Author: THIRUNARAYANAN BALATHANDAYUTHAPANI
Author Date: 2018-05-15 05:46:51 UTC

MDEV-13626 Merge InnoDB test cases from MySQL 5.7

Imported the following test case from mysql to MariaDB

1) innodb.alter_kill
2) innodb.alter_foreign_crash
3) innodb.alter_rename_files
4) innodb.analyze_table
5) Appended the case in innodb-online-alter-gis

Author: Sachin Setiya
Author Date: 2018-05-14 06:45:38 UTC

MDEV-14472 Assertion `is_current_stmt_binlog_format_row()' failed...
in THD::binlog_write_table_map

try 1 patch

Author: Sergey Petrunia
Author Date: 2018-05-11 11:50:36 UTC

Tentatively enable a few myrocks tests

Author: Oleksandr "Sanja" Byelkin
Author Date: 2018-04-26 17:58:43 UTC

MDEV-13024: Server crashes in my_store_ptr upon DELETE from sequence in multi-table format

Make operation possition() dummy for sequences (should not be used really)

Author: Vicențiu Ciorbaru
Author Date: 2018-04-02 21:07:58 UTC

Force build?

Author: Oleksandr "Sanja" Byelkin
Author Date: 2018-03-27 06:42:31 UTC

Walking throuch query tables ON expression added.

Author: Michael Widenius
Author Date: 2018-03-15 10:31:59 UTC

Added flush tables to acl_load_mutex-5170.test

This is needed as otherwise a lot of MyISAM tables may be marked
as crashed, depending on in which order tests are run

Author: Sergey Petrunia
Author Date: 2018-03-08 10:43:55 UTC

MDEV-15245: Assertion `false' failed in myrocks::ha_rocksdb::position

Don't call handler->position() if the last call to read a row did
not succeed.

Author: Alexander Barkov
Author Date: 2017-08-18 19:36:42 UTC

MDEV-11952 Oracle-style packages: stage#5

- CREATE PACKAGE [BODY] statements are now
  entirely written to mysql.proc with type='PACKAGE' and type='PACKAGE BODY'.
- CREATE PACKAGE BODY now supports IF NOT EXISTS
- DROP PACKAGE BODY now supports IF EXISTS
- CREATE OR REPLACE PACKAGE [BODY] is now supported
- CREATE PACKAGE [BODY] now support the DEFINER clause:

    CREATE DEFINER user@host PACKAGE pkg ... END;
    CREATE DEFINER user@host PACKAGE BODY pkg ... END;

- CREATE PACKAGE [BODY] now supports SQL SECURITY and COMMENT clauses, e.g.:

    CREATE PACKAGE p1 SQL SECURITY INVOKER COMMENT "comment" AS ... END;

- Package routines are now created from the package CREATE PACKAGE BODY
  statement and don't produce individual records in mysql.proc.

- CREATE PACKAGE BODY now supports package-wide variables.
  Package variables can be read and set inside package routines.
  Package variables are stored in a separate sp_rcontext,
  which is cached in THD on the first packate routine call.

- CREATE PACKAGE BODY now supports the initialization section.

- All public routines (i.e. declared in CREATE PACKAGE)
  must have implementations in CREATE PACKAGE BODY

- Only public package routines are available outside of the package

- {CREATE|DROP} PACKAGE [BODY] now respects CREATE ROUTINE and ALTER ROUTINE
  privileges

- "GRANT EXECUTE ON PACKAGE BODY pkg" is now supported

- SHOW CREATE PACKAGE [BODY] is now supported

- SHOW PACKAGE [BODY] STATUS is now supported

- CREATE and DROP for PACKAGE [BODY] now works for non-current databases

- mysqldump now supports packages

- "SHOW {PROCEDURE|FUNCTION) CODE pkg.routine" now works for package routines

- "SHOW PACKAGE BODY CODE pkg" now works (the package initialization section)

- A new package body level MDL was added

- Recursive calls for package procedures are now possible

- Routine forward declarations in CREATE PACKATE BODY are now supported.

- Package body variables now work as SP OUT parameters

- Package body variables now work as SELECT INTO targets

- Package body variables now support ROW, %ROWTYPE, %TYPE

Author: Marko Mäkelä
Author Date: 2018-02-24 20:04:11 UTC

Merge 10.3 into bb-10.3-MDEV-15058

Author: Jacob Mathew
Author Date: 2018-02-20 03:19:03 UTC

MDEV-14500: Support engines without rnd_pos() and
            engines with inefficient rnd_pos()

Some engines have not implemented rnd_pos(). There are other engines whose
implementation of rnd_pos() is inherently inefficient. Spider is such an
engine, whose implementation of rnd_pos() needs to access a table on a remote
data node to retrieve a single table row.

To address these limitations, a new temporary table has been added to filesort.
When filesort sequentially reads the table being sorted, each row is written to
the filesort temp table in addition to being copied to the sort buffer.
Subsequent calls to rnd_pos() will then access the table row in the filesort
temp table instead of in the table being sorted.

The following logic changes incorporate the new filesort temp table into the
server:
- A new handler method to determine whether a call to the engine's
  rnd_pos() is expensive. The default return value is FALSE. Engines without
  rnd_pos() or with an inefficient rnd_pos() should return TRUE.
- Create the filesort temp table only if:
  - There are no add-on columns for filesort; and
  - The engine's implementation of rnd_pos() is expensive.
- Write to the temp table each row that is read from the table being sorted.
- Do subsequent row retrievals that use rnd_pos() on the temp table instead of
  on the table being sorted. Upon retrieving a row from the temp table, copy
  its column values to the record of the table being sorted.
- Upon completion of retrieval of the sorted result rows, delete the filesort
  temp table and free the memory allocated for using it.

The logic changes are in the following areas:
- Table handler.
- Partition engine.
- Spider engine.
- Filesort.
- Read record manager.

Note that these changes only address the use of rnd_pos() by filesort. They do
not address the use of rnd_pos() in other areas such as:
- Quick select.
- Insert.
- Update.
- Window functions.
- Multi Range Read.

Author:
  Jacob Mathew.

Reviewer:
  Sergei Golubchik.

Author: Monty
Author Date: 2018-02-19 09:23:20 UTC

Move alter partition flags to alter_info->partition_flags

This is done to get more free flag bits for alter_info->flags

Renamed all ALTER PARTITION defines to start with ALTER_PARTITION_
Renamed ALTER_PARTITION to ALTER_PARTITION_INFO
Renamed ALTER_TABLE_REORG to ALTER_PARTITION_TABLE_REORG

Other things:
- Shifted some ALTER_xxx defines to get empty bits at end

Author: Monty
Author Date: 2018-02-10 12:42:59 UTC

Added Max_index_length and Temporary to SHOW TABLE STATUS

- Max_index_length is supported by MyISAM and Aria tables.
- Temporary is a placeholder to signal that a table is a
  temporary table. For the moment this is always "N", except
  "Y" for generated information_schema tables and NULL for
  views. Full temporary table support will be done in another task.
  (No reason to have to update a lot of result files twice in a row)

Author: Sergey Petrunia
Author Date: 2018-01-13 22:22:54 UTC

Merge remote-tracking branch 'origin/10.2' into bb-10.3-mariarocks

Author: Vicențiu Ciorbaru
Author Date: 2018-01-12 01:57:02 UTC

To avoid travis network timeouts, use travis's retry script

Author: Vicențiu Ciorbaru
Author Date: 2018-01-09 07:05:48 UTC

Workaround for dtoa.c

There is no obvious reason why multiplying by 1.0 helps solve a
precision problem on gcc-7 or clang-5, but it does work. Casting to
ulong however makes the problem come up again. Windows warns about loss
of precision, so to quiet warnings until an appropriate fix is
implemented, ifdef 2 versions of the code.

Author: Vicențiu Ciorbaru
Author Date: 2017-12-30 15:18:24 UTC

Tentative fix for travis

This undoes a lock-unlock for LOCK_thread_count.

Author: Monty
Author Date: 2017-12-02 19:50:08 UTC

Remove compiler warnings

Author: Andrei Elkin
Author Date: 2017-11-27 19:06:17 UTC

MDEV-13073. This patch is a followup of the previous one to convert the trailing underscore identifier to mariadb standard. For identifier representing class private members the underscore is replaced with a `m_` prefix. Otherwise `_` is just removed.

Author: Andrei Elkin
Author Date: 2017-11-16 19:14:59 UTC

MDEV-13073. Fixed failing on BB sys_vars.rpl_semi_sync_master_enabled_basic. The issue forced to add up aan argument to THD::THD and THD::init to propagate skip of a mutex locking. The lock could not be released instead as this leads to clear deadlock; the test is received a piece for concurrent ack thread start/stop.

Author: Jun Su
Author Date: 2017-10-25 09:41:23 UTC

Support CRC32 SSE2 implementation under Windows