Add the missing STL headers to allow compiling on VS2022

Change-Id: I2e87cc97fa4365d22ddd18f1d5373c389fda4c86

…error to mtr output.

MTR tests that uses run_ndbapitest.inc or run_java.inc were intended to
record the last 512 or 200 output lines of test output in result file on
error to eventually show up in mtr output.

But mtr only showed the last 20 lines of difference between test result
file and the pre-recorded test results.

An empty file include/full_result_diff.inc is introduced to be included
in test files that do not want the 20 lines limitation of result diff
output.

The file is included by run_ndbapitest.inc or run_java.inc.

This will make it easier to see for example why ndb.test_mgmd fails in
PB2 without need to download tar file with all test failures and for
PB2 builds such as gcov which does not provide such tar file.

Change-Id: I457d0f9430bfe0960ebd60fcbe5eb25e3b71a986

Change-Id: Idefbe610f8bd2f435ed4cc0bac32898cfe9ff2fc

…array_YY

This patch fixes three issues

     - For hashed set operations we compute the number of chunk files
       as a multiple of two. If the statistics for the number of rows
       in the left set operand is wrong (in this case it was -1), we
       would get a calculated number of chunk files of zero. When the
       bug report was made, this led to us not having any chunk files,
       since the ceiling method used (to get multiple of two) returned
       0 for 0 input.  This ceiling function has since been changed
       (commit 88d716a - Bug#35813111: Use C++ standard library for
       more bit operations [my_round_up_to_next_power, noclose]), so
       we now get 1 on zero input, so we end up with just one chunk
       file.  This makes the crash seen with the old optimizer go
       away; we would quickly run out of space again and revert to
       de-duplicating via tmp table index.  This is fixed by sanity
       checking the number of rows estimate and making a wild guess at
       the number of rows in the result set, currently set to 8 * rows
       already in hash table when we overflow memory.

     - The estimate of -1 for the windowed left operand is obviously
       wrong, and we changed the code to propagate the number of rows
       of the child table. The added tests shows new behavior for this
       case.

     - even if the estimate is not -1, but a reasonable number, fixing
       the above saw another crash (using the repro for the hypergraph
       given, but with old optimizer enabled): we ran out of space in
       the dedicated hash table upon re-reading one of the on-disk
       chunks into the hash table. If the estimate is too low, we may
       end up in a situation where one of the chunks just barely fits
       into the dedicated mem_root when processing the left
       operand. Later, when processing operands 2..n we have to
       re-read the chunks into the hash table in preparation for
       matching with right operand rows. We asserted that this should
       always succeed, since these rows already fit for the left
       operand. However, the order in which the rows are entered into
       the hash table for operand two will in general differ from that
       of operand one. This will in general lead to different
       fragmentation of the dedicated mem_root - notably if we have
       blobs of different sizes as in the repro - and the chunk that
       previously *just* fit, will may not fit this time around. In
       the example we saw the last row in the chunk file not fit (row
       percona#290 out of 290 rows in the chunk file).  We fix this by
       falling back on the thread's main mem_root if this
       happens. This should be very rare.  Note that in most cases if
       the estimate is low too low, we would run out of space when
       processing the left operand, and fall back on index based
       de-duplication using tmp table.

Change-Id: Ie2b299ff6f106df727866b3cd3631b7a273c1c9b

…failed.

This issue involves setting a user variable inside the argument of a
window function, which in turn is evaluated using the window frame
buffer (Window::m_needs_frame_buffering), row optimizable
(Window::m_row_optimizable).

Setting of user variables inside expressions is deprecated, in part
since the semantics are hard to define, but this patch avoids the
assert at least.

Normally when evaluating a function argument for a window function,
e.g.  AVG( 2 * c1 ), the function (here multiplication) will be
evaluated just before writing a row to the frame buffer, whereas
functions *containing* a window function will be evaluated later,
i.e. after the value of the window function is ready,
cf. split_sum_func2.

Now, for the first case when creating the window's output and frame
buffer tmp tables, we replace the function (here the multiplication)
with a result field in the window's tmp table,
cf. change_to_use_tmp_fields, so that the window function when
evaluated will pick up its input (the result of the multiplication)
from the window frame buffer. However, there is an explicit exception
for setting of user variables (which is implemented as a function),
cf. this comment in change_to_use_tmp_fields:

  /*
    Replace "@:=<expression>" with "@:=<tmp table column>". Otherwise, we
    would re-evaluate <expression>, and if expression were a subquery,
    this would access already-unlocked tables.
  */

which stems from Bug#11764371 at commit 6a2402b, which involves
locking issues when executing a subquery when evaluating a setting of
a user variable, a join and group by. The exception mentioned is a
problem for evaluating window functions, because the window function
(AVG) will try to evaluate Item_func_set_user_var when reading from
the frame buffer. But the underlying column is taken from the input
file, not from the output/frame buffer (it is not replaced by an
Item_ref subject to the slice indirection).

In the repro, the two first rows are null. The third row is non-null.
When trying to evaluate the wf for the third row, we invert the first
(null) row, but this time, since the third input row has been read,
the Item_func_set_user_var's argument points into the input buffer
which has a non-null row, where as AVG expects to invert a null row
(it has counters indicating that rows 1 and 2 are both null) from the
frame buffer. This causes the assert error seen.

The fix is to *not* apply the mentioned exception for setting user
variables when windowing: that way the user variable is only set when
we read the input result set for the windowing action, and the frame
buffer will contain that value. As far as the locking issue of
11764371, I tried windowing on the query mentioned for the issue
(included in test), and saw no issue.

Change-Id: I75dc538bea55d5888783586d31ced5e9643d6fa3

…tions behind

During a data node restart, the starting node asks nodegroup peers
to copy the details of the current event subscribers to the starting
node, so that it can take over responsibility for part of the event
forwarding and buffering work when it is started.

This implies that the starting node must also monitor for subscribing
API node failures from this point forward.

As the copy of subscribers happens before the API nodes are directly
connected to the starting node, it is necessary for the starting data
node to handle API failures even when those APIs are not directly
connected during restarts.

This was already handled to some extent in QMGR, but appears to have
regressed over time.

This could result in e.g. :
 - Leaked subscribers at SUMA
 - Wasted effort forwarding events to disconnected subscribers
 - Wasted effort forwarding events to reconnected subscribers
 - Crashes / undefined behaviour if reconnected subscribers reuse
   NdbApi object identitied.

test_event_mysqld is created to give some coverage of the MySQLD
ha_ndbcluster plugin stack event consumption behaviour during data
node restarts and MySQLD asynchronous disconnects.

  test_event_mysqld -n MySQLDEventsRestarts
  test_event_mysqld -n MySQLDEventsDisconnects
  test_event_mysqld -n MySQLDEventsRestartsDisconnects

The third testase is invoked from a new MTR testcase to give simple
automated coverage.

  ndb_binlog_testevent_rd

Change-Id: Icef0fd0972fb5646bd27dbf2b9ff194938a30ba9

…FAILREP handling

Background

NdbApi events are buffered in NdbApi to match the rate of production and
rate of consumption by user code.
A maximum buffer size can be set to avoid unbounded memory usage when
the rate is mismatched for a long time.
When the maximum size is reached event buffering stops until the buffer
usage drops below a lower threshold.
A TE_OUT_OF_MEMORY event is added to the buffer to inform the consumer
that there can be missing events.

Problem

Testing and experience shows some issues with this mechanism, particularly as
it stops the buffering of both 'table data' events (TE_INSERT, TE_UPDATE,
TE_DELETE) and other meta events (TE_DROP_TABLE, TE_NODE_FAILURE,
TE_CLUSTER_FAILURE, ...).  Discarding the meta events results in issues
in NdbApi and in consumer code as the state of the event stream is not
clear.

One specific example is where the cluster is disconnected while the event
buffer is full - in this case, the local ClusterMgr thread should add
TE_NODE_FAILURE and TE_CLUSTER_FAILURE events to the buffered events for
each active EventOperation, but this was not possible and resulted in
a crash.  Other undesirable behaviours may be possible.

Solution

In an event buffer overload situation, only the table data events
TE_INSERT, TE_UPDATE, TE_DELETE are discarded.  Meta events are buffered
and eventually processed as normal.

Effects

This means that the buffer usage is not necessarily as tightly bounded
as in the original implementation, though the rate of production of meta
events is expected to be minimal.

Additionally it means that from the consumer's point of view, when they
iterate to a position in the event stream that corresponds to an out
of memory situation, they may still observe some meta events 'during'
the out of memory period.  These events will not include data, and
specifically will not include TE_EMPTY_EPOCH events which could be
misleading.

The consumer can therefore safely assume that the out-of-memory situation
persists until it receives TE_EMPTY_EPOCH (if enabled) or data carrying
events afer a TE_OUT_OF_MEMORY event.

An additional effect of this change is that the event API's latest epoch
as returned by getLatestGCI() no longer is frozen while the event buffer
producer is in an epoch gap - it continues to climb.

This behaviour was not documented before as an indication of event buffer
OOM handling, but was used by some existing test_event testcase.  These
are modified to use observations of the buffer fill level instead to
detect when an OOM gap is being buffered.

Testing

Four new tests added to test_event_mysqld :

  test_event_mysqld -n MySQLDEventsEventBufferOverload
  test_event_mysqld -n MySQLDEventsEventBufferOverloadRestarts
  test_event_mysqld -n MySQLDEventsEventBufferOverloadDisconnects
  test_event_mysqld -n MySQLDEventsEventBufferOverloadRestartsDisconnects

New MTR test added invoking

  test_event_mysqld -n MySQLDEventsEventBufferOverloadRestartsDisconnects

  ndb_binlog_testevent_ord

Change-Id: If904fb60be2798b53cc4cb25ea3607c4289739b9

…or drop table

Problem

During NdbApi event buffer overflow, buffering of data and metadata
events is paused, resulting in data and metadata events being
discarded.

This means that event subscribers waiting for metadata events
such as TE_DROP_TABLE may never receive them and will timeout.

Specifically, MySQL Cluster schema distribution uses the arrival
of a TE_DROP_TABLE event on an NdbEventOperation to know when it
is required and safe to drop the NdbEventOperation.

If this event does not arrive within a bounded time then the
NdbEventOperation is not dropped, leaking resources and potentially
resulting in later problems.

Solution

The fix to
  Bug#35663761 NdbApi fails to find container for latest epoch in NODE_FAILREP handling

modified the NdbApi event buffer overload handling to only pause
the buffering of data events, but continue buffering metadata
events.
This means that in event buffer overload situations,
TE_DROP events will not be lost.

This patch adds a testcase covering this scenario :

  test_event_mysqld -n MySQLDEventsEventBufferOverloadDDL

This test is invoked from MTR as :
  ndb_binlog_testevent_os

Change-Id: I654fd0aa0c83d35956d4d01632f16f610951bd0c

…tions behind

During a data node restart, the starting node asks nodegroup peers
to copy the details of the current event subscribers to the starting
node, so that it can take over responsibility for part of the event
forwarding and buffering work when it is started.

This implies that the starting node must also monitor for subscribing
API node failures from this point forward.

As the copy of subscribers happens before the API nodes are directly
connected to the starting node, it is necessary for the starting data
node to handle API failures even when those APIs are not directly
connected during restarts.

This was already handled to some extent in QMGR, but appears to have
regressed over time.

This could result in e.g. :
 - Leaked subscribers at SUMA
 - Wasted effort forwarding events to disconnected subscribers
 - Wasted effort forwarding events to reconnected subscribers
 - Crashes / undefined behaviour if reconnected subscribers reuse
   NdbApi object identitied.

test_event_mysqld is created to give some coverage of the MySQLD
ha_ndbcluster plugin stack event consumption behaviour during data
node restarts and MySQLD asynchronous disconnects.

  test_event_mysqld -n MySQLDEventsRestarts
  test_event_mysqld -n MySQLDEventsDisconnects
  test_event_mysqld -n MySQLDEventsRestartsDisconnects

The third testase is invoked from a new MTR testcase to give simple
automated coverage.

  ndb_binlog_testevent_rd

Change-Id: Icef0fd0972fb5646bd27dbf2b9ff194938a30ba9

…FAILREP handling

Background

NdbApi events are buffered in NdbApi to match the rate of production and
rate of consumption by user code.
A maximum buffer size can be set to avoid unbounded memory usage when
the rate is mismatched for a long time.
When the maximum size is reached event buffering stops until the buffer
usage drops below a lower threshold.
A TE_OUT_OF_MEMORY event is added to the buffer to inform the consumer
that there can be missing events.

Problem

Testing and experience shows some issues with this mechanism, particularly as
it stops the buffering of both 'table data' events (TE_INSERT, TE_UPDATE,
TE_DELETE) and other meta events (TE_DROP_TABLE, TE_NODE_FAILURE,
TE_CLUSTER_FAILURE, ...).  Discarding the meta events results in issues
in NdbApi and in consumer code as the state of the event stream is not
clear.

One specific example is where the cluster is disconnected while the event
buffer is full - in this case, the local ClusterMgr thread should add
TE_NODE_FAILURE and TE_CLUSTER_FAILURE events to the buffered events for
each active EventOperation, but this was not possible and resulted in
a crash.  Other undesirable behaviours may be possible.

Solution

In an event buffer overload situation, only the table data events
TE_INSERT, TE_UPDATE, TE_DELETE are discarded.  Meta events are buffered
and eventually processed as normal.

Effects

This means that the buffer usage is not necessarily as tightly bounded
as in the original implementation, though the rate of production of meta
events is expected to be minimal.

Additionally it means that from the consumer's point of view, when they
iterate to a position in the event stream that corresponds to an out
of memory situation, they may still observe some meta events 'during'
the out of memory period.  These events will not include data, and
specifically will not include TE_EMPTY_EPOCH events which could be
misleading.

The consumer can therefore safely assume that the out-of-memory situation
persists until it receives TE_EMPTY_EPOCH (if enabled) or data carrying
events afer a TE_OUT_OF_MEMORY event.

An additional effect of this change is that the event API's latest epoch
as returned by getLatestGCI() no longer is frozen while the event buffer
producer is in an epoch gap - it continues to climb.

This behaviour was not documented before as an indication of event buffer
OOM handling, but was used by some existing test_event testcase.  These
are modified to use observations of the buffer fill level instead to
detect when an OOM gap is being buffered.

Testing

Four new tests added to test_event_mysqld :

  test_event_mysqld -n MySQLDEventsEventBufferOverload
  test_event_mysqld -n MySQLDEventsEventBufferOverloadRestarts
  test_event_mysqld -n MySQLDEventsEventBufferOverloadDisconnects
  test_event_mysqld -n MySQLDEventsEventBufferOverloadRestartsDisconnects

New MTR test added invoking

  test_event_mysqld -n MySQLDEventsEventBufferOverloadRestartsDisconnects

  ndb_binlog_testevent_ord

Change-Id: If904fb60be2798b53cc4cb25ea3607c4289739b9

…or drop table

Problem

During NdbApi event buffer overflow, buffering of data and metadata
events is paused, resulting in data and metadata events being
discarded.

This means that event subscribers waiting for metadata events
such as TE_DROP_TABLE may never receive them and will timeout.

Specifically, MySQL Cluster schema distribution uses the arrival
of a TE_DROP_TABLE event on an NdbEventOperation to know when it
is required and safe to drop the NdbEventOperation.

If this event does not arrive within a bounded time then the
NdbEventOperation is not dropped, leaking resources and potentially
resulting in later problems.

Solution

The fix to
  Bug#35663761 NdbApi fails to find container for latest epoch in NODE_FAILREP handling

modified the NdbApi event buffer overload handling to only pause
the buffering of data events, but continue buffering metadata
events.
This means that in event buffer overload situations,
TE_DROP events will not be lost.

This patch adds a testcase covering this scenario :

  test_event_mysqld -n MySQLDEventsEventBufferOverloadDDL

This test is invoked from MTR as :
  ndb_binlog_testevent_os

Change-Id: I654fd0aa0c83d35956d4d01632f16f610951bd0c

Change-Id: I5488e06be61992c7d817d6febfdbb0b8d84d2e2c

Description:
------------
Each SELECT on a view causes a small memory leak on the
Windows platform.

Analysis:
---------
In case of select view Security_context(sctx) is allocated
in the MEM_ROOT. Memory allocated by the members of sctx is
not freed until the MEM_ROOT goes out of scope as a result
memory keeps on accumulating. Security_context employed a
mechanism logout() to avoid such memory accumulation but for
it to work correctly, we need to own the life cycle of the
sctx members keeping in mind that it is a frequently
accessed code path so that we avoid frequent memory
allocations/deallocations.

DB_restrictions is owned by the Security_context(sctx). The
former has an associative container std::unordered_map
(m_restrictions) member that is allocated on the stack. We
need to clear the memory allocated to this member at the
time of logout(). We had a similar memory growth reported
in Bug#31919448. At that time we used the temporary object
and swap() trick to free the content of m_restrictions.
Unfortunately, that trick worked on Linux but not on
Windows. This is because constructors of associative
containers can throw which means compilers are free to
allocate memory on the heap in the constructor of
associative containers. As it turned out to be the case on
Windows, std::unordered_map allocates the memory in the
heap. It is not cleared unless the object is destroyed. That
means the swap() trick would not work in such cases because
memory allocated by the temporary object is transferred to
the m_restrictions hence memory growth is still seen.

We have the following options to fix this situation :

- Refactor the code in the select view area that may have
  a wider repercussions, hence it is not the preferred
  solution.

- Create the m_restrictions map on the heap but not on the
  stack. This is the preferred option.

Fix:
----
- Change m_restrictions object creation from stack to heap.
- Allocate the memory to m_restriction only if that is
  required.
- Handle the unsafe usage of the partial_revokes APIs.
- Got rid of the swap() trick in DB_restrictions::clear().
- Removed is_not_empty() as we already have is_empty().

Change-Id: Iee828909d23991db86227f9664eb7f130a3d6d56

Change-Id: I84cfb3b47dfdf4002d1ee9c3c690f8788f84bad2

Change-Id: I2c0899e95d9cb25d4942557351e209244931f77a

Problem is in Item_field::replace_equal_field where we try to
replace fields with equivalent ones in order to push some of
these predicates down, we do not take into consideration
nullability, so some non-nullable fields were replaced with nullable
ones.
The fix was to add a function called allow_replacement which determines
if a non-nullable field can be replaced with a nullable one
(UNKNOWN results can be treated as false), otherwise skip such changes.

Change-Id: Id4370babd6d6e29e36a05ed46e4ca951507670d6

The assertion occurs because there is missing error checking in
udf_handler::call_init_func(), after evaluating the arguments to
the UDF function.

Fixed by adding error checks.

Change-Id: Icbbd256bdd4c6b21f4ec2c8b8b8ce856ebe68a47

…r to execute k nearest neighbors (kNN) queries. Works with innodb distance scan and hypergraph optimizer.

Change-Id: I3c25f586b5cd6b1ae5fd74ebb0897071f23336d4

This patch added some abstaction layers between physical libraries
(bundled or system) and their names on disk vs name in cmake code:
    Bug#35057542 Create INTERFACE libraries for bundled/system zlib/zstd/lz4

When generating the files mysqlclient.pc (for pkg-config) and
mysql_config (a shell script), we need to go through these abstraction
layers, and produce library names as required by the linker of the
host platform. This is done by the cmake macro EXTRACT_LINK_LIBRARIES.

The fix is to store these explicitly as "-lz" and "-lzstd"
respectively.  The lz4 library is not used by our client library, so
there is no need to store the linker option for that.

Change-Id: Ief70f66900bec2d07fdb3daa88caf60a138c953b

The fix for bug#35865438 'ndb_mgm hangs without --ndb-tls-search-path'
misused the ndb_mgm_get_clusterlog_severity_filter() API.

Change-Id: I0ef255c83b2ff3efaf0ebc1ad502e61e0e483575

This worklog will remove the system variable `slave-rows-search-algorithms`

Change-Id: Ic8264bce904d1102c90eb83cbb48ea78cdeef264

Removed the option
Removed the tests which uses the option.

Change-Id: I5316b2c49eb4cf1dc583d97891e7b43a5c472eca

… missing DDL statements

WL#15497 lets group replication determine whether any session
is running a DDL statement that would make it inadvisable to
change the primary server at this point.

This changeset adds support for additional DDL statements
as well as some DCL statements:

13.1.19 CREATE SPATIAL REFERENCE SYSTEM Statement
13.1.31 DROP SPATIAL REFERENCE SYSTEM Statement

13.1.12 CREATE DATABASE Statement
13.1.2  ALTER DATABASE Statement
13.1.24 DROP DATABASE Statement

13.1.18 CREATE SERVER Statement
13.1.8  ALTER SERVER Statement
13.1.30 DROP SERVER Statement

13.1.5  ALTER INSTANCE Statement
13.1.36 RENAME TABLE Statement

13.1.23 CREATE VIEW Statement
13.1.11 ALTER VIEW Statement
13.1.35 DROP VIEW Statement

13.1.22 CREATE TRIGGER Statement
13.1.34 DROP TRIGGER Statement

13.1.21 CREATE TABLESPACE Statement
13.1.10 ALTER TABLESPACE Statement
13.1.33 DROP TABLESPACE Statement

13.1.17 CREATE PROCEDURE and CREATE FUNCTION Statements
13.1.7  ALTER PROCEDURE Statement
13.1.4  ALTER FUNCTION Statement
13.1.29 DROP PROCEDURE and DROP FUNCTION Statements

13.1.14 CREATE FUNCTION Statement       (SONAME)
13.1.26 DROP FUNCTION Statement         (SONAME)

13.1.13 CREATE EVENT Statement
13.1.3  ALTER EVENT Statement
13.1.25 DROP EVENT Statement

13.1.15 CREATE INDEX Statement          (WL#15497)
13.1.27 DROP INDEX Statement            (WL#15497)

13.1.20 CREATE TABLE Statement          (WL#15497)
13.1.9  ALTER TABLE Statement           (WL#15497)
13.1.37 TRUNCATE TABLE Statement        (WL#15497)
13.1.32 DROP TABLE Statement            (WL#15497)

13.7.1.3 CREATE USER Statement
13.7.1.1 ALTER USER Statement
13.7.1.5 DROP USER Statement

13.7.1.2 CREATE ROLE Statement
13.7.1.9 SET DEFAULT ROLE Statement
13.7.1.11 SET ROLE Statement
13.7.1.4 DROP ROLE Statement

13.7.1.6 GRANT Statement
13.7.1.8 REVOKE Statement

13.7.1.7 RENAME USER Statement
13.7.1.10 SET PASSWORD Statement

Change-Id: Ia472d9ce1b4ac9bfe00d3ec619baa5a32f719857

Change-Id: I3d7820d23a7f4f80136eab131fd9e59e7c9175d9

…er WL#13448

After

  WL#13448 Remove COM_XXX command which are deprecated.

the test:

  ...classic_protocol_reconnect_all_commands

fails as the error-codes changed.

Change
======

- updated the expected error-codes for COM_PROCESSKILL,
  COM_LIST_FIELDS and COM_REFRESH

Change-Id: Icb09aa6253e850353b54a2ba4b90c7c23c9f3965

In the past the pattern:

  std::shared_ptr<void> exit_guard(nullptr,
    [this](void *) { ... });

has been used as adhoc Scope_guard.

Change
======

- use Scope_guard explicitly

Change-Id: I7593730da0f90a028231b580c443c3d0062b2cf6

In the past:

  std::shared_ptr<void> exit_trigger(nullptr,
    [&](void *) { ... });

has been used as adhoc scope guard.

In this case, the guard is used around a std::recursive_mutex
for which a std::lock_guard exists.

Change
======

- use a std::lock_guard instead of the adhoc scope-guard

Change-Id: Iae88f4d60db70894620cd3d108056588182c15d0

Additional patch, removing reference to Ubuntu 16 and Ubuntu 18.

Change-Id: I91ca91e91b4609d542d034f60fc1ec9e62ce4a10

…ttently

This is a follow up patch to fix the intermittent failure of this
recently added test.

There were following problems:

1. Test was restarting the server with an unknown option '--nocore` .
Apparently, this option is known to the mysqltest_safe_process that
is forked by the mtr to monitor the mysqld process. In situations
when a test wants to skip producing the core file, mtr framework
expects it to pass --skip-core-file to set this option. We also know
that server produces core file only if --core-file option is specified.
mtr scripts pass specify this option if the server is started through
the mtr scripts. But in this case since I start the server without
the mts scripts hence --core-file remains false thus there was no
need to pass any option to the server.

2. Test tried to evict page#0 belonging to tablespace t1 from
double-write buffer by inserting the 512 records to another tablespace
t2. This was not enough, there could still be situations where the
page#0 of the tablespace t1 could remain in the double-write buffer.

3. On windows different error code is returned.

4. The script corrupt_page.inc doesn't wait for server to close/die
   completely. This leads to sporadic error messages. Ideally this
   should be the responsibility of the caller of the script but it is
   safe to wait for the server to close inside the corrupt_page.inc

Fixes:
======
1. Removed the unknown option '--nocore`. By default server does not
produce the core files anyways.

2. To simulate the situation reliably that corrupt page is not found
in the double-write buffer, taken the backup of the double-write buffer
files before creating the table to be corrupted and restore them at
the time of starting the server with corrupt tablespace.
This makes it reliable test. Also reorganized the test now it corrupts
the first page from tablespace t2.ibd as opposed to earlier from t1.ibd

3. Added the error code returned on Windows.

4. Test now uses the var/tmp dir for temporary files: my_restart.err
and t1.ibd. I think this is better choice than earlier.

5. corrupt_page.inc calls wait_until_disconnected.inc. Also improved
the error messages.

Change-Id: I4184bcac8d9aee030360dc23940eeb752b78acc0

  Removed the option
  Removed the tests which uses the option.

Change-Id: Ife2ee01abc7a5f13efac4c2faf41bce86a5e846c

  Removed the option
  Removed the tests which uses the option.

Change-Id: I626c1f7d6d45dd13bd8695f34c3ee5932be1e911

Refactoring patch removing invariant where there
could be a MultiTransporter created, but not set up
as theNodeIdTransporter[] yet.

1) When a data node is restarted, the MultiTransporter is
created already when reading the config. However it is not
inserted into the theNodeIdTransporter[] until after
QMGR has 'switched' to use the the MultiTransporter.

2) When we are reconnecting to a data node where a MultiTransporter
were previously used, it already existed in theNodeIdTransporter[],
even if the reconnect process had not yet reached the 'switch'
point.

At the same code in QMGR, TRPMAN and TransporterRegistry is handling
both the intial connect and reconnect, it is complicated by
that a connecting MultiTransporter may or may not exist in
theNodeIdTransporter[] when setting up such a connection.

This patch insert the MultiTransporter into theNodeIdTransporter[]
as soon as it is created. Note that if the QMGR handshaking with
neighbour node will then later conclude that multiple transporters can
not be used, we will simply continue to communicate over the
single base transporter which is always connected first.

This allows us to simplify get_node_multi_transporter()
and remove a couple of other get'ters for multi_transporter.

Change-Id: I152b3a2821e0e1eac087f6b864fa97962f98dacf

Refactoring patch introducing the get_node_base_transporter(NodeId)
method. Replacing some common code patterns with calling
this method instead

Change-Id: Ibba5bbd8f75687ee1e13d41ebc3c75486fc63d52

Refactoring patch required to allow the transporter
CONNECTING/DISCONNECTING protocol to be used for the
individual PartsOfMultiTransporter's.

The usage of a 'NodeId' to identify a Transporter in the interface
is replaced by using its 'TrpId' instead. NodeBitmask is also
replaced by TrpBitmask.

Naming of methods now taking modified arguments are mostly
kept unchanged. The execpetion is where variants taking
both a NodeId and TrpId are needed, where the later then is
named with a *_trp suffix.

Format of EventLogger messages as well as different
DEBUG output has been addapted such that ut should be
clear whether a NodeId or a TrpId is reported in the logs.
(Somethimes both as well)

Some comments starting with 'OJA' are added by the patch,
pointing out where MultiTransporters does not follow the
correct protocol to disconnect or connect. These comments
will be removed again by the following patches which are
the real fixes.

Change-Id: I02ab4daa46e4b19d162c234bf4377cf3e918372a

TransporterRegistry::connect_server() handled connect of a
multi transporter instance as a special case of a Transporter connect().

With refactorings in previous patches a multi Transporter instance
now has its own states++ available, and in most aspects are just a
plain Transporter, which enable some cleanup and simplifications.

In particular:

  - For a non-multi Transporter we used to check that performStates[]
    was 'CONNECTING'.
  - As a multi Transported instance used to not have its own
    performStates[], we instead checked !isConnected().

This patch introduce the utility method get_node_transporter_instance
which is now used to locate the Transporter instance to connect.

The performStates[TrpId] is used to check for CONNECTING state,
independent of whether a multi Transporter is used or not.

-> All special handling of a multi Transporter is removed.
... A multi Transporter instance is just a Transporter.

Change-Id: I4ae73b89846206e7afa499ff340930a230cf2745

'reconnect' and 'disconnect' seems overused in naming
methods and the nameing might not always help to clearify
the intended usage of each method. In particular some
of these methods do an immediate connect and waits for
the connection with the other peer to be
established/teared down, while others starts the asynch
CONNECTING/DISCONNECING protocol.

Some of the root cause for this bug might be that the
naming of the methods doing the different steps in connect/
disconnect of transporters were confusing. Many of them were
named 'do connect', with different variants of upper case and '_'
in their naming, and some simply 'connect'. In these cases
'do' might be misleading and indicate an immediate
connect/disconnect taking place.

That made it unclear wheteher the particular method did an
immediate connect or just initated the CONNECTING/DISCONNECTING
transporter protocol which would eventually result in the transporter
being reported as CONNECTED/DISCONNECTED, but relied on the
start_clients_threads to do the real connect/disconnect, and the
final CONNECTED/DISCONNECTED state to be set by report_connect/
report_disconnect.

This patch rename such 'do connect/disconnect' methods to
'start connecting/disconnecting', in particular:

  TransporterRegistry::do_connect        -> start_connecting
  TransporterRegistry::do_connect_trp    -> start_connecting_trp

  TransporterRegistry::do_disconnect     -> start_disconnecting
  TransporterRegistry::do_disconnect_trp -> start_disconnecting_trp

  Transporter::do_disconnect             -> start_disconnecting

  TransporterFacade::doConnect           -> startConnecting
  TransporterFacade::doDisconnect        -> startDisonnecting

Change-Id: I02577788ff66cfaee50f30019ad18329ec1f246e

…rter'

We used to connect the PartsOfMultiTransporter when
start_clients_thread discovered that the the base
transporter had CONNECTED by calling
Transporter::connect_client() directly.

It basically did some of the same steps as if the
transporters were in a CONNECTING state. However,
as the CONNECTING-protocol was not used, all the
different state transitions, enable_send_buffers,
epoll_add and such had to be patched up as an
afterthought by QMGR.

Generally the protocol to set up connections were not
followed either, were there are strict rules of where the
performState[] transitions can be done, E.g. the CONNECTED
state should only be set by the receive thread responsible
for the specific TrpId.

Patch is to let Qmgr::connect_multi_transporter() use
start_connecting_trp(TrpId) to initate the CONNECTING
protocol. It will the connect as any other
transporters.

Also note that the 'hack' to set_s_port() on the
transporter now is obsolete - will be done by
start_clients_thread when seeing the CONNECTING
transporter.

Removed epoll_add functions and other code now obsolete.

Change-Id: I8937d96a2a1b7440a84b0dc3a0c8d2b857b7a2b7

…sporter'

We used to disconnect the MultiTransporters when report_disconnect
of *one* of the MultiTransporters was observed by calling
Transporter::forceUnsafeDisconnect(). Even if the disconnect
being 'reported' followed the disconnect protocoll, the other
disconnects of the remaing transporters *did not*. Besides
the protocol breakage, this could also cause  real time execution
problems as closing of sockets could stall, waiting for the
other peer or timeouts.

Patch fixes this by starting an asynchronous disconnect
instead with start_disconnecting_trp().

Transporter::forceUnsafeDisconnect is now obsolete
and is removed.

Transporter::start_disconnecting can be simplified
as there should be no difference between a disconnecting
multiTransporter and a plain transporter any more.

Change-Id: I3101603a66b7f712d2e51f2c9c0ae963d67edddb

When QMGR has concluded that all the PartsOfMultiTransporter has
CONNECTED, it will disconnect the base transporter which is
not needed anymore.

This used the Transporter::shutdown() method which forcefully just
closed theSocket, not following the transporter protocol of first
setting the performState[] to DISCONNECTING, and letting the asynch
disconnect happening out of the way of the executing block / QMGR.

Patch replace usage of ::shutdown() with start_disconnecting_trp().
::shutdown becomes obsolete and is removed.

Change-Id: Iee22f2c48df0e650fd29e5e3d12d6f0d06b5059b

Change-Id: Ibc0ce929215a4e21907396ec9d08ab6080a90c94

Remove one of the few dependencies of Multi_Transporter
having to inherit from Transporter

Change-Id: Ic106614281e47ea32099131a0c79b8663ea294b7

…d and overridden by Multi_Transporter.

Step#2 in breaking out Multi_Transporter dependencies on subclassing Transporter

Change-Id: I0804840f3fff21d52b9a02f0457ac9861a8712b4

…dden from Transporter, but never used.

Insert a 'require(false)' in the implementation of these methods in order
to prove that they are never used -> Ran Pb2 tests

Change-Id: I2ce2ee479d365c4d3c43a59a2e0e5532d7870390

…ods overridden by Multi_Transporter.

Change implementation of TransporterRegistry::get_bytes_sent and ::get_bytes_received() to
collect the sent and received over all the node transporters.

Change-Id: I2cebb5db6a3e98f5b72dea6643415857b15aafad

… Multi_Transporter.

Note: With all patches upto, including this, all Transporter methods
previously overridden by Multi_Transporter has now
been eliminated, or implemented as methods with a 'require(false)' to
prove that they are never used.

Change-Id: Ia1f774300168b968942e3705986c92fd6676282b

…ock_send_transporter()

Use these to eliminate 'lock' / 'unlock' on the Multi_Transporter

Change-Id: I124339222839aee600c31d8da78eefeb665acc72

…being a Transporter any more

As the Multi-Transporter is not a sub class of Transporter any more,
it can not be stored in theNodeIdTransporters[] any more.

-> theNodeIdTransporters[] is enhanced to always store the initial
base transporter to a specific NodeId. This also enables use to
simplify the implementation of ::get_node_base_transporter() as we
not need to handle that that theNodeIdTransporters[] may be a
Multi_Transporter any more.

In case a Multi_Transporter is created for a NodeId, it is now
stored in theNodeIdMultiTransporters[] instead. This array replace
theMultiTransporters[] with a slightly changes semantic - indexed
by NodeId instead, instead of 0..nMultiTransporters.

::get_node_multi_transporter() is also enhanced to look up
the Multi_Transporter directly from theNodeIdMultiTransporters[].

Change-Id: I8d9ad9dde262380da8dc5555a310eeffbdf0c487

…y more.

Class Multi_Transporter will not inherit from class Transporter.
Remove unused overridden methods inherited from class Transporter.

Change-Id: I87feb2673f02d1eeaac8331058cddc1a24d2ca77

Remove one of the few dependencies of Multi_Transporter
having to inherit from Transporter

Change-Id: Ic106614281e47ea32099131a0c79b8663ea294b7
(cherry picked from commit 5faec76d06469f66bf17cc1adb2a225717919059)

…d and overridden by Multi_Transporter.

Step#2 in breaking out Multi_Transporter dependencies on subclassing Transporter

Change-Id: I0804840f3fff21d52b9a02f0457ac9861a8712b4
(cherry picked from commit d6b3cd5ff5ce3cc49e07c6a4e3993bc99baa6dd3)

…dden from Transporter, but never used.

Insert a 'require(false)' in the implementation of these methods in order
to prove that they are never used -> Ran Pb2 tests

Change-Id: I2ce2ee479d365c4d3c43a59a2e0e5532d7870390
(cherry picked from commit 974ec94ef4a67ff7257b92935fe3af6bbf26b76c)

…ods overridden by Multi_Transporter.

Change implementation of TransporterRegistry::get_bytes_sent and ::get_bytes_received() to
collect the sent and received over all the node transporters.

Change-Id: I2cebb5db6a3e98f5b72dea6643415857b15aafad
(cherry picked from commit 22716326ce545e11494f897c20f96d23b4747450)

… Multi_Transporter.

Note: With all patches upto, including this, all Transporter methods
previously overridden by Multi_Transporter has now
been eliminated, or implemented as methods with a 'require(false)' to
prove that they are never used.

Change-Id: Ia1f774300168b968942e3705986c92fd6676282b
(cherry picked from commit ee86144475a85481a315d6593e536b4d5dc0fb61)

…ock_send_transporter()

Use these to eliminate 'lock' / 'unlock' on the Multi_Transporter

Change-Id: I124339222839aee600c31d8da78eefeb665acc72
(cherry picked from commit de0565ffa027e8e0b1790daacdb7c426349ea77a)

…being a Transporter any more

As the Multi-Transporter is not a sub class of Transporter any more,
it can not be stored in theNodeIdTransporters[] any more.

-> theNodeIdTransporters[] is enhanced to always store the initial
base transporter to a specific NodeId. This also enables use to
simplify the implementation of ::get_node_base_transporter() as we
not need to handle that that theNodeIdTransporters[] may be a
Multi_Transporter any more.

In case a Multi_Transporter is created for a NodeId, it is now
stored in theNodeIdMultiTransporters[] instead. This array replace
theMultiTransporters[] with a slightly changes semantic - indexed
by NodeId instead, instead of 0..nMultiTransporters.

::get_node_multi_transporter() is also enhanced to look up
the Multi_Transporter directly from theNodeIdMultiTransporters[].

Change-Id: I8d9ad9dde262380da8dc5555a310eeffbdf0c487
(cherry picked from commit cb6f19b151cb5f2731e43874d1804d7a3856656b)

…y more.

Class Multi_Transporter will not inherit from class Transporter.
Remove unused overridden methods inherited from class Transporter.

Change-Id: I87feb2673f02d1eeaac8331058cddc1a24d2ca77
(cherry picked from commit a93039734304105d3d23de49ffea5e280a412a0b)

Change-Id: Ie49c590f21f9479e1c8c1fffa305f54e36242683

Test executes a number of updates against a table it created.
Some of the updates are executed as 'NoCommit', with a final 'Commit'
for the entire transaction.

ERROR_INSERT is used to crash the node on the commit (as apposed
to the intermediate execute-NoCommit's)

Intention of test case is to check for regression against Bug#34216,
where: 'During TC-take-over (during NF) commit messages can come
out of order to TUP' - Thus presumable hitting the ERROR_INSERT
when execute-commit. Such failures are now randomly detected
by the failing test case.

Root cause is that there is other background activity going on on
the data nodes as well. In particular the update operations may
trigger ndb_index_stat updates performing READ operations on
the table being updated, as well as accessing the system tables.

Patch enhance the ERROR_INSERT 5048/5049 code to also require that
it is a ZUPDATE being commited, as well as that the table is
an UserTable.

Change-Id: I4061fa4521b8f670b3783a3aa0b6256bca15a7d0

Change-Id: I2f471fdb6936244c923bc7506afc4acf745b29ba

When setting up a new set of neighbour transporters, the old set
of neighbours may still have pending data awaiting to be sent.

In order to ensure that available data on these transporters
will still be sent, we need to ensure that the becoming non-neighbour
transporters are inserted into the list of transporter we need
to send on.

Patch enhance startChangeNeighbour() such that it will check
the old set of neighbour transporters if any has 'm_data_available'
and use insert_trp() to insert them into the list of non-neighbour
transporters.

Note that we now also need to clear the m_neighbour_trp flag before
doing such inserts, else insert_trp() would not have inserted the
TrpId into the non-neighbour list.

Patch also redeclares some local variables refering a 'transporter id'
from an Uint32 to a TrpId.

A few asserts are also added to ensure the consistency of the
Transporter list structures.

Change-Id: I87bb539868ef33eb19e37ed90b30f1a3e1e07680

Change-Id: Ie7255da6abb25906e71b2b19af2d066da59f3259

…t an equijoin

This commit fixes a regression introduced by the fix for bug
bug#34764211 "Too high row estimate for semijoin" (Change-Id:
I231cd0c8ef504d64cd835184a39f9975066b61bf). Some semijoins may be
transformed into an inner join between the right hand side aggregated
on the join fields, and the original left hand side. That fix
utilized this transform to make better row estimates, by estimating the
number of output rows as:

CARD(left_hand_relation) * inner_join_selectivity * CARD(d)

where:

* inner_join_selectivity is the cardinality of an inner join on the same
  predicate, divided by the cardinality of a cross join.

* 'd' is the set of distinct rows from right_hand_relation, when only
  looking at those columns that appear in the join predicate.

The regression happens for a semijoin that is not a pure
equijoin. That is, the join predicate is something else than a
conjunction of 'left_table.field=right_table.field' terms. In this
case JoinPredicate.semijoin_group is empty, because the semijoin to inner
join transform will never happen. In this case the
number of aggregated rows was wrongly set to 1. This fix corrects that.

This fix adds a new function EstimateSemijoinFanOut() that collects the
fields from the right hand_relation that appear in the join predicate.
Unlike JoinPredicate.semijoin_group, this works for both semijoin and
antijoin, and for arbitrary predicates, not just conjunctions of
field=field. That function then estimates CARD(d), using the same apparatus
that we use for e.g. DISTINCT.

Change-Id: I02224ff4f64315f2b0b92d6b883fe08e2f6a9975

PROBLEM
-------

1. When innodb_validate_tablespace_paths=off, innodb does not
   validate the tablespace during startup.
2. Since the tablespace is not validated ,we fail to initialize
   in memory filsystem hash map which maps space id with the
   tablespace.
3. If ibuf entries are present during startup, a background
   thread tries to merge these into the appropriate tablespace.
4. The background thread will search for tablespace in the hash
   map ,since it cannot find the tablespace it assumes that
   tablespace is deleted and drop the ibuf entries silently.
5. This leads to a corruption in the tablespace because number
   of primary and secondary entries differ.

FIX
---
1. If ibuf entries are present during startup , irrespective
   of the innodb_validate_tablespace_paths setting ,validate
   all the tablespaces.

Change-Id: I41bf5f39f654ce50c9fa47b6dd3d0153ba829308

…t an equijoin

Post-push fix: This commit fixes the following warning:
"sql/join_optimizer/cost_model.h:228:1: error: control reaches end of
 non-void function [-Werror=return-type]".

Change-Id: I41fbdd94b6442458a906d12bc176bb72c6b9000a

Problem:
The MySQL Server fails when replicating "GRANT NDB_STORED_USER ..." with
replication filter turned on. This occurs since the replication filter
causes all non-updating queries to return an error - due to the
assumption that only changes need to be replicated.

Analysis:
To handle the GRANT statement a `SELECT ... FROM
information_schema.user_privileges` is used, this is a non-updating
query and thus triggers an error in the MySQL Server when replication
filters are in use.

Solution:
Install an empty replication filter while running distributed privilege
queries, thus sucessfully getting a result returned.

Thanks to Mikael Ronström for providing the steps necessary how
to reproduce this problem.

Contributed by: Mikael Ronström

Change-Id: I2c171f64b7776ac2410d2d918f591b16bc8c1a04

In the Acl_change_notification constructor, remove the ambiguity
between a function parameter and a member variable both called
"users".

Change-Id: I8fba20f562026606411eb8b67cd11d56af09e11e

Problem:
Crash in replication applier when handling NDB synchronized privilieges.

Analysis:
Fatal failure occurs when query return code indicates sucess but no
result set is available. Such problem scenario can be seen in the
replication applier, for example in problem described in BUG#35928350.

Solution:
Handle the case when expected result set is not available by logging an
error message and returning failure. This should make for a more stable
behaviour when the queries does not return the expected result set.

Change-Id: I4623ff8be5e59cbc3edcbd1627cbedda6dbab5ce

Post-push fix: revert bad .result file.

Change-Id: I78f6aa63e4dd3d98be94b98d410ef4e598a23718

…O 8.0

Backport from mysql-trunk.

1) Bug#35211828: Derived condition pushdown with rollup gives wrong results
2) Bug#35498378: MYSQLD CRASH - ASSERTION NULLPTR != DYNAMIC_CAST<TARGET>(ARG) FAILED

Change-Id: I26833f6cc240bc0484e689021a5346029419a0cc

Change-Id: If4b103cff903f3994f8f38410d7c0f6bfeabeabb

authentication_ldap_sasl_client plug-in

So far LDAP authentication was limited depending on authentication
mechanism and the platform server or client runs.
This worklog will improve both the client and the server side Windows plugins, so they support SASL SCRAM and SASL GSSAPI authentication mechanisms.

Change-Id: I60c2ce4925f8d6c18e202b59c54a96303ea8e2ba

…the thread to start

Problem:
When the Transaction_monitor_thread is created, the thread that requested for
the creation waits until Transaction_monitor_thread starts running.
However upon running, the Transaction_monitor_thread thread does not unlock the
mutex causing the creator of a thread to wait for a long time to read the thread
running status.

Analysis:
Checked and observed all other threads on running unlocks the mutex.
In Transaction_monitor_thread the lock was never released post setting the
status of the thread as running.

Fix:
Code has benn improved to unlock the mutex upon thread creation after setting
the status of the thread as being running. The scope of the lock has been
reduced.

Change-Id: I07f961346b99a740c76b1e315850921a2f4fcdb8

authentication_ldap_sasl_client plug-in

Post-push fix: auth_ldap_sasl_mechanism.cc:111:2: error: extra ';'
outside of a function is incompatible with C++98
[-Werror,-Wc++98-compat-extra-semi]

Change-Id: Ida99c78d538f1422e32b8fe4b5caca24fae7f28b

…gIfPossible

A query with redundant elements in the ORDER BY clause could fail if
the query optimization took so long time that the secondary engine
requested a restart of the optimization with a different set of
parameters to restrict the search space.

BuildInterestingOrders() got confused because Query_block::order_list
was in an inconsistent state when the optimization was restarted. It
was inconsistent because the optimizer had modified the intrusive list
pointers in JOIN::order to remove redundant elements. JOIN::order
shares some data with Query_block::order_list, so some of the
underlying data of Query_block::order_list was modified as a result
without the parent object's knowledge, and the parent object became
inconsistent.

Query_block::order_list is restored before the next execution of a
prepared statement for this exact reason; see
Query_block::restore_cmd_properties() and the doxygen comment for
Query_block::order_list. But it is not restored when the optimization
is restarted. Since it's not safe to access Query_block::order_list
after such modifications, the optimizer should use JOIN::order
instead.

Fixed by making BuildInterestingOrders() inspect JOIN::order instead
of Query_block::order_list.

For completeness and consistency, the patch also replaces usages of
Query_block::group_list in the hypergraph optimizer with corresponding
data structures in JOIN, even though the hypergraph optimizer doesn't
currently mutate the group list in a similar way.
BuildInterestingOrders() now uses JOIN::group_list instead of
Query_block::group_list, and EstimateAggregateRows() uses
JOIN::group_fields. (EstimateAggregateRows() already used
JOIN::group_fields for some cases, but only when called by the old
optimizer. It could not use it for the hypergraph optimizer, because
the hypergraph optimizer didn't populate group_fields until after
EstimateAggregateRows(). The patch moved the hypergraph optimizer's
call to make_group_fields() a little earlier so that
EstimateAggregateRows() could use the same code for both optimizers.)

Change-Id: I4790482416e8935b7918cccd36f316d88b1d5700

Original patch broke the Pb2 tests basic_tls.test and tls_required.test.

Post push patch, fixing TransporterRegistry::is_encrypted_link(NodeId).

Root cause was an incorrect transfer of the Multi_Transporter::is_encrypted method
into TransporterRegistry::is_encrypted_link(). We need to get the 'is_encrypted'
property from the first active multitransporter, not from the base-transporter
which may already have been closed if we have switched to use the multi transporters.

Change-Id: If77531ca151839499e1c89358a5f9e6ea336a7c1

Post-push fix:
Broken build for clang on windows: error: unused variable 'trp_id'

Change-Id: I9f637a53267280c90fbcb6266a17b3bb7f7650bf

When the client aborts a TLS handshake because a the certificate can't
be verified (unknown CA) router logs:

   ERROR ... classic::loop() processor failed:
   error:0A000418:SSL routines::tlsv1 alert unknown ca
   (tls_err:167773208)

That error shouldn't be logged as ERROR.

Change
------

- close the connection without raising a "processor failed" error,
  if the TLS handshake fails with the client
  - log a message at INFO level why the tls-handshake failed.
- close the connection without raising a "processor failed" error,
  if it is closed without a COM_QUIT
- decode more TLS alert values for debugging.

Change-Id: I3a492189288d2c430744ec2a32dc40b70ffd0f11

Change-Id: I5fcdc6fe6e06b030f42422627d794b4267b2384c

…bal pointers

Fix this TODO:
// TODO(tdidriks) check name rather than address:
extern MYSQL_STRINGS_EXPORT CHARSET_INFO my_charset_gb18030_chinese_ci;
extern MYSQL_STRINGS_EXPORT CHARSET_INFO my_charset_utf16le_general_ci;

Character sets/collations should always be loaded/initialized properly
with get_collation_number() or some other defined function in the
mysys/strings API.

Change-Id: Ifa89308481d9a3db7428c1f3ed4ba375d1681209

Problem:

Test `ndb_rpl.ndb_rpl_log_updates` fails occasionally on PB2.

Analysis:

The errors are mostly related with synchronization issues, namely:
- Some inserts are not yet applied in the replica cluster in time (but
the updates are in the relay log)
- Table definition may not have been coordinated throughout the
cluster

Solution:

Make each source change wait for them to be applied on the binlog.

Change-Id: I2e79de9d784c87a2b0151b6a9a19413dd5f81b96

Change-Id: I544eb2c5281860ae524294f7d783875da6ac4efd

Problem: Inadequate P_S instrumentation for TP

Solution: Add a new P_S plugin table for TP called tp_connections which
displays the information in the tp_client_low_level_t struct for each
connection. In addition the exiting P_S tables tp_thread_group_state and
tp_thread_state are extended with additional columns for existing data structure
members which were missing, and also with new information which will be
useful for performance tuning/diagnostics.

In addition, some limited refactoring was done to simply information
retrieval for, and the modification of the schema for, the TP P_S tables.

Change-Id: I94414afd00b5a6702ee03ec341c9c0b773a4b94c

'mysql.ndb_apply_status' for synchronization" is submitted every minute

Problem:
On MySQL Cluster 8.0.34, messages like below are submitted to the error
log every minute:

  > Metadata: Failed to submit table 'mysql.ndb_apply_status' for
synchronization

This is essentially useless, and fills the error log unnecessarily.

Analysis:
The mysql.ndb_apply_status is a util table managed by the binlog thread,
as such there is no need to check if it has changed.

Solution:
Remove table from list of tables in DD, this is done in a similar fashion
to how it's removed from the list of tables in NDB. This  means that the
table will not be submitted for change.

Change-Id: I7af9579bcd98b0e7db5b2acccf0b0a7df89a7265

…ystemd isn't notified

Problem:
---------
- During shutdown, InnoDB purge thread can sometimes take a couple of
  hours to complete its work (depending on the data) and it can make
  user feel that the shutdown is stuck.

Solution:
---------
- Add an "externally" visible systemd notification indicating the step.

Note:
-----
- The message is kept general to accommodate pre_dd_shutdown
  handlertons of other SEs (if there are any).

Change-Id: Ibb3f4ab8cb1da45f2fbd374c8ef88c8657ea7b68

When the SqlClient uses the MySQL library it allocate resources which
need to be released using mysql_thread_end(). Such resource leakage is
visible with asan or valgrind.

This is fixed by changing SqlClient to activate a thread_local guard to
release the MySQL thread resources when thread ends.

Also make sure to only initialize the MySQL C API library once, this is
important when running with multiple threads.

Change-Id: I6f49b2725c725c809f3cacb1300619e078e449c7

 fails when Partial LCP is OFF

Patch for 7.6 only

Problem:
LCP_with_many_parts_drop_table is intended to cover a scenario where
an LCP is restored from ~2048 partial LCPs with 1 part each one.
To increase the number of partial-LCPs needed to restore an LCP the
ERROR insert 10048 is used to force each one to use only 1 part.
This works when partial LCP is enabled but for full LCP the error
insert forces the number of parts calculated to be 1 instead of
2048 (default).
This causes the assertion:
ndbassert(is_partial_lcp_enabled() || num_change_parts == 0)
to fail due to the wrong number of parts.

Solution:
Disable LCP_with_many_parts_drop_tables when EnablePartialLcp = 0
since test is useless in these case.
Also, to prevent future issues caused by EI 10048 when partial LCP
is disabled, error insert 10048 handler changed to set the number
of parts to 1 only when partial lcp is enabled. For full lcp EI
10048 does nothing.

Change-Id: I7008d4e3f6ec45621e4a5f811fc0c935f21cc496

Changing test case from using function HLL to using METAPHON, since
HLL cannot be loaded from a default installation.

Change-Id: I96e5d7f702a7be6ff63a0e2f0db44130a4560d34

Add dependency to generate mysqlclient_ername.h

Also add indentation to entries in top level .clang-tidy file
(it was rejected by clang-16)

Remove .clang-tidy and source_downloads from .gitignore

Fix bad comment about clang 10 vs. clang 12.

Change-Id: Idbc6b02a6c7f0018c54a43202d31b40c790596b2

WL#13959 Remove relay_log_info_repository and master_info_repository
POST-PUSH-FIX

After error messages has been changed in the scope of
WL#13959, mysql_57_inplace_upgrade test started to fail
because it expected the obsolete error message:

Error in checking mysql.slave_master_info repository info type of TABLE

Changed mysql_57_inplace_upgrade.test and removed unnecessary
suppressions related to the OBSOLETE_ER_RPL_ERROR_CHECKING_REPOSITORY
and OBSOLETE_ER_RPL_CHANNELS_REQUIRE_TABLES_AS_INFO_REPOSITORIES
error types.

Change-Id: I881d3b89c2f21162bf8ccc6ff034ecfd84c188ef

Change-Id: I9ad641191ce7385e0fa37f241d67c2bf7fc385cb

Change-Id: I01c7c06d016b1b1df4a0f1493a0b540b081240ac

 fails when Partial LCP is OFF

Post-push fix

Removed the usage of nodeId as index to the configuration
stuff. Indexes and nodeId can be different.
Instead, the index of the nodeId section is searched
and then used to retrieve the value of the desired
parameters.

Change-Id: I64b55ee96d471d17049072060ced9a0f8b09f36c

Change-Id: Ib74cdd246055604d3f3d72e5d0420ddff872e65f

In suite ndb_tls:
  Remove large_txn.cnf, since large_txn.test has been removed.
  Remove SendBufferMemory & related params from generic my.cnf.

Change-Id: I64e72bb4410246b9faf4c80bb18bdb98a37d018d

Any additional certificates in the stack must be freed before
open_one() returns.

Change-Id: I363f7c93fc68affa02e2ab7054d808ae5b91111e

…:4399

A query using GROUP BY WITH ROLLUP fails because it is not able to map
an expression in the HAVING clause to the corresponding grouping
column.

The reason why it doesn't find the grouping column, is that Item::eq()
gets confused by Item_cache items being added to one of the items
being compared for equivalence, but not to the other. Item::eq() does
try to see through Item_cache and other wrappers, but the current
logic works only if the wrappers are applied in a given order. For
example, if a cache is on top of a rollup group wrapper, it is able to
see through both the cache and the rollup group wrapper. However, if
the rollup group wrapper is on top of the cache, it sees through the
rollup group wrapper, but not through the cache.

Fixed by making the unwrapping logic keep going as long as there is
something more to unwrap.

Change-Id: I0359f8a56f23ea93c80d8256f02ddbdb582e7f69