Replacing collapsing merge tree

[youennL-cs]

Changelog category (leave one):

• New Feature

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Implementation of a new MergeTree engine: ReplacingCollapsingMergeTree.

It is similar to ReplacingMergeTree, but have an extra sign column (possible values to discuss: 1 / -1 (or 1 / 0 depending if we can avoid to return rows with -1 sign))
No matter the operation on the data, version must be increased. If two inserted rows have the same version number, the last inserted is the one kept.

More details:
It allows to easily "update" a row by inserting row with (arbitrary) greater version ("replacing" merge algorithm collapses all rows with the same key into one row with the greatest version)
It allows to easily "delete" a row by inserting row with (arbitrary) greater version and -1 sign ("replacing" merge will leave only one row with sign = -1 and then it will be filtered out)
It allows to "update" key columns by deleting a row (as described above) and inserting new one
"Replacing" merge algorithm removes duplicates as well, so it's safe to insert multiple rows with the same version.

To discuss:
How to filter out data:
First solution: We can easily do it in the query
Second solution: Set sign to 0, so when we sum we don't count this rows and KPIs go well
Last solution: Find how to filter out a rows in CH source code and return an ReplacingCollapsingMergeTree row only if sign != -1.

Information about CI checks: https://clickhouse.com/docs/en/development/continuous-integration/

[gontarzpawel]

Why do we have 3 times the same d1?
Why do we have 3 times d2, with different values? etc

[youennL-cs]

This "reference" file is the expected output of the {same name}.sql
If we look at the sql file, we will see 3 calls to select * from test; that's why. The three output are following without blank/newline in-between.
So you should have 2 times:

d1	1	5
d2	1	1
d3	1	1
d4	1	3
d5	1	1
d6	-1	2

and the last insert add overlapping data with new ones, so the output is a little different with

d1	1	5
d2	1	3
d3	1	3
d4	1	3
d5	1	1
d6	-1	2

new versions of d2 and d3 are in this new batch.

[gontarzpawel]

oh I somehow missed that there're 3 SELECT calls 👍

[gontarzpawel]

can we shuffle the input values so that we have:

('d1', 1, 3),('d1', -1, 2),('d1', 1, 1)

?

[youennL-cs]

We can, I just want to check that if we insert values in "random" order, it doesn't affect the output.

[gontarzpawel]

It can still be random, I was just curious about sorting (but it has to work otherwise next tests wouldn't work)