GLOBAL RIGHT/FULL JOIN samples wrong table after side swap, dropping rows
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 72/100
- Issue type
- Bug
- Clarity
- Clearly specified
- Activity status
- Active
- Domain
- databases, distributed-systems
Research direction
Start by running the provided reproducer with enable_analyzer=1 and at least two shards. Trace tryRewriteGlobalRightJoinAsLeftJoin in src/Planner/PlannerJoinTree.cpp, then inspect getSampling in src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp and ASTSelectQuery::sampleSize in src/Parsers/ASTSelectQuery.cpp. Done means SAMPLE applies only to its own table for GLOBAL RIGHT/FULL JOINs, producing the expected 1498-row results without SAMPLING_NOT_SUPPORTED.
Written by the indexing model from the issue text.
Description
Describe what's wrong
SAMPLEon a sharded left table leaks onto the other joined table, or the query is rejected- Triggered by
GLOBAL RIGHT JOIN/GLOBAL FULL JOINagainst aDistributedtable with >= 2 shards carryingSAMPLE - Expected 1498 rows returned, actual returns 996 or fails with
SAMPLING_NOT_SUPPORTED - Single-shard
Distributedtables are unaffected, only >= 2 shards trigger this
Root cause: tryRewriteGlobalRightJoinAsLeftJoin swaps table expressions or wraps the leftmost table in a subquery, so the join no longer runs per shard against a GLOBAL temp table; getSampling then falls back to ASTSelectQuery::sampleSize, which returns the sample of the AST's first table expression instead of the correct node's own modifiers.
Analysis details (evidence, affected locations, impact)
Why we believe this is a bug: buildJoinTreeQueryPlan (src/Planner/PlannerJoinTree.cpp:3436) calls tryRewriteGlobalRightJoinAsLeftJoin, which at :3421 swaps the join's table expressions, and at :3599 the new has_global_join_preserving_broadcast_rows branch wraps the leftmost table in a subquery. Either way the join is no longer executed per shard against a GLOBAL temporary table, so the non-Distributed side becomes an ordinary in-query MergeTree read -> MergeTreeDataSelectExecutor::getSampling -> the else branch at src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp:289 -> ASTSelectQuery::sampleSize (src/Parsers/ASTSelectQuery.cpp:414-421) returns the sample of the AST's FIRST table expression.
Affected locations:
src/Planner/PlannerJoinTree.cpp:3421—tryRewriteGlobalRightJoinAsLeftJoinswaps the join's table expressions; gate at :3380 isgetShardCount() < 2src/Planner/PlannerJoinTree.cpp:3599—has_global_join_preserving_broadcast_rowsforcesshould_wrap_left_tableforFULL/ non-swappableRIGHTjoinssrc/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp:289—getSamplingfalls back to the query AST's first table expression when the node carries noTableExpressionModifiers
Impact: Silent row loss on SAMPLE + GLOBAL RIGHT/FULL JOIN over a sharded Distributed table, or outright SAMPLING_NOT_SUPPORTED refusal of a query that works today. Reachable without the GLOBAL keyword whenever distributed_product_mode = 'global' or prefer_global_in_and_join = 1 is set, since isGlobalJoin (:3339) treats those settings as global.
Does it reproduce on most recent release?
Yes — confirmed on current master (commit b1be561a03fd).
How to reproduce
Requires enable_analyzer = 1 and a Distributed table with at least 2 shards
Reproducer
-- SAMPLE must apply to the table it is written on, also under GLOBAL RIGHT / FULL JOIN.
DROP TABLE IF EXISTS t_sampled_05183;
DROP TABLE IF EXISTS t_dim_05183;
SET enable_analyzer = 1;
SET enable_parallel_replicas = 0;
SET max_parallel_replicas = 1;
SET prefer_localhost_replica = 1;
DROP TABLE IF EXISTS t_sampled_05183;
DROP TABLE IF EXISTS t_dim_05183;
CREATE TABLE t_sampled_05183 (k UInt32) ENGINE = MergeTree ORDER BY intHash32(k) SAMPLE BY intHash32(k);
CREATE TABLE t_dim_05183 (k UInt32) ENGINE = MergeTree ORDER BY intHash32(k) SAMPLE BY intHash32(k);
INSERT INTO t_sampled_05183 SELECT number FROM numbers(1000);
INSERT INTO t_dim_05183 SELECT number FROM numbers(1000);
SELECT count()
FROM remote('127.0.0.1', currentDatabase(), t_sampled_05183) AS l SAMPLE 0.5
GLOBAL RIGHT JOIN t_dim_05183 AS r ON l.k = r.k;
-- Subquery keeps the join on the initiator and defines the correct result for two shards.
SELECT count()
FROM (SELECT * FROM remote('127.0.0.1,127.0.0.1', currentDatabase(), t_sampled_05183) SAMPLE 0.5) AS l
RIGHT JOIN t_dim_05183 AS r ON l.k = r.k;
SELECT count()
FROM remote('127.0.0.1,127.0.0.1', currentDatabase(), t_sampled_05183) AS l SAMPLE 0.5
GLOBAL RIGHT JOIN t_dim_05183 AS r ON l.k = r.k;
SELECT count()
FROM remote('127.0.0.1,127.0.0.1', currentDatabase(), t_sampled_05183) AS l SAMPLE 0.5
GLOBAL FULL JOIN t_dim_05183 AS r ON l.k = r.k;
DROP TABLE t_sampled_05183;
DROP TABLE t_dim_05183;
Verified: this block reproduces on current master (b1be561a03fd) when pasted as is.
Expected behavior
SAMPLE should apply only to the table it is written on regardless of join side swapping
Expected output of the reproducer above:
1000
1498
1498
1498
Error message and/or stacktrace
The other table is sampled too, causing row loss, or the query fails with SAMPLING_NOT_SUPPORTED if that table has no sampling key
Actual output of the reproducer above on master (b1be561a03fd):
1000
1498
996
996
Suggested fix
Either skip the rewrite and the subquery wrap when the leftmost table expression carries TableExpressionModifiers with a sample ratio (trading the sampling regression back for the row-duplication this PR fixes), or fix the root cause in MergeTreeDataSelectExecutor::getSampling / ReadFromMergeTree::isQueryWithSampling so that under the analyzer the sample is taken only from the node's own TableExpressionModifiers and never from ASTSelectQuery::sampleSize.
Additional context
Open risks:
ReadFromMergeTree::isQueryWithSampling(src/Processors/QueryPlan/ReadFromMergeTree.cpp:4243) andStorageMerge(src/Storages/StorageMerge.cpp:1116) read the same AST fallback, so aMergetable on the non-sampled side is likely affected in the same way; not measured here.- The rewrite also moves the join off the shards for
RIGHT/FULLjoins wheneverdistributed_product_mode = 'global'orprefer_global_in_and_join = 1is set, including when the other side is a purely local table thatbuildQueryTreeForShardnever globalizes. On a real cluster that changes which shard's local table is read; not reproducible on a single-server sandbox.
Found during automated review of PR #118611; whether that PR introduced it could not be established, so nobody is tagged. Severity P2 · Finding h_pr118611_201
- Dominant language
- C++
- Stars
- 50k
- Forks
- 9k
- Avg merge
- 18h 29m
- Merged PRs (30d)
- 511
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from ClickHouse/ClickHouse
-
comp-datalake
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
ClickHouse/ClickHouse#121222 ·
-
comp-sql-syntax minor
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
ClickHouse/ClickHouse#121170 ·
-
comp-sql-syntax
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
ClickHouse/ClickHouse#121150 ·
-
comp-sql-syntax fuzz
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
ClickHouse/ClickHouse#121027 · 2 comments ·
-
comp-sql-syntax fuzz
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
ClickHouse/ClickHouse#121025 · 2 comments ·
All issues in ClickHouse/ClickHouse
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
-
Sensor initialization takes very long when `--initial-sim-time` is set to current UNIX timestamp Open
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
gazebosim/gz-sensors#662 · 1 comment ·
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
LadybirdBrowser/ladybird#12123 ·
-
[adam] AdamNet network read doesn't cap to MAX_ADAM_PACKET_LEN, overflows client receive buffers Open
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
FujiNetWIFI/fujinet-firmware#1649 · 2 comments ·