ClickHouse / ClickHouse/ClickHouse
Aggregating over `x IN (subquery using shardNum())` from `remote()`/`Distributed` fails: 10 NOT_FOUND_COLUMN_IN_BLOCK — initiator and shard disagree on the `__set_` id
- Dominant language
- C++
- Stars
- 49.9k
- Forks
- 9k
- Avg merge
- 21h 32m
- Merged PRs (30d)
- 515
Description
## Describe the problem
When a query over `remote()` or a `Distributed` table (2+ shards) aggregates over an `IN (subquery)` whose subquery uses a shard-dependent function such as `shardNum` or `hostName`, the query fails with exception 10 (`NOT_FOUND_COLUMN_IN_BLOCK`) at pure default settings.
The initiator expects a column named with the `__set_` id from its own analysis, while each shard returns a column named with the `__set_` id from its own analysis of the shipped query — the shard folds `shardNum()` to its local constant, so the set hash (and hence the column name) differs — and `RemoteQueryExecutor` fails to match the block by name:
```
Code: 10. DB::Exception: Not found column sum(in(__table3.number, __set_4038495261199527788_15042459581097328311)) in block.
There are only columns: sum(in(__table3.number, __set_16854109006789820482_7702020288925574204)): While executing Remote. (NOT_FOUND_COLUMN_IN_BLOCK)
```
The semantically equivalent plain-`WHERE` form (`WHERE x IN (...)`) works and returns the expected per-shard-set result, so this is purely a naming/header mismatch on the projection path, not a semantics question about `shardNum` in a distributed subquery.
## How to reproduce
Version: `26.8.1.653` (public master build). All settings at defaults; no DDL needed:
```sql
SELECT sum(number IN (SELECT number FROM numbers(10) WHERE number = shardNum()))
FROM remote('127.0.0.{1,2}', numbers(10));
```
Observed (deterministic, 20/20 runs):
```
Code: 10. DB::Exception: Not found column sum(in(__table3.number, __set_4038495261199527788_15042459581097328311)) in block. There are only columns: sum(in(__table3.number, __set_16854109006789820482_7702020288925574204)): While executing Remote. (NOT_FOUND_COLUMN_IN_BLOCK)
```
Expected: `2` (each of the two shards contributes one matching row; both the pre-analyzer path `enable_analyzer = 0` and the plan-shipping path `serialize_query_plan = 1` return `2`).
The equivalent filter form works at defaults and returns the correct result:
```sql
SELECT count()
FROM remote('127.0.0.{1,2}', numbers(10))
WHERE number IN (SELECT number FROM numbers(10) WHERE number = shardNum());
-- 2
```
A plain `Distributed` table triggers it identically (no `remote()` needed):
```sql
CREATE TABLE t_local (x UInt64) ENGINE=MergeTree ORDER BY x;
INSERT INTO t_local SELECT number FROM numbers(100);
-- any 2-shard cluster works the same
CREATE TABLE dist_n (x UInt64) ENGINE=Distributed(test_cluster_two_shards, currentDatabase(), t_local); -- any 2-shard cluster
SELECT sum(x IN (SELECT number FROM numbers(10) WHERE number = shardNum())) FROM dist_n;
-- Code: 10, same two mismatched __set_ ids
```
What is REQUIRED vs incidental:
- REQUIRED: default settings only — nothing needs to be changed.
- REQUIRED (query shape): 2 or more shards (a single remote shard works); a shard-dependent function inside the `IN` subquery — `shardNum()` and `hostName()` both trigger it, while a constant subquery, `rand()`, and `currentDatabase()` do not; the `IN` must be evaluated in the shard-side projection (an aggregate such as `sum`/`countIf`, or the bare expression in `SELECT`) — the same `IN` in `WHERE` is fine.
- Incidental: the aggregate wrapper (`sum`, `countIf`, `ifNull(sum(...))`, `toUInt64(... != 0)` all fail the same way), row counts, whether the initiator's own shard participates (`prefer_localhost_replica = 0` and a pure-remote shard list fail identically).
- Not failing on: `enable_analyzer = 0`, or `serialize_query_plan = 1` (plan shipping instead of SQL-text shipping).
Note the mismatched `__set_` id in "There are only columns" also changes with which shard the offending block came from, consistent with each shard hashing the subquery after folding `shardNum()` to its own value: `remote('127.0.0.{1,2}')` and `remote('127.0.0.{1..3}')` report `__set_16854...`, `remote('127.0.0.{2,3}')` reports `__set_11166...`, while the expected id `__set_40384...` is always the initiator's.
## Additional context
The bare-projection form without aggregation (`SELECT number IN (...) FROM remote(...) LIMIT 1`) fails only sporadically (a race — roughly 4 out of 5 runs with `LIMIT 1`, never without `LIMIT`), while the aggregate form is deterministic, so the aggregate form above is the reliable repro.
Nearest prior reports, both closed and mechanically different: `shardNum` folding breaking the outer projection name (no sets involved), and a set-identity mismatch specific to shipped `INTERPOLATE` expression packages:
Related: https://github.com/ClickHouse/ClickHouse/issues/80691 Related: https://github.com/ClickHouse/ClickHouse/issues/111728
Contributor guide
Assessment
This issue has not been assessed yet.