ClickHouse / ClickHouse/ClickHouse
KeyCondition over-prunes the signed-zero granule for f = 0.0 when the primary key is an injective float transform
- Dominant language
- C++
- Stars
- 49.9k
- Forks
- 9k
- Avg merge
- 21h 32m
- Merged PRs (30d)
- 515
Description
**Describe what's wrong**
When the primary key is an injective transform of a floating-point column (for example `ORDER BY toString(f)` or `ORDER BY reinterpretAsUInt64(f)`) and a query filters the raw float column with `f = 0.0`, the primary-key index silently prunes the granule that holds the `-0.0` rows. Those rows satisfy the predicate (`-0.0 = 0.0` is true) but are never read and never row-filtered, so the query returns fewer rows than it should. This happens at default settings (`use_primary_key = 1`).
The cause is in `KeyCondition::canConstantBeWrappedByDeterministicFunctions` (`src/Storages/MergeTree/KeyCondition.cpp`). When the key is a transform of a column and the predicate constant is pushed through the transform, the atom is marked *exact* whenever the transform is injective. Injective float-to-non-float transforms distinguish `-0.0` from `+0.0` (`toString(-0.0) = '-0'` vs `toString(0.0) = '0'`; `reinterpretAsUInt64(-0.0) = 9223372036854775808` vs `reinterpretAsUInt64(0.0) = 0`), while IEEE float equality compares them equal. So the exact key atom `toString(f) IN ['0','0']` is stricter than the original predicate `f = 0.0`, and the granule containing `-0.0` is dropped.
This is the same family as the NaN case being fixed in the open PR #117501 ("Do not treat a key range as exact when the constant reaches the key transform as a NaN"). That PR guards only NaN (`!transform_input_has_nan`); the signed-zero shape described here escapes the fix because `0.0` is not NaN.
**How to reproduce**
Version: 26.9.1.1 (reproduces on current master; the code path is unchanged by #117501 for the signed-zero case).
```sql
CREATE TABLE t (f Float64) ENGINE = MergeTree ORDER BY toString(f)
SETTINGS index_granularity = 1;
INSERT INTO t VALUES (-0.0), (-0.5), (0.0), (2.0);
SELECT count() FROM t WHERE f = 0.0; -- returns 1 (wrong)
SELECT count() FROM t WHERE f = 0.0 SETTINGS use_primary_key = 0; -- returns 2 (correct)
```
`EXPLAIN indexes = 1 SELECT count() FROM t WHERE f = 0.0` shows `Condition: (toString(f) in ['0', '0'])` and `Granules: 2/4` — the `-0.0` granule is pruned by the primary key index. The same happens for the predicate `f = -0.0`, and with `ORDER BY reinterpretAsUInt64(f)` as the key. `index_granularity = 1` is only a minimal-repro convenience so the `-0.0` and `+0.0` rows land in different granules; with default granularity the same divergence occurs once a table has enough float rows to spread `-0.0` and `+0.0` across granule boundaries. The row is not lost on disk — `OPTIMIZE TABLE t FINAL` keeps all four rows and a full scan still returns 2 — so this is purely query-time granule over-pruning.
**Expected behavior**
`SELECT count() FROM t WHERE f = 0.0` returns 2, matching both the `-0.0` and `+0.0` rows, regardless of `use_primary_key`.
Contributor guide
Assessment
This issue has not been assessed yet.