ClickHouse / ClickHouse/ClickHouse

transform_null_in=1 fails for non-nullable String IN nullable subquery result

Open
#111,340 3 comments 0 reactions 0 assignees View on GitHub
bug comp-query-execution external
Dominant language
C++
Stars
49.9k
Forks
9k
Avg merge
21h 32m
Merged PRs (30d)
515

Description

### What happened?

`IN` with `transform_null_in = 1` fails when its left argument is a non-nullable `String` column and a subquery on the right produces `NULL`.

### How to reproduce

```sql
CREATE OR REPLACE TABLE t (s String) ENGINE = MergeTree ORDER BY s;
INSERT INTO t VALUES ('a'), ('b'), ('c');

SELECT s
FROM t
WHERE s IN (SELECT s FROM t UNION ALL SELECT NULL)
SETTINGS transform_null_in = 1;
```

### Expected behavior

The query should return `a`, `b`, and `c`.

A non-nullable left argument cannot match `NULL`; this should behave consistently with the literal form:

```sql
SELECT s FROM t WHERE s IN ('a', NULL) SETTINGS transform_null_in = 1;
```

The same subquery also succeeds when the left expression is explicitly nullable:

```sql
SELECT s
FROM t
WHERE toNullable(s) IN (SELECT s FROM t UNION ALL SELECT NULL)
SETTINGS transform_null_in = 1;
```

### Actual behavior

```
Code: 349. DB::Exception: Cannot convert NULL value to non-Nullable type.
(CANNOT_INSERT_NULL_IN_ORDINARY_COLUMN)
```

### Versions

Reproduced on:

- ClickHouse 25.8.16
- ClickHouse 26.3.5.12

The problem is also type-specific: replacing `String` with `Int64` in this reproduction succeeds.

[PR #81584](https://github.com/ClickHouse/ClickHouse/pull/81584) fixes the opposite nullability direction (nullable left argument and non-nullable subquery result), so it does not cover this case.

Contributor guide

Open the contributing guide

Research direction

Start by running the supplied ClickHouse SQL reproduction with transform_null_in=1, then trace how the IN subquery result is converted when the left String expression is non-nullable. Compare the behavior with the literal and toNullable examples, and use the successful Int64 case to narrow the type-specific path. Done means the original query returns a, b, and c without a null-conversion exception.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, sql
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.