pingcap / pingcap/tidb

`IN` subquery produces duplicated rows (semi-join semantics violated) under implicit numeric type coercion

Open
#70,546 3 comments 0 reactions 0 assignees View on GitHub
contribution may-affects-25.10 may-affects-26.3 may-affects-7.5 may-affects-8.1 may-affects-8.5 severity/critical sig/planner type/bug
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Bug Report

Please answer these questions before submitting your issue. Thanks!

Summary

When a numeric column is filtered by `col IN (SELECT string_col FROM ...)`, TiDB returns **duplicated rows**, violating the semi-join semantics of `IN`. A row of the outer table that satisfies the `IN` predicate should be returned **at most once**, but TiDB returns it once **per matching subquery row**, causing the result set to be spuriously enlarged.

The bug is triggered when the subquery returns multiple **distinct** string values that all coerce to the **same numeric value** as the outer column. TiDB deduplicates the subquery side by the *original* (string) values (so distinct strings are all kept), while the join equality is evaluated on *coerced* (double) values (so they all match), degrading the intended semi-join into an inner join.

- **Affected:** TiDB `v8.5.7` (also reproduced on `v9.0.0-beta.1`). MySQL 8.0.30 is **not** affected (returns correct results), confirming this is a TiDB-specific logic bug.

### 1. Minimal reproduce step (Required)

```sql
CREATE TABLE t0(c0 TINYINT(1));
CREATE TABLE t1(c0 BLOB, KEY i0(c0(79)));
INSERT INTO t0 VALUES (0),(0);
INSERT INTO t1 VALUES ('a'),('b'),('c');

-- Q1 (buggy): IN is a semi-join; each qualifying t0 row must appear at most once.
SELECT t0.c0 FROM t0 WHERE t0.c0 IN (SELECT t1.c0 FROM t1 WHERE t1.c0 IS NOT NULL);
-- TiDB returns 6 rows (0,0,0,0,0,0) <-- WRONG
-- Expected 2 rows (0,0)

-- Q2 (semantically equivalent, correct): materialize the IN predicate as a boolean column.
SELECT ref0 FROM (
SELECT t0.c0 AS ref0,
t0.c0 IN (SELECT t1.c0 FROM t1 WHERE t1.c0 IS NOT NULL) AS ref1
FROM t0
) s WHERE ref1;
-- TiDB returns 2 rows (0,0) <-- CORRECT
```

`Q1` and `Q2` are semantically equivalent; they must return identical multisets. TiDB returns 6 vs 2.

### 2. What did you expect to see? (Required)
| Engine | Q1 (`WHERE ... IN (subquery)`) | Q2 (relocated) | Correct? |
|---|---|---|---|
| MySQL 8.0.30 | 2 rows | 2 rows | ✅ correct |
### 3. What did you see instead (Required)
| Engine | Q1 (`WHERE ... IN (subquery)`) | Q2 (relocated) | Correct? |
|---|---|---|---|
| MySQL 8.0.30 | 2 rows | 2 rows | ✅ correct |
| **TiDB v8.5.7** | **6 rows** | 2 rows | ❌ Q1 wrong |

### 4. What is your TiDB version? (Required)

| 字段 | 值 |
|---|---|
| Release Version | v8.5.7 |
| Edition | Community |
| Git Commit Hash | 202b7f47286a1109b5c957401d34c9358d130ae0 |
| Git Branch | HEAD |
| UTC Build Time | 2026-08-11 04:12:50 |
| GoVersion | go1.25.10 |
| Race Enabled | false |
| Check Table Before Drop | false |
| Store | unistore |

Contributor guide

Open the contributing guide

Research direction

Start by running the supplied Q1 and Q2 SQL reproducer on TiDB v8.5.7 and compare the row counts. Trace the optimizer and execution path for the IN subquery, focusing on semi-join deduplication when string values are implicitly coerced to numeric values. Done means Q1 and Q2 return identical multisets, with each qualifying outer row appearing at most once.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, sql
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
58/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.