pingcap / pingcap/tidb

`tidb_enable_index_merge_join=ON` causes nondeterministic wrong results (random subset of rows) in IN-subquery queries executed by IndexMergeJoin

Open
#71,168 4 comments 0 reactions 0 assignees View on GitHub
affects-25.10 affects-8.5 contribution severity/critical sig/execution type/bug
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Bug Report

**TiDB version:**
`8.0.11-TiDB-v8.5.8` (official docker image `pingcap/tidb:v8.5.8`)

**Steps to reproduce:**

```sql
DROP TABLE IF EXISTS ta, tb;
CREATE TABLE ta (id INT PRIMARY KEY, c0 INT, c1 INT, c2 VARCHAR(50));
CREATE TABLE tb (id INT PRIMARY KEY, c0 INT, c1 INT, c2 VARCHAR(50));
CREATE INDEX ix_tb_c0 ON tb(c0);
INSERT INTO ta VALUES (1,10,100,'a'),(2,20,200,'b'),(3,30,300,'c'),(4,NULL,400,'d'),(5,10,500,'e'),(6,20,NULL,'f'),(7,40,700,'g'),(8,50,800,'h');;
INSERT INTO tb VALUES (1,10,1000,'r1'),(2,20,2000,'r2'),(3,20,2000,'r3'),(4,10,1000,'r4'),(5,60,1000,'r5'),(6,10,1000,'r6'),(7,20,2000,'r7'),(8,10,2000,'r8'),(9,60,2000,'r9'),(10,NULL,NULL,'r10'),(11,10,2000,'r11'),(12,60,NULL,'r12'),(13,20,2000,'r13'),(14,20,NULL,'r14'),(15,10,1000,'r15'),(16,60,1000,'r16'),(17,30,NULL,'r17'),(18,20,1000,'r18'),(19,NULL,1000,'r19'),(20,60,1000,'r20'),(21,20,NULL,'r21'),(22,20,1000,'r22'),(23,10,2000,'r23'),(24,20,1000,'r24'),(25,20,1000,'r25'),(26,60,NULL,'r26'),(27,NULL,NULL,'r27'),(28,20,NULL,'r28'),(29,30,2000,'r29'),(30,20,1000,'r30'),(31,20,2000,'r31'),(32,20,416,'r32'),(33,60,NULL,'r33'),(34,20,NULL,'r34'),(35,10,2000,'r35'),(36,10,NULL,'r36'),(37,60,NULL,'r37'),(38,10,2000,'r38'),(39,30,2000,'r39'),(40,NULL,520,'r40'),(41,NULL,2000,'r0'),(42,20,2000,'r1'),(43,20,NULL,'r2'),(44,60,572,'r3'),(45,30,2000,'r4'),(46,20,598,'r5'),(47,10,1000,'r6'),(48,10,2000,'r7'),(49,20,637,'r8'),(50,10,650,'r9'),(51,60,663,'r10'),(52,20,1000,'r11'),(53,10,NULL,'r12'),(54,30,1000,'r13'),(55,20,715,'r14'),(56,20,728,'r15'),(57,10,NULL,'r16'),(58,20,1000,'r17'),(59,20,2000,'r18'),(60,20,NULL,'r19'),(61,20,1000,'r20'),(62,30,806,'r21'),(63,10,1000,'r22'),(64,30,NULL,'r23'),(65,20,1000,'r24'),(66,20,1000,'r25'),(67,10,871,'r26'),(68,10,2000,'r27'),(69,20,897,'r28'),(70,20,NULL,'r29'),(71,60,2000,'r30'),(72,20,NULL,'r31'),(73,60,NULL,'r32'),(74,NULL,962,'r33'),(75,10,2000,'r34'),(76,20,1000,'r35'),(77,30,1000,'r36'),(78,20,2000,'r37'),(79,10,1000,'r38'),(80,10,2000,'r39'),(81,10,1000,'r40'),(82,30,1000,'r0'),(83,20,NULL,'r1'),(84,NULL,2000,'r2'),(85,20,1105,'r3'),(86,20,1118,'r4'),(87,60,2000,'r5'),(88,10,1000,'r6'),(89,60,NULL,'r7'),(90,60,1170,'r8'),(91,NULL,1000,'r9'),(92,10,1000,'r10'),(93,60,NULL,'r11'),(94,10,2000,'r12'),(95,20,2000,'r13'),(96,NULL,2000,'r14'),(97,60,2000,'r15'),(98,20,1274,'r16'),(99,20,1000,'r17'),(100,NULL,1000,'r18'),(101,10,1000,'r19'),(102,10,2000,'r20'),(103,20,1339,'r21'),(104,NULL,1352,'r22'),(105,20,1365,'r23'),(106,10,2000,'r24'),(107,60,1000,'r25'),(108,60,NULL,'r26'),(109,NULL,NULL,'r27'),(110,60,1430,'r28'),(111,20,2000,'r29'),(112,20,2000,'r30'),(113,10,1000,'r31'),(114,30,1000,'r32'),(115,10,1495,'r33'),(116,20,1000,'r34'),(117,10,2000,'r35'),(118,10,1000,'r36'),(119,20,1547,'r37'),(120,10,2000,'r38'),(121,10,1000,'r39'),(122,60,NULL,'r40'),(123,20,2000,'r0'),(124,30,2000,'r1'),(125,20,1625,'r2'),(126,20,NULL,'r3'),(127,NULL,NULL,'r4'),(128,10,1000,'r5'),(129,NULL,1000,'r6'),(130,10,2000,'r7'),(131,20,2000,'r8'),(132,30,1000,'r9'),(133,20,NULL,'r10'),(134,20,2000,'r11'),(135,NULL,NULL,'r12'),(136,10,NULL,'r13'),(137,10,2000,'r14'),(138,20,1000,'r15'),(139,10,2000,'r16'),(140,20,NULL,'r17'),(141,20,NULL,'r18'),(142,20,NULL,'r19'),(143,NULL,NULL,'r20'),(144,10,1000,'r21'),(145,60,NULL,'r22'),(146,10,1000,'r23'),(147,30,2000,'r24'),(148,20,2000,'r25'),(149,NULL,1937,'r26'),(150,10,1000,'r27'),(151,10,2000,'r28'),(152,10,NULL,'r29'),(153,20,1989,'r30'),(154,20,1000,'r31'),(155,20,NULL,'r32'),(156,10,NULL,'r33'),(157,20,2000,'r34'),(158,10,NULL,'r35'),(159,60,2000,'r36'),(160,20,2000,'r37'),(161,20,2093,'r38'),(162,10,2000,'r39'),(163,30,2119,'r40'),(164,20,NULL,'r0'),(165,20,1000,'r1'),(166,60,1000,'r2'),(167,NULL,2000,'r3'),(168,20,2184,'r4'),(169,30,NULL,'r5'),(170,20,2000,'r6'),(171,10,2000,'r7'),(172,60,NULL,'r8'),(173,20,1000,'r9'),(174,20,NULL,'r10'),(175,60,NULL,'r11'),(176,10,1000,'r12'),(177,20,2000,'r13'),(178,20,1000,'r14'),(179,10,2327,'r15'),(180,30,NULL,'r16'),(181,60,1000,'r17'),(182,60,2000,'r18'),(183,20,1000,'r19'),(184,60,1000,'r20'),(185,20,NULL,'r21'),(186,60,1000,'r22'),(187,30,NULL,'r23'),(188,10,NULL,'r24'),(189,20,2457,'r25'),(190,30,2470,'r26'),(191,20,2000,'r27'),(192,20,2496,'r28'),(193,60,2000,'r29'),(194,20,2522,'r30'),(195,10,NULL,'r31'),(196,20,2000,'r32'),(197,60,NULL,'r33'),(198,NULL,2574,'r34'),(199,NULL,2000,'r35'),(200,NULL,2000,'r36'),(201,10,NULL,'r37'),(202,30,1000,'r38'),(203,20,NULL,'r39'),(204,20,2000,'r40'),(205,20,1000,'r0'),(206,10,2000,'r1'),(207,NULL,1000,'r2'),(208,NULL,2704,'r3'),(209,20,2717,'r4'),(210,NULL,2730,'r5'),(211,20,2000,'r6'),(212,10,1000,'r7'),(213,60,2000,'r8'),(214,20,2782,'r9'),(215,10,2000,'r10'),(216,10,2808,'r11'),(217,20,2821,'r12'),(218,30,2834,'r13'),(219,60,2847,'r14'),(220,20,2860,'r15'),(221,NULL,NULL,'r16'),(222,20,NULL,'r17'),(223,NULL,2000,'r18'),(224,20,2912,'r19'),(225,10,NULL,'r20'),(226,20,NULL,'r21'),(227,30,NULL,'r22'),(228,10,2000,'r23'),(229,20,2000,'r24'),(230,60,2000,'r25'),(231,20,1000,'r26'),(232,60,3016,'r27'),(233,30,3029,'r28'),(234,60,1000,'r29'),(235,20,3055,'r30'),(236,60,1000,'r31'),(237,60,3081,'r32'),(238,10,NULL,'r33'),(239,20,3107,'r34'),(240,60,2000,'r35'),(241,NULL,2000,'r36'),(242,20,3146,'r37'),(243,NULL,1000,'r38'),(244,60,NULL,'r39'),(245,60,2000,'r40'),(246,NULL,2000,'r0'),(247,10,3211,'r1'),(248,10,1000,'r2'),(249,60,2000,'r3'),(250,NULL,2000,'r4'),(251,10,NULL,'r5'),(252,60,NULL,'r6'),(253,20,3289,'r7'),(254,30,NULL,'r8'),(255,60,NULL,'r9'),(256,60,NULL,'r10'),(257,10,3341,'r11'),(258,10,1000,'r12'),(259,30,2000,'r13'),(260,10,1000,'r14'),(261,10,2000,'r15'),(262,20,1000,'r16'),(263,20,2000,'r17'),(264,20,3432,'r18'),(265,10,2000,'r19'),(266,NULL,NULL,'r20'),(267,30,2000,'r21'),(268,10,2000,'r22'),(269,20,1000,'r23'),(270,10,NULL,'r24'),(271,60,3523,'r25'),(272,20,1000,'r26'),(273,20,1000,'r27'),(274,20,1000,'r28'),(275,10,NULL,'r29'),(276,60,NULL,'r30'),(277,10,NULL,'r31'),(278,10,3614,'r32'),(279,NULL,1000,'r33'),(280,60,NULL,'r34'),(281,NULL,2000,'r35'),(282,60,2000,'r36'),(283,20,3679,'r37'),(284,NULL,3692,'r38'),(285,20,NULL,'r39'),(286,20,1000,'r40'),(287,20,3731,'r0'),(288,20,3744,'r1'),(289,60,NULL,'r2'),(290,10,3770,'r3'),(291,30,2000,'r4'),(292,NULL,2000,'r5'),(293,30,NULL,'r6'),(294,30,NULL,'r7'),(295,20,1000,'r8'),(296,20,1000,'r9'),(297,20,3861,'r10'),(298,NULL,2000,'r11'),(299,NULL,3887,'r12'),(300,NULL,1000,'r13'),(301,10,NULL,'r14'),(302,20,3926,'r15'),(303,20,NULL,'r16'),(304,30,3952,'r17'),(305,30,3965,'r18'),(306,30,NULL,'r19'),(307,NULL,NULL,'r20'),(308,20,NULL,'r21'),(309,20,1000,'r22'),(310,20,NULL,'r23'),(311,10,2000,'r24'),(312,20,2000,'r25'),(313,NULL,NULL,'r26'),(314,20,1000,'r27'),(315,20,NULL,'r28'),(316,20,NULL,'r29'),(317,20,NULL,'r30'),(318,10,2000,'r31'),(319,20,1000,'r32'),(320,10,NULL,'r33'),(321,20,4173,'r34'),(322,10,1000,'r35'),(323,20,4199,'r36'),(324,NULL,4212,'r37'),(325,30,2000,'r38'),(326,10,NULL,'r39'),(327,NULL,1000,'r40'),(328,10,4264,'r0'),(329,NULL,1000,'r1'),(330,10,2000,'r2'),(331,20,NULL,'r3'),(332,10,2000,'r4'),(333,10,4329,'r5'),(334,20,4342,'r6'),(335,NULL,4355,'r7'),(336,20,4368,'r8'),(337,20,1000,'r9'),(338,10,2000,'r10'),(339,20,NULL,'r11'),(340,10,2000,'r12'),(341,20,2000,'r13'),(342,10,2000,'r14'),(343,10,4459,'r15'),(344,NULL,4472,'r16'),(345,20,1000,'r17'),(346,20,NULL,'r18'),(347,20,4511,'r19'),(348,10,2000,'r20'),(349,20,2000,'r21'),(350,60,1000,'r22'),(351,20,2000,'r23'),(352,20,2000,'r24'),(353,10,1000,'r25'),(354,20,NULL,'r26'),(355,20,4615,'r27'),(356,10,4628,'r28'),(357,20,4641,'r29'),(358,20,4654,'r30'),(359,NULL,1000,'r31'),(360,10,4680,'r32'),(361,30,NULL,'r33'),(362,10,1000,'r34'),(363,20,1000,'r35'),(364,20,1000,'r36'),(365,20,4745,'r37'),(366,NULL,NULL,'r38'),(367,20,4771,'r39'),(368,NULL,1000,'r40'),(369,NULL,NULL,'r0'),(370,60,NULL,'r1'),(371,30,1000,'r2'),(372,20,NULL,'r3'),(373,60,4849,'r4'),(374,20,4862,'r5'),(375,10,4875,'r6'),(376,10,NULL,'r7'),(377,20,NULL,'r8'),(378,10,4914,'r9'),(379,10,4927,'r10'),(380,60,1000,'r11'),(381,20,NULL,'r12'),(382,NULL,4966,'r13'),(383,10,2000,'r14'),(384,20,2000,'r15'),(385,20,5005,'r16'),(386,NULL,1000,'r17'),(387,10,2000,'r18'),(388,20,NULL,'r19'),(389,10,5057,'r20'),(390,10,2000,'r21'),(391,10,5083,'r22'),(392,10,2000,'r23'),(393,NULL,NULL,'r24'),(394,20,5122,'r25'),(395,NULL,5135,'r26'),(396,20,5148,'r27'),(397,20,5161,'r28'),(398,20,2000,'r29'),(399,10,NULL,'r30'),(400,60,NULL,'r31'),(401,20,1000,'r32'),(402,20,NULL,'r33'),(403,NULL,2000,'r34'),(404,NULL,5252,'r35'),(405,30,NULL,'r36'),(406,60,5278,'r37'),(407,30,2000,'r38'),(408,20,5304,'r39'),(409,20,5317,'r40'),(410,10,NULL,'r0'),(411,NULL,5343,'r1'),(412,60,2000,'r2'),(413,NULL,1000,'r3'),(414,20,NULL,'r4'),(415,10,5395,'r5'),(416,10,5408,'r6'),(417,10,2000,'r7'),(418,60,2000,'r8'),(419,10,5447,'r9'),(420,20,NULL,'r10'),(421,60,1000,'r11'),(422,30,5486,'r12'),(423,30,5499,'r13'),(424,10,1000,'r14'),(425,20,NULL,'r15'),(426,20,1000,'r16'),(427,60,1000,'r17'),(428,10,5564,'r18'),(429,20,NULL,'r19'),(430,10,1000,'r20'),(431,30,1000,'r21'),(432,20,NULL,'r22'),(433,30,5629,'r23'),(434,20,2000,'r24'),(435,60,2000,'r25'),(436,20,2000,'r26'),(437,10,5681,'r27'),(438,20,5694,'r28'),(439,20,2000,'r29'),(440,NULL,NULL,'r30'),(441,NULL,NULL,'r31'),(442,10,5746,'r32'),(443,20,2000,'r33'),(444,10,5772,'r34'),(445,30,NULL,'r35'),(446,60,NULL,'r36'),(447,NULL,NULL,'r37'),(448,20,5824,'r38'),(449,NULL,1000,'r39'),(450,20,5850,'r40'),(451,30,NULL,'r0'),(452,20,1000,'r1'),(453,60,NULL,'r2'),(454,10,1000,'r3'),(455,NULL,NULL,'r4'),(456,20,NULL,'r5'),(457,20,2000,'r6'),(458,20,2000,'r7'),(459,10,1000,'r8'),(460,20,5980,'r9'),(461,10,NULL,'r10'),(462,20,1000,'r11'),(463,20,NULL,'r12'),(464,60,2000,'r13'),(465,20,2000,'r14'),(466,30,NULL,'r15'),(467,20,NULL,'r16'),(468,NULL,NULL,'r17'),(469,30,1000,'r18'),(470,NULL,1000,'r19'),(471,20,2000,'r20'),(472,60,NULL,'r21'),(473,10,6149,'r22'),(474,10,NULL,'r23'),(475,10,6175,'r24'),(476,30,NULL,'r25'),(477,60,NULL,'r26'),(478,10,2000,'r27'),(479,NULL,1000,'r28'),(480,30,2000,'r29'),(481,10,6253,'r30'),(482,20,6266,'r31'),(483,10,2000,'r32'),(484,10,1000,'r33'),(485,20,6305,'r34'),(486,10,1000,'r35'),(487,20,2000,'r36'),(488,60,6344,'r37'),(489,30,6357,'r38'),(490,20,6370,'r39'),(491,10,6383,'r40'),(492,60,NULL,'r0'),(493,10,NULL,'r1'),(494,20,6422,'r2'),(495,NULL,2000,'r3'),(496,30,1000,'r4'),(497,30,NULL,'r5'),(498,10,6474,'r6'),(499,20,2000,'r7'),(500,10,6500,'r8');
ANALYZE TABLE ta, tb;

SET SESSION tidb_enable_index_merge_join = ON;
-- run the following query ~10 times on unchanged data:
SELECT ta.c0 FROM ta WHERE ta.c0 IN (SELECT c0 FROM tb) GROUP BY ta.c0 ORDER BY ta.c0;
```

**Expected result:** `(10, 20, 30)` — stable across runs (ta.c0 values 10/20/30 all
appear in tb.c0). This is what the default configuration (`tidb_enable_index_merge_join=OFF`,
the v8.5 default) returns, 10/10 runs.

**Actual result:** With `tidb_enable_index_merge_join = ON`, the same query returns a
**random subset** across identical re-runs, e.g. from one session of 10 runs:
`(20,30), (10,30), (20,30), (20,30), (30), (30), (10,20,30), (20,30), (10,20,30), (30)`
— rows are silently dropped, ~80-90% of runs are wrong. No error is raised.

**Key observations (triage):**

1. `EXPLAIN` is byte-identical between correct and wrong runs — the plan is
deterministic; the defect is in execution:
```
└─IndexMergeJoin_31 inner join, inner:HashAgg_28,
outer key:ta.c0, inner key:tb.c0 (Concurrency:5)
├─TableReader(Build): Selection not(isnull(ta.c0)) over TableFullScan ta
└─HashAgg(Probe) over IndexRangeScan tb, range decided by eq(tb.c0, ta.c0)
```
2. `EXPLAIN ANALYZE` on two runs with the same plan shows the IndexMergeJoin
actual output row count flapping (1 vs 3) with fixed input (8 rows), no
concurrent writes.
3. Trigger surface (measured):
- affected: `IN (subquery)` shapes that get the join+count rewrite
(`tidb_opt_insubq_to_join_and_agg`), with or without an outer derived table;
- not affected: `EXISTS` / `NOT IN ... IS NOT NULL` (Apply/anti-join paths);
`tidb_opt_insubq_to_join_and_agg=OFF` + IMJ ON; IMJ OFF (default).
- cost threshold: the IndexMergeJoin operator is only selected when the inner
side is ≳500 rows with an index on the correlation column (≤250 rows did
not select it and did not exhibit the issue).
4. Cross-check: MySQL 8.x-compatible semantics — MySQL 9.7.2 on identical data
returns `(10,20,30)` 6/6 runs.

**Significance:** silent wrong results + nondeterminism, gated behind the new
IndexMergeJoin physical operator (re-introduced v8.4+, default OFF in v8.5).
Any workload or test harness that enables this flag will get randomly wrong
query results without any error signal.

**Environment:** single-node TiDB (docker), no concurrent load, TiUP-free
minimal setup. Reproduced across separate sessions/connections and on a second
freshly initialized instance.

Contributor guide

Open the contributing guide

Research direction

Start by running the reproducible SQL with tidb_enable_index_merge_join enabled and compare repeated results with EXPLAIN ANALYZE. Trace the IndexMergeJoin execution path and the IN-subquery join-and-aggregation rewrite, then add a regression test that repeats the query and verifies the stable (10,20,30) result without concurrent writes.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, sql
Domain
backend, database
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.