pingcap / pingcap/tidb

REORGANIZE PARTITION can silently drop a row when EXCHANGE PARTITION ... WITHOUT VALIDATION produced duplicate _tidb_rowid

Open
#69,648 0 comments 0 reactions 1 assignee Claimed by @bb7133 View on GitHub
affects-9.0 component/ddl found-by-ai may-affects-7.5 may-affects-8.1 may-affects-8.5 severity/critical sig/sql-infra type/bug
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Bug Report

### 1. Minimal reproduce step (Required)

`REORGANIZE PARTITION` on a **non-clustered** partitioned table can silently drop a row when two old partitions contain rows that share the same `_tidb_rowid` **and** identical row bytes. Such duplicates can be produced by `EXCHANGE PARTITION ... WITHOUT VALIDATION`.

```sql
SET @@tidb_enable_exchange_partition = 1;

CREATE TABLE t(a INT, b INT)
PARTITION BY RANGE (b) (
PARTITION p0 VALUES LESS THAN (10),
PARTITION p1 VALUES LESS THAN (20),
PARTITION pmax VALUES LESS THAN (MAXVALUE)
);
INSERT INTO t VALUES (1,1); -- lands in p0, _tidb_rowid = 1

-- a non-partitioned table holding a byte-identical row with the same _tidb_rowid = 1
CREATE TABLE tx LIKE t;
ALTER TABLE tx REMOVE PARTITIONING;
INSERT INTO tx VALUES (1,1); -- _tidb_rowid = 1

-- after the exchange, p1 holds a row byte-identical to p0's row, same _tidb_rowid
ALTER TABLE t EXCHANGE PARTITION p1 WITH TABLE tx WITHOUT VALIDATION;
SELECT COUNT(*) FROM t; -- 2 (correct)

-- merge p0 and p1
ALTER TABLE t REORGANIZE PARTITION p0,p1 INTO (PARTITION p01 VALUES LESS THAN (20));
SELECT COUNT(*) FROM t; -- 1 <-- one visible row is silently lost
```

### 2. What did you expect to see? (Required)

`REORGANIZE PARTITION` should preserve the visible row multiset. The rows in `p0` and `p1` are two real, independent rows, so `COUNT(*)` must remain `2` after the reorganize (as the guard cases below show TiDB already does whenever the rows differ in any byte or in their `_tidb_rowid`).

### 3. What did you see instead (Required)

`COUNT(*)` drops from `2` to `1`. One user-visible row is deleted by the DDL, with no error and no warning.

The loss is specific to the combination *duplicate `_tidb_rowid` **and** byte-identical row across the old partitions*. Guard cases that isolate the trigger (all verified on the version below):

| p0 row | p1 row (via exchange) | after `REORGANIZE PARTITION` |
| --- | --- | --- |
| `(1,1)`, rowid 1 | `(1,1)`, rowid 3 (same bytes, different rowid) | `COUNT(*)` 2 → 2 ✅ |
| `(1,1)`, rowid 1 | `(2,1)`, rowid 1 (same rowid, different bytes) | `COUNT(*)` 2 → 2 ✅ (rowid regenerated) |
| `(1,1)`, rowid 1 | `(1,1)`, rowid 1 (same rowid **and** same bytes) | `COUNT(*)` 2 → **1** ❌ |

### 4. What is your TiDB version? (Required)

```
Release Version: v9.0.0-beta.2.pre-1774-g81ec977cb8
Edition: Community
Git Commit Hash: 81ec977cb8bf97e0c9805dfc0be8ffcbd4b0bbeb
Git Branch: HEAD
UTC Build Time: 2026-05-28 03:35:30
GoVersion: go1.25.9
Store: tikv
```

Likely root cause — non-clustered reorg backfill treats byte-equality as row identity

In `pkg/ddl/partition.go`, `reorgPartitionWorker.BackfillData` deliberately handles duplicate `_tidb_rowid` produced by `EXCHANGE PARTITION`. It probes the target key with `kv.BatchGetValue` and then:

```go
if vals, ok := found[string(key)]; ok {
if len(vals) == len(prr.vals) && bytes.Equal(vals, prr.vals) {
// Already backfilled or double written earlier by concurrent DML
continue // <-- the row is skipped
}
// Not same row, due to earlier EXCHANGE PARTITION.
// Update the current read row by Remove it and Add it back (which will give it a new _tidb_rowid)
...
}
```

The `bytes.Equal` branch assumes an existing target key with identical bytes means the row was already written by this backfill or a concurrent double-write, so it skips it. That assumption does not hold when the two source rows come from **different old physical partitions** and happen to be byte-identical with the same handle — the second one is a distinct logical row but is dropped as a duplicate. The sibling branch already regenerates `_tidb_rowid` when the bytes differ; the same regeneration is needed for the byte-equal case when the row originates from a different source partition (e.g. carry the source physical table ID so a genuine duplicate can be told apart from a same-bytes collision).

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.