pingcap / pingcap/tidb

Equivalent COUNT predicates can return different values inside one SELECT under concurrent upserts

Open
#69,538 2 comments 0 reactions 0 assignees View on GitHub
contribution may-affects-7.5 may-affects-8.1 may-affects-8.5 severity/critical sig/transaction type/bug
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Bug Report

Please answer these questions before submitting your issue. Thanks!

### 1. Minimal reproduce step (Required)

Create a simple table:

```sql
DROP TABLE IF EXISTS compare_kv;

CREATE TABLE compare_kv (
id BIGINT PRIMARY KEY,
v BIGINT NOT NULL,
note VARCHAR(64) NOT NULL
);
```

Run several writer sessions. Each writer repeatedly upserts deterministic values into the same table:

```sql
INSERT INTO compare_kv (id, v, note)
VALUES (?, ?, 'tidb')
ON DUPLICATE KEY UPDATE
v = ?,
note = 'tidb';
```

The value of `v` is deterministic for each `id`:

```text
v = (id * 7919) % 100000
```

For example, one writer iteration is:

```sql
INSERT INTO compare_kv (id, v, note)
VALUES (146, (146 * 7919) % 100000, 'tidb')
ON DUPLICATE KEY UPDATE
v = (146 * 7919) % 100000,
note = 'tidb';
```

At the same time, run reader sessions which repeatedly execute the following single SQL statement:

```sql
SELECT
(SELECT COUNT(*) FROM compare_kv WHERE v BETWEEN 10 AND 90000) AS c_between,
(SELECT COUNT(*) FROM compare_kv WHERE v >= 10 AND v <= 90000) AS c_range;
```

The two predicates are logically equivalent:

```sql
v BETWEEN 10 AND 90000
```

and

```sql
v >= 10 AND v <= 90000
```

The read statement contains both subqueries in the same `SELECT` list, so both counts should be evaluated against the same statement snapshot.

A shell-style reproduction outline is:

```bash
# Session 1..N: writers
# Keep inserting/updating ids in a loop.
# For each id, compute v = (id * 7919) % 100000 and upsert it.

# Session N+1..M: readers
# Keep executing the equivalent-count comparison query shown above.
```

The mismatch appeared while writes were committing concurrently with the comparison query.

### 2. What did you expect to see? (Required)

The two counts should always be identical:

```text
c_between = c_range
```

For example:

```text
+-----------+---------+
| c_between | c_range |
+-----------+---------+
| 80 | 80 |
+-----------+---------+
```

Even with concurrent writes, a single SQL statement should not return two different results for equivalent predicates over the same table.

### 3. What did you see instead (Required)

The query sometimes returned different values in the two columns.

Observed result examples include:

```text
c_between | c_range
----------+--------
14 | 15
48 | 49
80 | 81
104 | 105
115 | 116
82 | 83
```

Representative operation sequence around one mismatch:

```text
writer: INSERT/UPSERT id = 146 completed
reader: SELECT equivalent COUNT predicates
result: c_between = 80, c_range = 81
```

Another representative sequence:

```text
writer: INSERT/UPSERT id = 191 completed
reader: SELECT equivalent COUNT predicates
result: c_between = 104, c_range = 105
```

This is concerning because the SQL statement itself proves the inconsistency:

```sql
SELECT
(SELECT COUNT(*) FROM compare_kv WHERE v BETWEEN 10 AND 90000),
(SELECT COUNT(*) FROM compare_kv WHERE v >= 10 AND v <= 90000);
```

The two subqueries should match the same set of rows. A mismatch suggests that the two scalar subqueries may not always use the same statement snapshot, or that equivalent predicate forms may be taking inconsistent execution paths under concurrent writes.

### 4. What is your TiDB version? (Required)

```text
SELECT tidb_version();

8.0.11-TiDB-v8.5.6
```

Contributor guide

Open the contributing guide

Research direction

Start by running the provided compare_kv schema, concurrent upsert workload, and single-SELECT reproduction on TiDB 8.0.11-TiDB-v8.5.6. Trace whether both scalar subqueries use the same statement snapshot or execution path, then verify that c_between and c_range remain identical during concurrent writes.

Written by the indexing model from the issue text.

Assessment

Tech stack
sql
Domain
databases, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.