Equivalent COUNT predicates can return different values inside one SELECT under concurrent upserts
- Dominant language
- Go
- Stars
- 40.5k
- Forks
- 6.2k
- PR merge metrics
- PR metrics pending
Description
## Bug Report
Please answer these questions before submitting your issue. Thanks!
### 1. Minimal reproduce step (Required)
Create a simple table:
```sql
DROP TABLE IF EXISTS compare_kv;
CREATE TABLE compare_kv (
id BIGINT PRIMARY KEY,
v BIGINT NOT NULL,
note VARCHAR(64) NOT NULL
);
```
Run several writer sessions. Each writer repeatedly upserts deterministic values into the same table:
```sql
INSERT INTO compare_kv (id, v, note)
VALUES (?, ?, 'tidb')
ON DUPLICATE KEY UPDATE
v = ?,
note = 'tidb';
```
The value of `v` is deterministic for each `id`:
```text
v = (id * 7919) % 100000
```
For example, one writer iteration is:
```sql
INSERT INTO compare_kv (id, v, note)
VALUES (146, (146 * 7919) % 100000, 'tidb')
ON DUPLICATE KEY UPDATE
v = (146 * 7919) % 100000,
note = 'tidb';
```
At the same time, run reader sessions which repeatedly execute the following single SQL statement:
```sql
SELECT
(SELECT COUNT(*) FROM compare_kv WHERE v BETWEEN 10 AND 90000) AS c_between,
(SELECT COUNT(*) FROM compare_kv WHERE v >= 10 AND v <= 90000) AS c_range;
```
The two predicates are logically equivalent:
```sql
v BETWEEN 10 AND 90000
```
and
```sql
v >= 10 AND v <= 90000
```
The read statement contains both subqueries in the same `SELECT` list, so both counts should be evaluated against the same statement snapshot.
A shell-style reproduction outline is:
```bash
# Session 1..N: writers
# Keep inserting/updating ids in a loop.
# For each id, compute v = (id * 7919) % 100000 and upsert it.
# Session N+1..M: readers
# Keep executing the equivalent-count comparison query shown above.
```
The mismatch appeared while writes were committing concurrently with the comparison query.
### 2. What did you expect to see? (Required)
The two counts should always be identical:
```text
c_between = c_range
```
For example:
```text
+-----------+---------+
| c_between | c_range |
+-----------+---------+
| 80 | 80 |
+-----------+---------+
```
Even with concurrent writes, a single SQL statement should not return two different results for equivalent predicates over the same table.
### 3. What did you see instead (Required)
The query sometimes returned different values in the two columns.
Observed result examples include:
```text
c_between | c_range
----------+--------
14 | 15
48 | 49
80 | 81
104 | 105
115 | 116
82 | 83
```
Representative operation sequence around one mismatch:
```text
writer: INSERT/UPSERT id = 146 completed
reader: SELECT equivalent COUNT predicates
result: c_between = 80, c_range = 81
```
Another representative sequence:
```text
writer: INSERT/UPSERT id = 191 completed
reader: SELECT equivalent COUNT predicates
result: c_between = 104, c_range = 105
```
This is concerning because the SQL statement itself proves the inconsistency:
```sql
SELECT
(SELECT COUNT(*) FROM compare_kv WHERE v BETWEEN 10 AND 90000),
(SELECT COUNT(*) FROM compare_kv WHERE v >= 10 AND v <= 90000);
```
The two subqueries should match the same set of rows. A mismatch suggests that the two scalar subqueries may not always use the same statement snapshot, or that equivalent predicate forms may be taking inconsistent execution paths under concurrent writes.
### 4. What is your TiDB version? (Required)
```text
SELECT tidb_version();
8.0.11-TiDB-v8.5.6
```
Contributor guide
Research direction
Start by running the provided compare_kv schema, concurrent upsert workload, and single-SELECT reproduction on TiDB 8.0.11-TiDB-v8.5.6. Trace whether both scalar subqueries use the same statement snapshot or execution path, then verify that c_between and c_range remain identical during concurrent writes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- sql
- Domain
- databases, distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100