TiDB returns incorrect JOIN results when explicit COLLATE is applied to a string expression
- Dominant language
- Go
- Stars
- 40.5k
- Forks
- 6.2k
- PR merge metrics
- PR metrics pending
Description
## Bug Report
### 1. Minimal reproduce step
Environment: TiDB v8.5.8, single-node TiUP playground using TiKV storage. The tables below contain no indexes and one row each.
Run the following SQL in a fresh TiDB test database:
```sql
SELECT tidb_version();
CREATE DATABASE IF NOT EXISTS tidb_collate_join_repro;
USE tidb_collate_join_repro;
DROP TABLE IF EXISTS x, y;
CREATE TABLE x (
c VARCHAR(10) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin
);
CREATE TABLE y (
c VARCHAR(10) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci
);
INSERT INTO x VALUES ('a');
INSERT INTO y VALUES ('A');
SELECT COUNT(*) AS result
FROM x JOIN y
ON CONVERT(x.c USING utf8mb4) COLLATE utf8mb4_unicode_ci = y.c;
```
The version output from the tested TiDB instance was:
```text
Release Version: v8.5.8
Edition: Community
Git Branch: HEAD
GoVersion: go1.25.12
Race Enabled: false
Check Table Before Drop: false
Store: tikv
```
### 2. What did you expect to see?
The query should return `1`.
The explicit `COLLATE utf8mb4_unicode_ci` is case-insensitive, so the values `'a'` and `'A'` should compare equal. MySQL 8.0 and the other tested MySQL-compatible databases also return `1` for this query.
### 3. What did you see instead
TiDB v8.5.8 returns `0`, which incorrectly omits the matching row from the JOIN.
The same TiDB instance returns `1` for the following controls:
```sql
SELECT COUNT(*) AS result
FROM x JOIN y
ON x.c COLLATE utf8mb4_unicode_ci = y.c;
```
```sql
SELECT COUNT(*) AS result
FROM x JOIN y
ON CONVERT(x.c USING utf8mb4) COLLATE utf8mb4_unicode_ci
= y.c COLLATE utf8mb4_unicode_ci;
```
```sql
SELECT COUNT(*) AS result
FROM x
WHERE CONVERT(c USING utf8mb4) COLLATE utf8mb4_unicode_ci = 'A';
```
All three controls return `1` on TiDB v8.5.8.
The failure is specific to using a string expression with an explicit collation as a JOIN comparison operand. Follow-up checks found the same pattern with several expressions:
| JOIN expression on the left | MySQL 8.4.7 | TiDB v8.5.8 |
| --- | ---: | ---: |
| `x.c COLLATE utf8mb4_unicode_ci` | 1 | 1 |
| `CONVERT(x.c USING utf8mb4) COLLATE utf8mb4_unicode_ci` | 1 | 0 |
| `CONCAT(x.c, '') COLLATE utf8mb4_unicode_ci` | 1 | 0 |
| `SUBSTRING(x.c, 1) COLLATE utf8mb4_unicode_ci` | 1 | 0 |
| `IFNULL(x.c, '') COLLATE utf8mb4_unicode_ci` | 1 | 0 |
| `LOWER(x.c) COLLATE utf8mb4_unicode_ci` | 1 | 0 |
The issue is reproducible without indexes or a large data set. `EXPLAIN` uses a HashJoin in the minimal case. Forcing another join algorithm with TiDB optimizer hints did not change the incorrect result in a follow-up check.
Cross-database comparison for the minimal query:
| Database | Result |
| --- | ---: |
| MySQL 8.0.46 | 1 |
| MySQL 8.4.7 | 1 |
| MariaDB 12.3.2 | 1 |
| Percona Server 8.4.11-11 | 1 |
| GreatSQL 8.4.4-5 | 1 |
| TiDB v8.5.8 | 0 |
This appears to be a collation propagation or comparison-key issue in JOIN execution for non-column string expressions. I have not confirmed the exact internal root cause.
### 4. What is your TiDB version?
```text
Release Version: v8.5.8
Edition: Community
Git Branch: HEAD
GoVersion: go1.25.12
Race Enabled: false
Check Table Before Drop: false
Store: tikv
```
Contributor guide
Research direction
Start by running the minimal SQL reproduction and its control queries, then inspect the HashJoin path and how collation is propagated for non-column string expressions. The work is done when the JOIN returns 1 for the explicit COLLATE cases, without regressing the controls or changing the result across join algorithms.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, sql
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100