pingcap / pingcap/tidb

utf8mb4_bin (NO PAD) is honored for literal comparisons but violated for column comparisons, WHERE equality, GROUP BY and DISTINCT — '' and ' ' are different values yet judged equal everywhere except in literal form

Open
#70,753 2 comments 0 reactions 1 assignee Claimed by @YangKeao View on GitHub
contribution may-affects-25.10 may-affects-26.3 may-affects-7.5 may-affects-8.1 may-affects-8.5 severity/critical sig/sql-infra type/bug
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Bug Report

Please answer these questions before submitting your issue. Thanks!

### 1. Minimal reproduce step (Required)

```sql
DROP DATABASE IF EXISTS repro_pad;
CREATE DATABASE repro_pad;
USE repro_pad;
CREATE TABLE t(c TEXT);
INSERT INTO t VALUES (''), (' '), ('x'), ('a'), ('a ');
-- table collation is utf8mb4_bin (NO PAD), verified with SHOW CREATE TABLE below

-- Base query (HAVING predicate over group aggregates — trivially TRUE for every group):
SELECT CONCAT('[', c, ']') FROM t GROUP BY c HAVING ((MAX(c) <= 642884535) OR (MAX(c) >= MIN(c)));
-- []
-- [x]
-- [a]
-- 3 rows <-- WRONG (expected 5: the HAVING is trivially TRUE — MAX(c) >= MIN(c)
-- holds for every group — so all FIVE NO-PAD-distinct values must each
-- form a group; instead '' merged with ' ' and 'a' merged with 'a ')

-- Rewritten query (same HAVING predicate relocated into the derived-table projection):
SELECT ref0 FROM (SELECT CONCAT('[', c, ']') AS ref0, ((MAX(c) <= 642884535) OR (MAX(c) >= MIN(c))) AS ref1 FROM t GROUP BY c) s WHERE ref1;
-- []
-- [x]
-- [a]
-- 3 rows <-- WRONG too (identically collapsed; on larger data the two shapes render
-- DIFFERENT representatives '' vs ' ' of the same collapsed group — that
-- content mismatch is what the my oracle registered)

-- (a) Literal comparison honors the NO PAD collation:
SELECT ('' = ' ');
-- 0 <-- correct: '' and ' ' are DIFFERENT values

-- (b) The SAME comparison through a column uses PAD semantics:
SELECT CONCAT('[', c, ']') AS val, (c = ' ') AS eq_space, (c = '') AS eq_empty FROM t;
-- [] | 1 | 1 <-- WRONG: the '' row is judged equal to BOTH ' ' and '' (PAD behavior)
-- [ ] | 1 | 1 <-- WRONG: the ' ' row is judged equal to both too
-- [x] | 0 | 0
-- [a] | 0 | 0
-- [a ] | 0 | 0

-- (c) So WHERE equality picks up both rows:
SELECT CONCAT('[', c, ']') FROM t WHERE c = ' ';
-- []
-- [ ] <-- WRONG (expected only [ ] — the literal comparison in (a) says '' != ' ')

-- (d) GROUP BY and DISTINCT merge the NO-PAD-distinct values:
SELECT CONCAT('[', c, ']') FROM t GROUP BY c;
-- []
-- [x]
-- [a]
-- 3 rows <-- WRONG (expected 5 groups)
```

Semantics: under a NO PAD collation `''` and `' '` are different values — the literal
comparison in (a) proves the engine knows this. Yet the identical comparison against a
column (b), the WHERE filter (c), and the GROUP BY/DISTINCT key comparison (d) all use
PAD SPACE semantics and merge them. The collation is applied inconsistently across
evaluation paths.

Verified trigger conditions (single-variable experiments):

- Table collation is `utf8mb4_bin` (NO PAD) — verified via `SHOW CREATE TABLE`.
- HEX probe confirms the stored bytes are distinct (`''` empty vs `' '` = 0x20).
- `CONVERT('' USING utf8mb4) = CONVERT(' ' USING utf8mb4)` also returns 1 (PAD), while
bare literal `'' = ' '` returns 0 — the inconsistency follows the value's coercibility
path, not the operator.
- All char-family types reproduce (`TEXT`, `VARCHAR`, `CHAR`); no index/stats/view needed.
- Standard NoREC cannot express the trigger: its row-level template
(`COUNT(*) WHERE p` vs `SUM(CASE WHEN p)`) never involves GROUP BY or column-vs-literal
coercibility differences — both measured forms agree (5 = 5, correct counts).
- Not a duplicate: #68053 is a binary-charset leak in REVERSE rendering (different shape);
no tracked issue covers the literal-vs-column equality contradiction under NO PAD.
### 2. What did you expect to see? (Required)
Under a NO PAD collation, `''` and `' '` are different values **everywhere**: in literal
comparison, in column comparison, in WHERE equality, and as GROUP BY/DISTINCT keys
(5 groups, not 3).

### 3. What did you see instead (Required)
Only literal comparison honors NO PAD. Column comparison, WHERE equality, GROUP BY and
DISTINCT all silently switch to PAD SPACE semantics: `''` is judged equal to `' '`,
`WHERE c = ' '` returns both rows, and the five values collapse into three groups.
Query results silently depend on which evaluation path a comparison happens to take.

### 4. What is your TiDB version? (Required)

```
Release Version: v9.0.0-beta.2.pre-2174-g65ac2fad58
Edition: Community
Git Commit Hash: 65ac2fad582510b92d3c4b79999e48217c989737
Git Branch: HEAD
UTC Build Time: 2026-08-28 20:22:53
GoVersion: go1.25.12
Race Enabled: false
Check Table Before Drop: false
Store: unistore
Kernel Type: Classic
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.