pingcap / pingcap/tidb

Erroneous `from_binary()` Wrapper on Join Build Side Reverses `BINARY(...)` Comparison Semantics, Causing Empty Result

Open
#69,354 2 comments 0 reactions 0 assignees View on GitHub
contribution may-affects-7.5 may-affects-8.1 may-affects-8.5 severity/critical sig/planner type/bug
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Bug Report

Please answer these questions before submitting your issue. Thanks!

### 1. Minimal Reproduce Step (Required)

```sql
DROP DATABASE IF EXISTS repro_tidb618_db0_min;
CREATE DATABASE repro_tidb618_db0_min;
USE repro_tidb618_db0_min;

SET SESSION sql_mode='ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION';
SET time_zone='+00:00';
SET tidb_opt_agg_push_down=1;

CREATE TABLE src(
vp_rowid BIGINT NOT NULL AUTO_INCREMENT PRIMARY KEY,
c0 BIGINT NULL
);

INSERT INTO src(c0) VALUES (-2100312475);

CREATE TABLE l(
vp_rowid BIGINT NOT NULL PRIMARY KEY
);

CREATE TABLE r(
vp_rowid BIGINT NOT NULL PRIMARY KEY,
c0 BIGINT NULL
);

INSERT INTO l SELECT vp_rowid FROM src;
INSERT INTO r SELECT vp_rowid, c0 FROM src;

-- Single‑table query (returns 1 row)
SELECT
NULL << 'aMI x*' AS a,
403837559 AS b,
BIN(c0) AS c
FROM src
WHERE BIN((BINARY(CAST(vp_rowid AS DECIMAL)) <= '|'))
ORDER BY 0.9328472781954409;

-- Split/join query (returns empty set)
SELECT
NULL << 'aMI x*' AS a,
403837559 AS b,
BIN(c0) AS c
FROM (
SELECT l.vp_rowid, r.c0
FROM l
JOIN (
SELECT vp_rowid, COUNT(*) AS vp_count
FROM r
GROUP BY vp_rowid
) rg ON l.vp_rowid = rg.vp_rowid
JOIN r ON r.vp_rowid = rg.vp_rowid
) v
WHERE BIN((BINARY(CAST(vp_rowid AS DECIMAL)) <= '|'))
ORDER BY 0.9328472781954409;
```

**Supplementary diagnostic query** (shows the semantic mismatch):

```sql
SELECT
vp_rowid,
CAST(vp_rowid AS DECIMAL) AS dec_v,
BINARY(CAST(vp_rowid AS DECIMAL)) AS bin_v,
HEX(BINARY(CAST(vp_rowid AS DECIMAL))) AS hex_bin,
(BINARY(CAST(vp_rowid AS DECIMAL)) <= '|') AS cmp_binary,
BIN((BINARY(CAST(vp_rowid AS DECIMAL)) <= '|')) AS pred_binary,
(FROM_BINARY(BINARY(CAST(vp_rowid AS DECIMAL))) <= '|') AS cmp_from_binary,
BIN((FROM_BINARY(BINARY(CAST(vp_rowid AS DECIMAL))) <= '|')) AS pred_from_binary
FROM src;
```

### 2. What Did You Expect to See? (Required)

Both queries should return the same result. For the single row in the base table (`vp_rowid=1`):

- `BINARY(CAST(1 AS DECIMAL))` returns the binary string `'1'`.
- `'1' <= '|'` is `TRUE` (ASCII `0x31 < 0x7C`), so `BIN(...)` yields `'1'` (non‑empty, truthy).
- The `WHERE` clause passes, and one row is returned.

Thus the expected result is `1 row` for both the single‑table and the join query.

### 3. What Did You See Instead? (Required)

- Single‑table query: returns `1 row` (correct).
- Split/join query: returns `Empty set` (incorrect).

### 4. What Is Your TiDB Version? (Required)

Version: TiDB‑v9.0.0

### 5. Execution Plan Differences

**Single‑table (correct)** — the predicate is used as‑is:

```
Selection_10 bin(le(cast(cast(src.vp_rowid, decimal(10,0) BINARY), binary(1)), "|"))
```

The comparison `BINARY(...) <= '|'` is preserved as a binary string comparison.

**Split/join (incorrect)** — predicate pushed to different sides, the Build side gets an extra `from_binary()`:

```
-- Build side (table r)
Selection_47 [cop[tikv]] bin(le(from_binary(cast(cast(r.vp_rowid, decimal(10,0) BINARY), binary(1))), "|"))

-- Probe side (table l)
Selection_44 [cop[tikv]] bin(le(cast(cast(l.vp_rowid, decimal(10,0) BINARY), binary(1)), "|"))
```

The predicate on the `r` side (Build) is incorrectly rewritten with `from_binary()`, changing the comparison semantics from binary string to numeric.

### 6. Root Cause

The expression `BINARY(CAST(vp_rowid AS DECIMAL)) <= '|'` performs a binary string comparison: the cast result is a binary string `'1'`, and the comparison `'1' <= '|'` is true because `0x31 < 0x7C`.

When the optimizer pushes this predicate down to both sides of the join, it mistakenly treats `BINARY(...)` and `FROM_BINARY(BINARY(...))` as equivalent. On the Build side (table `r`) it rewrites the predicate as `FROM_BINARY(BINARY(CAST(r.vp_rowid AS DECIMAL))) <= '|'`. `FROM_BINARY` reverses the effect of `BINARY`, returning the original numeric value (`1`). The comparison `1 <= '|'` then becomes a numeric comparison: `'|'` is interpreted as `0`, so `1 <= 0` is `FALSE`. The Probe side retains the original binary string comparison, but the Build side filters out all rows, leading to an empty join result.

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the single-table and split/join queries on TiDB v9.0.0, then inspect the optimizer's predicate pushdown for the join build side and compare the shown execution plans. Done means the build-side predicate no longer changes BINARY comparison semantics and both queries return one row.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, sql
Domain
databases, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.