Complex UNION/EXCEPT query returns binary representation instead of CHAR
- Dominant language
- Go
- Stars
- 40.5k
- Forks
- 6.2k
- PR merge metrics
- PR metrics pending
Description
## Bug Report
Please answer these questions before submitting your issue. Thanks!
### 1. Minimal reproduce step (Required)
```sql
USE test;
DROP DATABASE IF EXISTS database33;
CREATE DATABASE database33;
USE database33;
CREATE TABLE t0(c0 CHAR NOT NULL);
CREATE TABLE t1 LIKE t0;
INSERT INTO t0(c0) VALUES ('-');
INSERT INTO t1(c0) VALUES ('7');
-- Simple SELECT
SELECT DISTINCT t1.c0, t0.c0 FROM t1 INNER JOIN t0 ON (CASE ((-727907032)^((CASE true WHEN 'l3hbn8Up9' THEN t1.c0 ELSE -2001834354 END ))) WHEN ((t0.c0) IS NULL) THEN ((((t1.c0) IS NULL))REGEXP(t0.c0)) ELSE t1.c0 END );
-- Complex SELECT
(SELECT DISTINCT t1.c0, t0.c0 FROM t1 LEFT JOIN t0 ON (CASE ((-727907032)^((CASE true WHEN 'l3hbn8Up9' THEN t1.c0 ELSE -2001834354 END ))) WHEN ((t0.c0) IS NULL) THEN ((((t1.c0) IS NULL))REGEXP(t0.c0)) ELSE t1.c0 END )
UNION
SELECT DISTINCT t1.c0, t0.c0 FROM t1 RIGHT JOIN t0 ON (CASE ((-727907032)^((CASE true WHEN 'l3hbn8Up9' THEN t1.c0 ELSE -2001834354 END ))) WHEN ((t0.c0) IS NULL) THEN ((((t1.c0) IS NULL))REGEXP(t0.c0)) ELSE t1.c0 END )
)
EXCEPT
(
(
SELECT DISTINCT t1.c0, t0.c0 FROM t1 LEFT JOIN t0 ON (CASE ((-727907032)^((CASE true WHEN 'l3hbn8Up9' THEN t1.c0 ELSE -2001834354 END ))) WHEN ((t0.c0) IS NULL) THEN ((((t1.c0) IS NULL))REGEXP(t0.c0)) ELSE t1.c0 END )
EXCEPT
SELECT DISTINCT t1.c0, t0.c0 FROM t1 RIGHT JOIN t0 ON (CASE ((-727907032)^((CASE true WHEN 'l3hbn8Up9' THEN t1.c0 ELSE -2001834354 END ))) WHEN ((t0.c0) IS NULL) THEN ((((t1.c0) IS NULL))REGEXP(t0.c0)) ELSE t1.c0 END )
)
UNION
(
SELECT DISTINCT t1.c0, t0.c0 FROM t1 RIGHT JOIN t0 ON (CASE ((-727907032)^((CASE true WHEN 'l3hbn8Up9' THEN t1.c0 ELSE -2001834354 END ))) WHEN ((t0.c0) IS NULL) THEN ((((t1.c0) IS NULL))REGEXP(t0.c0)) ELSE t1.c0 END )
EXCEPT
SELECT DISTINCT t1.c0, t0.c0 FROM t1 LEFT JOIN t0 ON (CASE ((-727907032)^((CASE true WHEN 'l3hbn8Up9' THEN t1.c0 ELSE -2001834354 END ))) WHEN ((t0.c0) IS NULL) THEN ((((t1.c0) IS NULL))REGEXP(t0.c0)) ELSE t1.c0 END )
)
);
```
**In fact, the second complex SELECT query provided above is logically equivalent to the first SELECT query and can be further simplified into the following SELECT statement. However, this simplified version does not trigger the bug. We suspect that the issue lies in TiDB's internal optimizer, which performs an incorrect type conversion on the CHAR column when executing such complex equivalent queries, resulting in the output being in BINARY rather than the original CHAR.**
```sql
(SELECT DISTINCT t1.c0, t0.c0 FROM t1 LEFT JOIN t0 ON true
UNION
SELECT DISTINCT t1.c0, t0.c0 FROM t1 RIGHT JOIN t0 ON true
)
EXCEPT
(
(
SELECT DISTINCT t1.c0, t0.c0 FROM t1 LEFT JOIN t0 ON true
EXCEPT
SELECT DISTINCT t1.c0, t0.c0 FROM t1 RIGHT JOIN t0 ON true
)
UNION
(
SELECT DISTINCT t1.c0, t0.c0 FROM t1 RIGHT JOIN t0 ON true
EXCEPT
SELECT DISTINCT t1.c0, t0.c0 FROM t1 LEFT JOIN t0 ON true
)
);
```
**Moreover, we observed an interesting phenomenon: executing either of the two complex SQL statements individually does not produce incorrect results. However, when keywords like UNION or EXCEPT are used, the type conversion issue appears. We believe this bug is also related to the optimizer's handling of operations involving UNION/EXCEPT.**
```sql
SELECT DISTINCT t1.c0, t0.c0 FROM t1 LEFT JOIN t0 ON (CASE ((-727907032)^((CASE true WHEN 'l3hbn8Up9' THEN t1.c0 ELSE -2001834354 END ))) WHEN ((t0.c0) IS NULL) THEN (false REGEXP(t0.c0)) ELSE t1.c0 END)
SELECT DISTINCT t1.c0, t0.c0 FROM t1 RIGHT JOIN t0 ON (CASE ((-727907032)^((CASE true WHEN 'l3hbn8Up9' THEN t1.c0 ELSE -2001834354 END ))) WHEN ((t0.c0) IS NULL) THEN (false REGEXP(t0.c0)) ELSE t1.c0 END)
```
### 2. What did you expect to see? (Required)
I tested it in MySQL with the following results:
```sql
mysql> -- Simple SELECT
mysql> SELECT DISTINCT t1.c0, t0.c0 FROM t1 INNER JOIN t0 ON (CASE ((-727907032)^((CASE true WHEN 'l3hbn8Up9' THEN t1.c0 ELSE -2001834354 END ))) WHEN ((t0.c0) IS NULL) THEN ((((t1.c0) IS NULL))REGEXP(t0.c0)) ELSE t1.c0 END );
+----+----+
| c0 | c0 |
+----+----+
| 7 | - |
+----+----+
1 row in set, 1 warning (0.00 sec)
mysql>
mysql> -- Complex SELECT
mysql> (SELECT DISTINCT t1.c0, t0.c0 FROM t1 LEFT JOIN t0 ON (CASE ((-727907032)^((CASE true WHEN 'l3hbn8Up9' THEN t1.c0 ELSE -2001834354 END ))) WHEN ((t0.c0) IS NULL) THEN ((((t1.c0) IS NULL))REGEXP(t0.c0)) ELSE t1.c0 END )
-> UNION
-> SELECT DISTINCT t1.c0, t0.c0 FROM t1 RIGHT JOIN t0 ON (CASE ((-727907032)^((CASE true WHEN 'l3hbn8Up9' THEN t1.c0 ELSE -2001834354 END ))) WHEN ((t0.c0) IS NULL) THEN ((((t1.c0) IS NULL))REGEXP(t0.c0)) ELSE t1.c0 END )
-> )
-> EXCEPT
-> (
-> (
-> SELECT DISTINCT t1.c0, t0.c0 FROM t1 LEFT JOIN t0 ON (CASE ((-727907032)^((CASE true WHEN 'l3hbn8Up9' THEN t1.c0 ELSE -2001834354 END ))) WHEN ((t0.c0) IS NULL) THEN ((((t1.c0) IS NULL))REGEXP(t0.c0)) ELSE t1.c0 END )
-> EXCEPT
-> SELECT DISTINCT t1.c0, t0.c0 FROM t1 RIGHT JOIN t0 ON (CASE ((-727907032)^((CASE true WHEN 'l3hbn8Up9' THEN t1.c0 ELSE -2001834354 END ))) WHEN ((t0.c0) IS NULL) THEN ((((t1.c0) IS NULL))REGEXP(t0.c0)) ELSE t1.c0 END )
-> )
-> UNION
-> (
-> SELECT DISTINCT t1.c0, t0.c0 FROM t1 RIGHT JOIN t0 ON (CASE ((-727907032)^((CASE true WHEN 'l3hbn8Up9' THEN t1.c0 ELSE -2001834354 END ))) WHEN ((t0.c0) IS NULL) THEN ((((t1.c0) IS NULL))REGEXP(t0.c0)) ELSE t1.c0 END )
-> EXCEPT
-> SELECT DISTINCT t1.c0, t0.c0 FROM t1 LEFT JOIN t0 ON (CASE ((-727907032)^((CASE true WHEN 'l3hbn8Up9' THEN t1.c0 ELSE -2001834354 END ))) WHEN ((t0.c0) IS NULL) THEN ((((t1.c0) IS NULL))REGEXP(t0.c0)) ELSE t1.c0 END )
-> )
-> );
+------+------+
| c0 | c0 |
+------+------+
| 7 | - |
+------+------+
1 row in set, 6 warnings (0.00 sec)
```
### 3. What did you see instead (Required)
```sql
mysql> -- Simple SELECT
mysql> SELECT DISTINCT t1.c0, t0.c0 FROM t1 INNER JOIN t0 ON (CASE ((-727907032)^((CASE true WHEN 'l3hbn8Up9' THEN t1.c0 ELSE -2001834354 END ))) WHEN ((t0.c0) IS NULL) THEN ((((t1.c0) IS NULL))REGEXP(t0.c0)) ELSE t1.c0 END );
+--------+----+
| c0 | c0 |
+--------+----+
| 0x37 | - |
+--------+----+
1 row in set, 1 warning (0.01 sec)
mysql>
mysql> -- Complex SELECT
mysql> (SELECT DISTINCT t1.c0, t0.c0 FROM t1 LEFT JOIN t0 ON (CASE ((-727907032)^((CASE true WHEN 'l3hbn8Up9' THEN t1.c0 ELSE -2001834354 END ))) WHEN ((t0.c0) IS NULL) THEN ((((t1.c0) IS NULL))REGEXP(t0.c0)) ELSE t1.c0 END )
-> UNION
-> SELECT DISTINCT t1.c0, t0.c0 FROM t1 RIGHT JOIN t0 ON (CASE ((-727907032)^((CASE true WHEN 'l3hbn8Up9' THEN t1.c0 ELSE -2001834354 END ))) WHEN ((t0.c0) IS NULL) THEN ((((t1.c0) IS NULL))REGEXP(t0.c0)) ELSE t1.c0 END )
-> )
-> EXCEPT
-> (
-> (
-> SELECT DISTINCT t1.c0, t0.c0 FROM t1 LEFT JOIN t0 ON (CASE ((-727907032)^((CASE true WHEN 'l3hbn8Up9' THEN t1.c0 ELSE -2001834354 END ))) WHEN ((t0.c0) IS NULL) THEN ((((t1.c0) IS NULL))REGEXP(t0.c0)) ELSE t1.c0 END )
-> EXCEPT
-> SELECT DISTINCT t1.c0, t0.c0 FROM t1 RIGHT JOIN t0 ON (CASE ((-727907032)^((CASE true WHEN 'l3hbn8Up9' THEN t1.c0 ELSE -2001834354 END ))) WHEN ((t0.c0) IS NULL) THEN ((((t1.c0) IS NULL))REGEXP(t0.c0)) ELSE t1.c0 END )
-> )
-> UNION
-> (
-> SELECT DISTINCT t1.c0, t0.c0 FROM t1 RIGHT JOIN t0 ON (CASE ((-727907032)^((CASE true WHEN 'l3hbn8Up9' THEN t1.c0 ELSE -2001834354 END ))) WHEN ((t0.c0) IS NULL) THEN ((((t1.c0) IS NULL))REGEXP(t0.c0)) ELSE t1.c0 END )
-> EXCEPT
-> SELECT DISTINCT t1.c0, t0.c0 FROM t1 LEFT JOIN t0 ON (CASE ((-727907032)^((CASE true WHEN 'l3hbn8Up9' THEN t1.c0 ELSE -2001834354 END ))) WHEN ((t0.c0) IS NULL) THEN ((((t1.c0) IS NULL))REGEXP(t0.c0)) ELSE t1.c0 END )
-> )
-> );
+------------+------+
| c0 | c0 |
+------------+------+
| 0x37000000 | - |
+------------+------+
1 row in set, 6 warnings (0.01 sec)
```
### 4. What is your TiDB version? (Required)
```sql
mysql> select version();
+--------------------------------------------+
| version() |
+--------------------------------------------+
| 8.0.11-TiDB-v9.0.0-beta.2.pre-193-g4852c06 |
+--------------------------------------------+
1 row in set (0.00 sec)
```
Contributor guide
Assessment
This issue has not been assessed yet.