Inconsistent `BIT_AND` Result Due to Zero-Date to `BIGINT` Conversion Mismatch Between TiKV and TiDB Root
- Dominant language
- Go
- Stars
- 40.5k
- Forks
- 6.2k
- PR merge metrics
- PR metrics pending
Description
## Bug Report
Please answer these questions before submitting your issue. Thanks!
### 1. Minimal reproduce step (Required)
```sql
DROP DATABASE IF EXISTS repro_tidb614_db14;
CREATE DATABASE repro_tidb614_db14 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin;
USE repro_tidb614_db14;
CREATE TABLE src (
id BIGINT NOT NULL AUTO_INCREMENT PRIMARY KEY,
c0 BOOLEAN NULL
);
INSERT INTO src(c0) VALUES (FALSE), (TRUE);
CREATE TABLE l (
id BIGINT NOT NULL PRIMARY KEY
);
CREATE TABLE r (
id BIGINT NOT NULL PRIMARY KEY,
c0 BOOLEAN NULL
);
INSERT INTO l SELECT id FROM src;
INSERT INTO r SELECT id, c0 FROM src;
-- Row‑by‑row conversion results
SELECT
id,
c0,
CAST(c0 AS DATE) AS c0_date,
CAST(CAST(c0 AS DATE) AS DATETIME) AS c0_dt
FROM src
ORDER BY id;
-- Single‑table source query (returns 0)
SELECT
BIT_AND(
CAST(
CASE 0.964765709243282
WHEN id THEN id
ELSE CAST(c0 AS DATE)
END AS DATETIME
)
) AS source_val
FROM src
WHERE '2023-05-28';
-- Split/join query (returns 18446744073709551615 = UINT64_MAX)
SELECT
BIT_AND(
CAST(
CASE 0.964765709243282
WHEN v.id THEN v.id
ELSE CAST(v.c0 AS DATE)
END AS DATETIME
)
) AS split_val
FROM (
SELECT id, r.c0 AS c0
FROM l JOIN r USING(id)
) v
WHERE '2023-05-28';
```
### 2. What did you expect to see? (Required)
According to the row‑by‑row conversion:
```
+----+------+------------+---------------------+
| id | c0 | c0_date | c0_dt |
+----+------+------------+---------------------+
| 1 | 0 | 0000-00-00 | 0000-00-00 00:00:00 |
| 2 | 1 | NULL | NULL |
+----+------+------------+---------------------+
```
- For `id=1` (`c0=0`), `CAST(c0 AS DATE)` gives `'0000-00-00'`, and the outer `CAST(... AS DATETIME)` yields `'0000-00-00 00:00:00'`. The outermost implicit cast to `BIGINT` (done by `BIT_AND`) should treat this as `0`.
- For `id=2` (`c0=1`), the conversion yields `NULL` and is ignored by `BIT_AND`.
Therefore the `BIT_AND` result should be `0` for both the single‑table and the join queries.
### 3. What did you see instead (Required)
- `source_val` = `0` (appears correct)
- `split_val` = `18446744073709551615` (`UINT64_MAX`, the identity value of `BIT_AND` when all inputs are `NULL`)
The join query incorrectly returns `UINT64_MAX`, indicating that the `DATETIME` value `'0000-00-00 00:00:00'` was treated as `NULL` during the final cast to `BIGINT`.
### 4. What is your TiDB version? (Required)
I tested such a case in TiDB-v8.5.6. and TiDB-v9.0.0. Maybe it is an issue that exists in all the versions.
## 5. Execution Plan Comparison
**Single-table (correct)**
```
Projection_7
└─Sort_9
└─HashAgg_17 [root]
└─TableReader_18 [root]
└─HashAgg_11 [cop[tikv]] ← ⚠️ BIT_AND pushed down to TiKV
└─TableFullScan_16 [cop[tikv]] table:src
```
TiKV-side `operator info`:
```
funcs:bit_and(
cast(cast(cast(cast(src.c0, date BINARY), var_string(10)), datetime BINARY), bigint BINARY)
)->Column#6
```
**JOIN derived table (wrong)**
```
StreamAgg_15 [root] ← BIT_AND stays at root
└─Projection_60 [root]
└─IndexJoin_46 [root]
├─TableReader_37 [root] table:l
└─TableReader_39 [root] table:r
```
`StreamAgg_15` operator info:
```
funcs:bit_and(Column#7)->Column#6
```
`Projection_60` operator info (cast chain evaluated at TiDB root):
```
cast(cast(cast(cast(r.c0, date BINARY), var_string(10)), datetime BINARY), bigint BINARY)->Column#7
```
## 6. Root Cause
This is a classic mismatch between the TiDB root executor and the TiKV coprocessor when handling the conversion `DATETIME '0000-00-00 00:00:00'` → `BIGINT`.
In the single‑table query, the aggregation is pushed down to TiKV. TiKV's coprocessor successfully casts the zero date to `BIGINT` (presumably `0`), and the `BIT_AND` correctly returns `0`.
In the join query, the subquery and join force the `BIT_AND` to run in the TiDB root layer. When TiDB's SQL executor encounters `DATETIME '0000-00-00 00:00:00'`, it may — depending on SQL mode or strict mode — treat this as an invalid or zero date and convert it to `NULL` during the final cast to `BIGINT`. As a result, both input values to `BIT_AND` become `NULL`, and the function returns its identity value `18446744073709551615` (`UINT64_MAX`).
The root problem is that TiDB's own expression evaluator in the root layer does not consistently handle the zero date when casting to `BIGINT`, while TiKV's coprocessor produces a non‑`NULL` result (`0`). This inconsistency causes different results depending on whether the expression can be pushed to TiKV.
**Related observation:** When the `CASE` expression is removed and a direct `BIT_AND(CAST(CAST(c0 AS DATE) AS DATETIME))` is used, both plans produce `0`, because the simpler expression avoids the problematic path.
Contributor guide
Assessment
This issue has not been assessed yet.