pingcap / pingcap/tidb

Inconsistent `BIT_AND` Result Due to Zero-Date to `BIGINT` Conversion Mismatch Between TiKV and TiDB Root

Open
#69,190 2 comments 0 reactions 0 assignees View on GitHub
contribution may-affects-7.5 may-affects-8.1 may-affects-8.5 severity/critical sig/execution type/bug
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Bug Report

Please answer these questions before submitting your issue. Thanks!

### 1. Minimal reproduce step (Required)

```sql
DROP DATABASE IF EXISTS repro_tidb614_db14;
CREATE DATABASE repro_tidb614_db14 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin;
USE repro_tidb614_db14;

CREATE TABLE src (
id BIGINT NOT NULL AUTO_INCREMENT PRIMARY KEY,
c0 BOOLEAN NULL
);

INSERT INTO src(c0) VALUES (FALSE), (TRUE);

CREATE TABLE l (
id BIGINT NOT NULL PRIMARY KEY
);

CREATE TABLE r (
id BIGINT NOT NULL PRIMARY KEY,
c0 BOOLEAN NULL
);

INSERT INTO l SELECT id FROM src;
INSERT INTO r SELECT id, c0 FROM src;

-- Row‑by‑row conversion results
SELECT
id,
c0,
CAST(c0 AS DATE) AS c0_date,
CAST(CAST(c0 AS DATE) AS DATETIME) AS c0_dt
FROM src
ORDER BY id;

-- Single‑table source query (returns 0)
SELECT
BIT_AND(
CAST(
CASE 0.964765709243282
WHEN id THEN id
ELSE CAST(c0 AS DATE)
END AS DATETIME
)
) AS source_val
FROM src
WHERE '2023-05-28';

-- Split/join query (returns 18446744073709551615 = UINT64_MAX)
SELECT
BIT_AND(
CAST(
CASE 0.964765709243282
WHEN v.id THEN v.id
ELSE CAST(v.c0 AS DATE)
END AS DATETIME
)
) AS split_val
FROM (
SELECT id, r.c0 AS c0
FROM l JOIN r USING(id)
) v
WHERE '2023-05-28';
```

### 2. What did you expect to see? (Required)

According to the row‑by‑row conversion:

```
+----+------+------------+---------------------+
| id | c0 | c0_date | c0_dt |
+----+------+------------+---------------------+
| 1 | 0 | 0000-00-00 | 0000-00-00 00:00:00 |
| 2 | 1 | NULL | NULL |
+----+------+------------+---------------------+
```

- For `id=1` (`c0=0`), `CAST(c0 AS DATE)` gives `'0000-00-00'`, and the outer `CAST(... AS DATETIME)` yields `'0000-00-00 00:00:00'`. The outermost implicit cast to `BIGINT` (done by `BIT_AND`) should treat this as `0`.
- For `id=2` (`c0=1`), the conversion yields `NULL` and is ignored by `BIT_AND`.

Therefore the `BIT_AND` result should be `0` for both the single‑table and the join queries.

### 3. What did you see instead (Required)

- `source_val` = `0` (appears correct)
- `split_val` = `18446744073709551615` (`UINT64_MAX`, the identity value of `BIT_AND` when all inputs are `NULL`)

The join query incorrectly returns `UINT64_MAX`, indicating that the `DATETIME` value `'0000-00-00 00:00:00'` was treated as `NULL` during the final cast to `BIGINT`.

### 4. What is your TiDB version? (Required)

I tested such a case in TiDB-v8.5.6. and TiDB-v9.0.0. Maybe it is an issue that exists in all the versions.

## 5. Execution Plan Comparison

**Single-table (correct)**

```
Projection_7
└─Sort_9
└─HashAgg_17 [root]
└─TableReader_18 [root]
└─HashAgg_11 [cop[tikv]] ← ⚠️ BIT_AND pushed down to TiKV
└─TableFullScan_16 [cop[tikv]] table:src
```

TiKV-side `operator info`:

```
funcs:bit_and(
cast(cast(cast(cast(src.c0, date BINARY), var_string(10)), datetime BINARY), bigint BINARY)
)->Column#6
```

**JOIN derived table (wrong)**

```
StreamAgg_15 [root] ← BIT_AND stays at root
└─Projection_60 [root]
└─IndexJoin_46 [root]
├─TableReader_37 [root] table:l
└─TableReader_39 [root] table:r
```

`StreamAgg_15` operator info:

```
funcs:bit_and(Column#7)->Column#6
```

`Projection_60` operator info (cast chain evaluated at TiDB root):

```
cast(cast(cast(cast(r.c0, date BINARY), var_string(10)), datetime BINARY), bigint BINARY)->Column#7
```

## 6. Root Cause

This is a classic mismatch between the TiDB root executor and the TiKV coprocessor when handling the conversion `DATETIME '0000-00-00 00:00:00'` → `BIGINT`.

In the single‑table query, the aggregation is pushed down to TiKV. TiKV's coprocessor successfully casts the zero date to `BIGINT` (presumably `0`), and the `BIT_AND` correctly returns `0`.

In the join query, the subquery and join force the `BIT_AND` to run in the TiDB root layer. When TiDB's SQL executor encounters `DATETIME '0000-00-00 00:00:00'`, it may — depending on SQL mode or strict mode — treat this as an invalid or zero date and convert it to `NULL` during the final cast to `BIGINT`. As a result, both input values to `BIT_AND` become `NULL`, and the function returns its identity value `18446744073709551615` (`UINT64_MAX`).

The root problem is that TiDB's own expression evaluator in the root layer does not consistently handle the zero date when casting to `BIGINT`, while TiKV's coprocessor produces a non‑`NULL` result (`0`). This inconsistency causes different results depending on whether the expression can be pushed to TiKV.

**Related observation:** When the `CASE` expression is removed and a direct `BIT_AND(CAST(CAST(c0 AS DATE) AS DATETIME))` is used, both plans produce `0`, because the simpler expression avoids the problematic path.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.