pingcap / pingcap/tidb

statistics: zero TIMESTAMP bounds log terror errors during plan-time cardinality estimation

Open
#69,440 1 comment 0 reactions 0 assignees View on GitHub
affects-8.5 component/statistics severity/moderate sig/planner type/bug
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Bug Report

### 1. Minimal reproduce step (Required)

`TIMESTAMP` values that cannot be converted to Go `time.Time`, such as `0000-00-00 00:00:00`, can still enter statistics histograms when SQL mode allows zero dates. During plan-time cardinality estimation, TiDB converts histogram/range bounds to scalar values through:

```text
statistics.convertDatumToScalar
-> types.Time.Sub
-> types.Time.GoTime
-> terror.Log
```

`Time.Sub` logs the `GoTime` error but does not return it to the optimizer, so the SQL statement continues while TiDB emits `ERROR`-level stack traces.

This is the broader root cause behind #68527. That issue is one concrete production trigger through empty-partition placeholder histograms. The same core issue is also reachable on a non-partitioned table when a zero `TIMESTAMP` value becomes a Stats V2 histogram bucket bound.

Non-partitioned table reproduction:

```sql
SET @@session.sql_mode = '';
SET @@session.time_zone = '+00:00';
SET @@session.tidb_analyze_version = 2;

DROP DATABASE IF EXISTS zts;
CREATE DATABASE zts;
USE zts;

CREATE TABLE t (
id INT PRIMARY KEY,
ts TIMESTAMP NULL,
KEY idx_ts(ts)
);

INSERT INTO t VALUES
(1, '0000-00-00 00:00:00'),
(2, '1970-01-01 00:00:01'),
(3, '1970-01-01 00:00:02'),
(4, '1970-01-01 00:00:03'),
(5, '1970-01-01 00:00:04');

ANALYZE TABLE t COLUMNS ts WITH 0 TOPN, 2 BUCKETS;

EXPLAIN FORMAT = 'brief'
SELECT * FROM t WHERE ts < '1970-01-01 00:00:01';

EXPLAIN FORMAT = 'brief'
SELECT * FROM t WHERE ts > '0000-00-00 00:00:00';

SELECT @@warning_count;
SHOW WARNINGS;
```

Notes:

- `sql_mode = ''` is needed because the default strict mode rejects zero `TIMESTAMP` values.
- `WITH 0 TOPN` makes the zero `TIMESTAMP` stay in the histogram bucket bounds instead of being absorbed by TopN on this small dataset.
- `SHOW STATS_BUCKETS` shows the column histogram lower bound as `0000-00-00 00:00:00`.
- `EXPLAIN` returns normally and `@@warning_count` remains `0`, but TiDB logs an `ERROR` stack.

Expected stack shape:

```text
[ERROR] [terror.go] ["encountered error"] [error="[types:1292]Incorrect time value: '{0 0 0 0 0 0 0}'"]
github.com/pingcap/tidb/pkg/parser/terror.Log
github.com/pingcap/tidb/pkg/types.(*Time).Sub
github.com/pingcap/tidb/pkg/statistics.convertDatumToScalar
github.com/pingcap/tidb/pkg/statistics.(*Histogram).OutOfRangeRowCount
github.com/pingcap/tidb/pkg/planner/cardinality...
```

Relevant code paths on current `master`:

- `pkg/statistics/scalar.go`: `convertDatumToScalar` handles `KindMysqlTime` by calling `valueTime.Sub(...)` and has no error return channel.
- `pkg/types/time.go`: `Time.Sub` handles `TIMESTAMP` by calling `GoTime(ctx.Location())`; the error is only passed to `terror.Log(errors.Trace(err))`.
- `pkg/types/core_time.go`: `GoTime` returns `ErrWrongValue` when the Go time round-trip does not match the original core time fields.
- `pkg/statistics/histogram.go`: `OutOfRangeRowCount` converts range bounds and histogram bounds to scalar values.
- `pkg/planner/cardinality/row_count_column.go` and `pkg/planner/cardinality/row_count_index.go`: column/index row count estimation calls `OutOfRangeRowCount`.

### 2. What did you expect to see? (Required)

Plan-time statistics/cardinality estimation should not emit `ERROR`-level `terror.Log` stacks for known invalid/zero `TIMESTAMP` values held in statistics or range bounds.

At minimum, this path should either:

- avoid `GoTime` for scalarizing `TIMESTAMP` values that may be zero/invalid, or
- return/fallback inside the statistics estimation layer without logging an `ERROR`, or
- explicitly recognize placeholder/zero histogram bounds and skip the invalid scalar conversion.

### 3. What did you see instead (Required)

The statement succeeds and the optimizer continues, but TiDB logs `ERROR` stacks from `terror.Log`.

The error is not surfaced to the SQL client:

- `EXPLAIN` returns normally.
- `@@warning_count` remains `0`.
- The observable impact is log noise or, under high-QPS workloads like #68527, log flooding large enough to fill disks.

The same mechanism can be triggered by:

- a normal non-partitioned table containing a zero `TIMESTAMP` in a Stats V2 histogram bound, and
- the empty-partition placeholder histogram scenario reported in #68527.

`DATE`/`DATETIME` have related silent estimation-risk edges because they also go through statistics scalarization, but the `terror.Log -> GoTime` stack is specific to the `TIMESTAMP` branch in `Time.Sub`.

### 4. What is your TiDB version? (Required)

Confirmed affected by code inspection and reproduction on:

- `v8.5.6`, via #68527.
- current `master` / `origin/master` as of 2026-06-24: `b53834a9d76500c8b1f2dd32ea105c7c8644d658`.

The non-partitioned reproduction above was also verified on a local master checkout (`e3706840585ecf5b85a31f8c309aa82e63537d6c`) with a temporary unistore TiDB server.

Related issues:

- #68527: same core root cause, concrete empty-partition placeholder-histogram production trigger.
- #56480 / #52615: earlier fix for a different call site; it does not cover the `OutOfRangeRowCount -> Time.Sub` path.

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the zero-TIMESTAMP case with the SQL statements in the issue, then read pkg/statistics/scalar.go and pkg/types/time.go around convertDatumToScalar and Time.Sub. Trace the calls through pkg/statistics/histogram.go and the planner cardinality files; done means EXPLAIN still succeeds without warnings while plan-time estimation no longer emits ERROR-level stacks.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.