statistics: zero TIMESTAMP bounds log terror errors during plan-time cardinality estimation
- Dominant language
- Go
- Stars
- 40.5k
- Forks
- 6.2k
- PR merge metrics
- PR metrics pending
Description
## Bug Report
### 1. Minimal reproduce step (Required)
`TIMESTAMP` values that cannot be converted to Go `time.Time`, such as `0000-00-00 00:00:00`, can still enter statistics histograms when SQL mode allows zero dates. During plan-time cardinality estimation, TiDB converts histogram/range bounds to scalar values through:
```text
statistics.convertDatumToScalar
-> types.Time.Sub
-> types.Time.GoTime
-> terror.Log
```
`Time.Sub` logs the `GoTime` error but does not return it to the optimizer, so the SQL statement continues while TiDB emits `ERROR`-level stack traces.
This is the broader root cause behind #68527. That issue is one concrete production trigger through empty-partition placeholder histograms. The same core issue is also reachable on a non-partitioned table when a zero `TIMESTAMP` value becomes a Stats V2 histogram bucket bound.
Non-partitioned table reproduction:
```sql
SET @@session.sql_mode = '';
SET @@session.time_zone = '+00:00';
SET @@session.tidb_analyze_version = 2;
DROP DATABASE IF EXISTS zts;
CREATE DATABASE zts;
USE zts;
CREATE TABLE t (
id INT PRIMARY KEY,
ts TIMESTAMP NULL,
KEY idx_ts(ts)
);
INSERT INTO t VALUES
(1, '0000-00-00 00:00:00'),
(2, '1970-01-01 00:00:01'),
(3, '1970-01-01 00:00:02'),
(4, '1970-01-01 00:00:03'),
(5, '1970-01-01 00:00:04');
ANALYZE TABLE t COLUMNS ts WITH 0 TOPN, 2 BUCKETS;
EXPLAIN FORMAT = 'brief'
SELECT * FROM t WHERE ts < '1970-01-01 00:00:01';
EXPLAIN FORMAT = 'brief'
SELECT * FROM t WHERE ts > '0000-00-00 00:00:00';
SELECT @@warning_count;
SHOW WARNINGS;
```
Notes:
- `sql_mode = ''` is needed because the default strict mode rejects zero `TIMESTAMP` values.
- `WITH 0 TOPN` makes the zero `TIMESTAMP` stay in the histogram bucket bounds instead of being absorbed by TopN on this small dataset.
- `SHOW STATS_BUCKETS` shows the column histogram lower bound as `0000-00-00 00:00:00`.
- `EXPLAIN` returns normally and `@@warning_count` remains `0`, but TiDB logs an `ERROR` stack.
Expected stack shape:
```text
[ERROR] [terror.go] ["encountered error"] [error="[types:1292]Incorrect time value: '{0 0 0 0 0 0 0}'"]
github.com/pingcap/tidb/pkg/parser/terror.Log
github.com/pingcap/tidb/pkg/types.(*Time).Sub
github.com/pingcap/tidb/pkg/statistics.convertDatumToScalar
github.com/pingcap/tidb/pkg/statistics.(*Histogram).OutOfRangeRowCount
github.com/pingcap/tidb/pkg/planner/cardinality...
```
Relevant code paths on current `master`:
- `pkg/statistics/scalar.go`: `convertDatumToScalar` handles `KindMysqlTime` by calling `valueTime.Sub(...)` and has no error return channel.
- `pkg/types/time.go`: `Time.Sub` handles `TIMESTAMP` by calling `GoTime(ctx.Location())`; the error is only passed to `terror.Log(errors.Trace(err))`.
- `pkg/types/core_time.go`: `GoTime` returns `ErrWrongValue` when the Go time round-trip does not match the original core time fields.
- `pkg/statistics/histogram.go`: `OutOfRangeRowCount` converts range bounds and histogram bounds to scalar values.
- `pkg/planner/cardinality/row_count_column.go` and `pkg/planner/cardinality/row_count_index.go`: column/index row count estimation calls `OutOfRangeRowCount`.
### 2. What did you expect to see? (Required)
Plan-time statistics/cardinality estimation should not emit `ERROR`-level `terror.Log` stacks for known invalid/zero `TIMESTAMP` values held in statistics or range bounds.
At minimum, this path should either:
- avoid `GoTime` for scalarizing `TIMESTAMP` values that may be zero/invalid, or
- return/fallback inside the statistics estimation layer without logging an `ERROR`, or
- explicitly recognize placeholder/zero histogram bounds and skip the invalid scalar conversion.
### 3. What did you see instead (Required)
The statement succeeds and the optimizer continues, but TiDB logs `ERROR` stacks from `terror.Log`.
The error is not surfaced to the SQL client:
- `EXPLAIN` returns normally.
- `@@warning_count` remains `0`.
- The observable impact is log noise or, under high-QPS workloads like #68527, log flooding large enough to fill disks.
The same mechanism can be triggered by:
- a normal non-partitioned table containing a zero `TIMESTAMP` in a Stats V2 histogram bound, and
- the empty-partition placeholder histogram scenario reported in #68527.
`DATE`/`DATETIME` have related silent estimation-risk edges because they also go through statistics scalarization, but the `terror.Log -> GoTime` stack is specific to the `TIMESTAMP` branch in `Time.Sub`.
### 4. What is your TiDB version? (Required)
Confirmed affected by code inspection and reproduction on:
- `v8.5.6`, via #68527.
- current `master` / `origin/master` as of 2026-06-24: `b53834a9d76500c8b1f2dd32ea105c7c8644d658`.
The non-partitioned reproduction above was also verified on a local master checkout (`e3706840585ecf5b85a31f8c309aa82e63537d6c`) with a temporary unistore TiDB server.
Related issues:
- #68527: same core root cause, concrete empty-partition placeholder-histogram production trigger.
- #56480 / #52615: earlier fix for a different call site; it does not cover the `OutOfRangeRowCount -> Time.Sub` path.
Contributor guide
Research direction
Start by reproducing the zero-TIMESTAMP case with the SQL statements in the issue, then read pkg/statistics/scalar.go and pkg/types/time.go around convertDatumToScalar and Time.Sub. Trace the calls through pkg/statistics/histogram.go and the planner cardinality files; done means EXPLAIN still succeeds without warnings while plan-time estimation no longer emits ERROR-level stacks.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100