apache / apache/doris

[Bug] [Bug] BE fails with [E-3110] parse date fail when __internal_schema.column_statistics contains '0000-00-00'

Open
#66,486 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
15.9k
Forks
3.9k
Avg merge
2d 23h
Merged PRs (30d)
520

Description

### Search before asking

- [x] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues.

### Version

4.1.0

### What's Wrong?

BE fails to initialize storage reader and continuously reports errors when `__internal_schema.column_statistics` contains illegal date strings like `'0000-00-00 00:00:00'`.

When a business table has invalid date strings (e.g. imported from legacy MySQL or skipping strict checks), the auto-analyze task collects `'0000-00-00 00:00:00'` as `min`/`max` values and writes them into `__internal_schema.column_statistics`.

When any query (or CBO optimizer) attempts to read statistics from `__internal_schema.column_statistics`, the BE Storage Reader fails to parse the string during initialization and cancels the entire query execution context with error:
`[E-3110][INVALID_ARGUMENT] parse date or datetime fail, string: '0000-00-00 00:00:00' failed to initialize storage reader.`

This creates a dead-lock where `__internal_schema.column_statistics` becomes completely unreadable (`SELECT count(1) FROM __internal_schema.column_statistics` fails), and CBO statistics loading for dependent queries gets broken.

### What You Expected?

1. Sanitization on Write: Statistics collection (ANALYZE) should sanitize or convert invalid date/datetime strings (like '0000-00-00 00:00:00') to NULL or skip them before persisting into `__internal_schema.column_statistics`.
2. Graceful Degrade on Read: BE Storage Reader should handle or ignore unparsable date values during statistics deserialization gracefully, rather than throwing a fatal error [E-3110] and canceling the entire query.

### How to Reproduce?

_No response_

### Anything Else?

- Temporary Workarounds & Mitigations:
1. Disabled global auto analyze to stop recurring failure:
`SET GLOBAL enable_auto_analyze = false;`
2. Dropped and truncated the corrupted internal statistics table to recover read ability:
`DROP STATS __internal_schema.column_statistics;`
`TRUNCATE TABLE __internal_schema.column_statistics;`

- Impact:
Because of this bug, we are forced to keep `enable_auto_analyze = false`, which prevents the cluster from automatically gathering CBO statistics for all other normal tables.

- Stack Trace snippet from BE:
0# doris::BetaRowsetReader::_init_iterator()
1# doris::BetaRowsetReader::_init_iterator_once()
2# doris::Status doris::BetaRowsetReader::_next_batch(doris::Block*)
3# doris::BetaRowsetReader::next_batch(doris::Block*)
4# doris::VCollectIterator::Level0Iterator::refresh_current_row()
5# doris::VCollectIterator::Level0Iterator::ensure_first_row_ref()
6# doris::VCollectIterator::Level1Iterator::ensure_first_row_ref()
7# doris::VCollectIterator::build_heap()
8# doris::BlockReader::_init_collect_iter()
9# doris::BlockReader::init()
10# doris::OlapScanner::_open_impl()

### Are you willing to submit PR?

- [ ] Yes I am willing to submit a PR!

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)

Contributor guide

Open the contributing guide

Research direction

Start by tracing reads of __internal_schema.column_statistics through the BE storage-reader path named in the stack trace, especially BetaRowsetReader::_init_iterator and BlockReader::init. Reproduce the failure with the reported invalid date value if possible, then inspect the ANALYZE statistics-writing path. Done means invalid date values no longer make the statistics table unreadable or cancel dependent queries.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, sql
Domain
backend, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.