Question: can we collect statistic if the table doesn't collect statistic during creating
- Dominant language
- Rust
- Stars
- 9.3k
- Forks
- 2.4k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 344
Description
I noticed that after creating a table, enabling `collect_stat`, and scanning the table, we won't get the statistics for the table.
We must enable `collect_stat` before creating a table, then when scan the table, we can get the statistics.
```sql
set datafusion.execution.collect_statistics = true;
CREATE EXTERNAL TABLE t2 (id INT not null, date DATE) STORED AS PARQUET LOCATION './data/' PARTITIONED BY (date) WITH ORDER (id ASC);
INSERT INTO t2 VALUES (4, '2025-03-01'), (3, '2025-3-02'), (2, '2025-03-03'), (1, '2025-03-04');
SELECT * FROM t2 ORDER BY id ASC; // we'll get the statistics
```
```sql
CREATE EXTERNAL TABLE t2 (id INT not null, date DATE) STORED AS PARQUET LOCATION './data/' PARTITIONED BY (date) WITH ORDER (id ASC);
INSERT INTO t2 VALUES (4, '2025-03-01'), (3, '2025-3-02'), (2, '2025-03-03'), (1, '2025-03-04');
set datafusion.execution.collect_statistics = true;
SELECT * FROM t2 ORDER BY id ASC; // we won't get the statistics
```
Do we have ways to get statistics if we don't enable `collect_statistics` before creating the table?
Contributor guide
Research direction
Reproduce the two SQL sequences using datafusion.execution.collect_statistics, comparing table creation before and after enabling the setting. Trace the table scan and statistics collection behavior to determine whether late enabling is supported; done means documenting the supported behavior or defining and testing a change.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust, sql
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100