Enabling `datafusion.explain.show_statistics` by default
- Dominant language
- Rust
- Stars
- 9.3k
- Forks
- 2.4k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 344
Description
### Is your feature request related to a problem or challenge?
Follow up based on https://github.com/apache/arrow-datafusion/pull/7459#pullrequestreview-1616135946
Displaying statistics in executions plans as a default behaviour is nice to have feature. The only challenge here is to enable/implement it without significantly growing the burden of tests maintenance, as some of them, in different parts of codebase, rely on text representation on execution plan, and it doesn't seem convenient to fix random tests across multiple packages in case of e.g. statistics calculation related changes.
### Describe the solution you'd like
One possible solution might be to explicitly disabling `show_statistics` setting everywhere in tests, except for cases requiring statistics in output.
Another one could be to have a common normalizer (like we already have in [sqllogictests](https://github.com/apache/arrow-datafusion/blob/4d4451294629940d340160cdd06be273139728b4/datafusion/sqllogictest/src/engines/datafusion_engine/normalize.rs#L32) and in [core/tests/sql](https://github.com/apache/arrow-datafusion/blob/4d4451294629940d340160cdd06be273139728b4/datafusion/core/tests/sql/mod.rs#L806) for datafiles path replacement) which could be accessible across all tests and would allow to replace statistics with empty or remove it from plan text. In this case the `datafusion.explain.show_statistics` setting is not needed anymore.
### Describe alternatives you've considered
_No response_
### Additional context
_No response_
Contributor guide
Research direction
Start with the existing normalizers in sqllogictest/src/engines/datafusion_engine/normalize.rs and core/tests/sql/mod.rs, then review tests that assert execution-plan text. Decide how show_statistics can be enabled by default while keeping plan assertions stable, and verify that statistics-specific output remains covered without widespread brittle updates.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- data, testing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100