apache / apache/datafusion

Add `auto_explain` mode

Open
#19,215 5 comments 4 reactions 0 assignees View on GitHub
enhancement
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

### Is your feature request related to a problem or challenge?

It would be useful to collect plans from queries automatically, without needing to explicitly add `EXPLAIN ANALYZE`. This would ensure that the original application would not need to change, as the queries would still return data as normal, while the plans would be printed to `stdout`/`stderr`.

Some existing systems already support this feature, like [`auto_explain`](https://www.postgresql.org/docs/18/auto-explain.html) in Postgres and [`eqp`](https://www.sqlite.org/eqp.html) in SQLite.

### Describe the solution you'd like

Some `datafusion.execution.auto_explain` config that would enable this feature.

I already created a small proof of concept and this feature would be relatively easy to implement. We just need to wrap execution plans in an `AnalyzeExec`. This `AnalyzeExec` would also need a flag to know when it's in the `auto_explain` mode, in which case it would print the plans and return the input batches.

I would be happy to create a PR with this.

### Describe alternatives you've considered

Adding `EXPLAIN ANALYZE` manually, but this changes the application.

### Additional context

_No response_

Contributor guide

Open the contributing guide

Research direction

Start by reviewing AnalyzeExec and the execution-plan path, then locate where datafusion.execution configuration is defined. Done means an auto_explain setting collects and prints plans while returning query data normally, without requiring application queries to change.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust, sql
Domain
data-engineering
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.