Add internal EXPLAIN ANALYZE metric level
- Dominant language
- Rust
- Stars
- 9.3k
- Forks
- 2.4k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 344
Description
### Is your feature request related to a problem or challenge?
Some execution metrics are useful for low-level operator debugging but are too noisy or high-cardinality for the default `EXPLAIN ANALYZE` output. Today the `summary` and `dev` levels do not provide a clean place for these internal development metrics.
### Describe the solution you would like
Add an `internal` `datafusion.explain.analyze_level` that includes `summary`, `dev`, and low-level internal metrics. Operators can use this level for targeted debugging metrics while keeping the default `dev` output concise and avoiding unnecessary metric registration overhead.
### Describe alternatives you have considered
Add operator-specific flags or environment variables for granular metrics. That makes metrics harder to discover and does not compose well across operators.
### Additional context
`RepartitionExec` granular timings are the first intended use case for this level.
Contributor guide
Research direction
Start by tracing how datafusion.explain.analyze_level is parsed and how EXPLAIN ANALYZE selects metrics, then inspect RepartitionExec as the stated first use case. Done means an internal level includes summary, dev, and low-level internal metrics while dev remains concise and avoids unnecessary metric registration overhead.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- backend, observability
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 50/100