Add a Benchmark for Planning Extension Nodes
- Dominant language
- Rust
- Stars
- 9.3k
- Forks
- 2.4k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 344
Description
### Is your feature request related to a problem or challenge?
Currently, as far as I am concerned, there are no benchmarks that focus on the planning of custom query plans. This issue tracks adding such a benchmark.
### Describe the solution you'd like
I think we should have an `extension_planner` (or similar) file with the following things:
- An `ExtensionPlanner` that implements a custom version of some built-in operator (e.g., a special node for doing a common projection in a system)
- A benchmark that plans the custom logical plan
- A benchmark that achieves the same goal using a built-in operator
- Quite likely more (e.g., planning of extension types etc.)
This should allow us to compare the overhead of planning an extension logical plan versus built-in logical plans. Ideally, we should try to get this down to "zero", such that custom plans do not suffer from any drawbacks.
### Describe alternatives you've considered
- Let downstream projects find the performance regressions ;)
### Additional context
This issue was raised in https://github.com/apache/datafusion/pull/23157
I am quite busy till end of July, but I'll try to add such a benchmark afterwards if no one beats me to it.
Contributor guide
Assessment
This issue has not been assessed yet.