Add per-operator benchmarks
- Dominant language
- Rust
- Stars
- 9.3k
- Forks
- 2.4k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 344
Description
### Is your feature request related to a problem or challenge?
Datafusion have benchmarks which would determine general performance of datafusion as a whole system, but when it comes to testing and analyzing each operator in datafusion, there is no major development so far.
We currently can potentially monitor performance metrics of DataFusion, but if a single operator could be responsible for slow performance then finding that operator could be a difficult task. This is something I discussed a bit in slack and in https://github.com/apache/datafusion/issues/5504 .
### Describe the solution you'd like
Like the implementation of benchmark `SortPreservingMerge` : https://github.com/apache/datafusion/blob/main/datafusion/core/benches/spm.rs , we could build a bunch of benchmarks for different operators.
We could start with a few like `Filter`, joins like `HashJoin`, `Projection` and so on.
There is a lot which could be added with passage of time, but so far I would like to listen to suggestions from the community about what they think as well.
### Describe alternatives you've considered
_No response_
### Additional context
I will be continuously updating this issue as more information is gained. This would be a rather large project in itself, maybe we could branch out more such tickets to gradually add more benchmarks for each operators.
Contributor guide
Research direction
Start with datafusion/core/benches/spm.rs to understand the existing SortPreservingMerge benchmark structure, then review the Filter, HashJoin, and Projection operators named in the issue. Define a focused initial set of per-operator benchmarks and confirm that each can measure its own performance independently; done means the selected benchmarks are added and runnable.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- data-engineering, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100