apache / apache/datafusion

Add per-operator benchmarks

Open
#15,214 1 comment 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

### Is your feature request related to a problem or challenge?

Datafusion have benchmarks which would determine general performance of datafusion as a whole system, but when it comes to testing and analyzing each operator in datafusion, there is no major development so far.

We currently can potentially monitor performance metrics of DataFusion, but if a single operator could be responsible for slow performance then finding that operator could be a difficult task. This is something I discussed a bit in slack and in https://github.com/apache/datafusion/issues/5504 .

### Describe the solution you'd like

Like the implementation of benchmark `SortPreservingMerge` : https://github.com/apache/datafusion/blob/main/datafusion/core/benches/spm.rs , we could build a bunch of benchmarks for different operators.
We could start with a few like `Filter`, joins like `HashJoin`, `Projection` and so on.

There is a lot which could be added with passage of time, but so far I would like to listen to suggestions from the community about what they think as well.

### Describe alternatives you've considered

_No response_

### Additional context

I will be continuously updating this issue as more information is gained. This would be a rather large project in itself, maybe we could branch out more such tickets to gradually add more benchmarks for each operators.

Contributor guide

Open the contributing guide

Research direction

Start with datafusion/core/benches/spm.rs to understand the existing SortPreservingMerge benchmark structure, then review the Filter, HashJoin, and Projection operators named in the issue. Define a focused initial set of per-operator benchmarks and confirm that each can measure its own performance independently; done means the selected benchmarks are added and runnable.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
data-engineering, performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.