apache / apache/datafusion-comet
Use end to end microbenchmarks to prevent performance regressions between releases
- Dominant language
- Scala
- Stars
- 1.3k
- Forks
- 373
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 198
Description
### What is the problem the feature request solves?
WE already have many microbenchmarks that test end to end performance of Spark vs Comet for operators and expressions, but they need some work (tracked in https://github.com/apache/datafusion-comet/issues/5363).
I would like to add documentation on running these benchmarks and checking the results into the repo using a specific EC2 instance type so that anybody can reproduce them.
Perhaps we can eventually automate this to run on a regular schedule during development cycles between releases.
### Describe the potential solution
_No response_
### Additional context
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.