Improve speed of benchmark verification tests
- Dominant language
- Rust
- Stars
- 9.3k
- Forks
- 2.4k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 344
Description
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
PR https://github.com/apache/arrow-datafusion/pull/4015 added a new CI workflow to verify that the benchmark queries produce the correct results. This prevents regressions so is important, but it takes a long time to run.
**Describe the solution you'd like**
I think the ideal solution would be to just run on more powerful hardware. I would also like be able to be able to run the benchmarks in CI against larger datasets. I would be willing to provide self-hosted hardware if there is a simple and secure way of doing this. I am looking into some ideas.
**Describe alternatives you've considered**
None
**Additional context**
None
Contributor guide
Research direction
The relevant entry point is the benchmark-verification CI workflow introduced by PR #4015; start there and identify which benchmark steps dominate runtime. Compare faster runners with larger datasets and assess a secure self-hosted option; done means CI still verifies correct results while meeting the agreed runtime and dataset goals.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github-actions
- Domain
- ci-cd, performance, testing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100