apache / apache/datafusion

Improve speed of benchmark verification tests

Open
#4,032 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
PR https://github.com/apache/arrow-datafusion/pull/4015 added a new CI workflow to verify that the benchmark queries produce the correct results. This prevents regressions so is important, but it takes a long time to run.

**Describe the solution you'd like**
I think the ideal solution would be to just run on more powerful hardware. I would also like be able to be able to run the benchmarks in CI against larger datasets. I would be willing to provide self-hosted hardware if there is a simple and secure way of doing this. I am looking into some ideas.

**Describe alternatives you've considered**
None

**Additional context**
None

Contributor guide

Open the contributing guide

Research direction

The relevant entry point is the benchmark-verification CI workflow introduced by PR #4015; start there and identify which benchmark steps dominate runtime. Compare faster runners with larger datasets and assess a secure self-hosted option; done means CI still verifies correct results while meeting the agreed runtime and dataset goals.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions
Domain
ci-cd, performance, testing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.