Consolidate benchmarking options/ENV settings
- Dominant language
- Rust
- Stars
- 9.3k
- Forks
- 2.4k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 344
Description
- part of https://github.com/apache/datafusion/issues/21706
as mentioned several places on
- https://github.com/apache/datafusion/pull/21707
There are several command line flags / environment variables that are duplicated in the benchmark runner (and @Omega359 just followed the same pattern with the new runner)
It would be great to consolidate all the options, and for any that have existing environment settings from SessionConfig, instead use session config
It feels like we maybe could just reuse https://docs.rs/datafusion/latest/datafusion/prelude/struct.SessionConfig.html#method.from_env rather than implementing special cases for many of these options
_Originally posted by @alamb in https://github.com/apache/datafusion/pull/21707#discussion_r3121443426_
Contributor guide
Research direction
Start with the benchmark runner and the option handling discussed in issues/21706 and pull/21707, then read SessionConfig::from_env in the DataFusion documentation. Compare the duplicated command-line flags and environment settings across runners and identify which can use SessionConfig. Done means the options are consolidated without separate special cases for settings already supported by SessionConfig.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- performance
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 50/100