FixedSizeBinary filter benchmarks vary by substantially from run to run
- Dominant language
- Rust
- Stars
- 3.6k
- Forks
- 1.3k
- Avg merge
- 2d 18h
- Merged PRs (30d)
- 169
Description
> I see other runs of the benchmarks have the same pattern (with the same numbers for the fsb benchmark). Somehow the second run is always takes 66% longer... #7463.
Yeah, it is really weird. It would be great if someone could look into that.
I'll file a ticket
_Originally posted by @alamb in https://github.com/apache/arrow-rs/issues/7573#issuecomment-2956613295_
Contributor guide
Research direction
Start by reproducing the FixedSizeBinary filter benchmark and compare repeated runs, including the behavior reported in #7463. Trace the benchmark setup and execution path to identify why the second run is consistently about 66% slower; done means the cause is explained and benchmark results are stable or the variance is documented and addressed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100