Consider porting compile-time defined benchmarks to a runtime benchmark utility
- Dominant language
- Rust
- Stars
- 503
- Forks
- 61
- Avg merge
- 2d 5h
- Merged PRs (30d)
- 91
Description
From https://github.com/apache/sedona-db/pull/312
> perhaps we have too many of the rust level benchmarks and should remove some, or change the process of running them
Currently we have a lot of Rust level benchmarks that are traditional Criterion benchmarks.
https://github.com/apache/sedona-db/blob/e8f4c5e338c61a52dd2994c11943f3f0b546536d/rust/sedona-functions/benches/native-functions.rs#L21-L28
These are useful when comparing between kernel implementations of the same function; however, we have a lot of them and we don't actually run them except to make sure they run once with tiny data or on specific PRs that update a kernel implementation. I think we may be able to reduce this to a single "benchmark" per function-providing crate that enabled the function name and the `ArrayArray(Point, LineString(10))` to be parsed at runtime somehow (e.g., `cargo bench -p sedona-geos -- st_make_valid "LineString(10)"`.
Contributor guide
Research direction
Start with rust/sedona-functions/benches/native-functions.rs and the discussion in pull request 312. Review how the current Criterion benchmarks are defined and run, then compare that with the proposed cargo bench -p sedona-geos -- st_make_valid "LineString(10)" entry point. Done means the benchmark setup can be evaluated at runtime with a function name and input shape while retaining useful per-function comparisons.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- performance
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100