apache / apache/sedona-db

Consider porting compile-time defined benchmarks to a runtime benchmark utility

Open
#313 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
503
Forks
61
Avg merge
2d 5h
Merged PRs (30d)
91

Description

From https://github.com/apache/sedona-db/pull/312

> perhaps we have too many of the rust level benchmarks and should remove some, or change the process of running them

Currently we have a lot of Rust level benchmarks that are traditional Criterion benchmarks.

https://github.com/apache/sedona-db/blob/e8f4c5e338c61a52dd2994c11943f3f0b546536d/rust/sedona-functions/benches/native-functions.rs#L21-L28

These are useful when comparing between kernel implementations of the same function; however, we have a lot of them and we don't actually run them except to make sure they run once with tiny data or on specific PRs that update a kernel implementation. I think we may be able to reduce this to a single "benchmark" per function-providing crate that enabled the function name and the `ArrayArray(Point, LineString(10))` to be parsed at runtime somehow (e.g., `cargo bench -p sedona-geos -- st_make_valid "LineString(10)"`.

Contributor guide

Open the contributing guide

Research direction

Start with rust/sedona-functions/benches/native-functions.rs and the discussion in pull request 312. Review how the current Criterion benchmarks are defined and run, then compare that with the proposed cargo bench -p sedona-geos -- st_make_valid "LineString(10)" entry point. Done means the benchmark setup can be evaluated at runtime with a function name and input shape while retaining useful per-function comparisons.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
performance
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.