datafusion-contrib / datafusion-contrib/datafusion-distributed
[Benchmark datasets] Add a sorted ClickBench benchmark dataset
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 139
- Forks
- 67
- Avg merge
- 3d 1h
- Merged PRs (30d)
- 35
Description
Part of #629.
Add a deterministic ClickBench dataset variant whose Parquet data is globally sorted by an explicitly documented key (or key sequence). This should make physical data ordering a controlled benchmark dimension, separate from the existing ClickBench layout.
Scope:
- Add the Rust preparation code and
gen-clickbench-sorted.shgeneration entry point. - Write the dataset to a distinct slash-style dataset path, such as
testdata/clickbench-sorted/<variant>.
For a more detailed explanation of how to wire this up, see https://github.com/datafusion-contrib/datafusion-distributed/issues/629
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading issue #629 and tracing the existing ClickBench dataset generation flow. Add the Rust preparation code and the gen-clickbench-sorted.sh entry point, writing a distinct dataset under testdata/clickbench-sorted/. Done means the dataset is deterministic, globally sorted by an explicitly documented key or key sequence, and separate from the existing ClickBench layout.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- bash, rust
- Domain
- data-engineering, distributed-systems
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100