ClickBench run.sh restarts process per try, losing hot-run caches
- Dominant language
- Rust
- Stars
- 9.3k
- Forks
- 2.4k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 344
Description
[`datafusion-partitioned/run.sh`](https://github.com/ClickHouse/ClickBench/blob/main/datafusion-partitioned/run.sh) invokes a fresh `datafusion-cli -f create.sql ...` for **each** of the 3 tries, not just try 1. Since `create.sql` does `CREATE EXTERNAL TABLE ... LOCATION 'partitioned'` and the default is `collect_statistics = true`, every try re-scans all Parquet footers in a cold process.
Other engines keep one process across tries — e.g. `duckdb-parquet-partitioned` runs all 3 tries in one `duckdb hits.db` session with `parquet_metadata_cache=true`. Per the [ClickBench rules](https://github.com/ClickHouse/ClickBench#caching), tries 2–3 are meant to be hot; our script makes them effectively cold.
**Fix:** drop OS cache once before try 1, then run all 3 tries in a single `datafusion-cli` session that has already executed `create.sql`.
Contributor guide
Research direction
Start with datafusion-partitioned/run.sh and inspect how it invokes datafusion-cli for each try; compare the session flow with duckdb-parquet-partitioned. Run the benchmark script and verify that OS cache is dropped once, create.sql is executed once, and all three tries share one process so later tries can use hot-run caches.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust, shell
- Domain
- cli, performance
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 72/100