apache / apache/datafusion

Allow overriding SQL path base for benchmarks

Open
#13,744 1 comment 1 reaction 0 assignees View on GitHub
enhancement
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

### Is your feature request related to a problem or challenge?

I am trying to run datafusion benchmarks on the hermit unikernel. Hermit does not support changing the current working directory. Currently, the TPC-H benchmark loads the SQL for queries from two possible paths (in `get_query_sql`):
```rust
let possibilities = vec![
format!("queries/q{query}.sql"),
format!("benchmarks/queries/q{query}.sql"),
];
```
Due to limitations of hermit, I can not put the files at either of those locations.

### Describe the solution you'd like

I would like to be able to specify a custom path, just like I can for data:
```shell
./benchmark_bin tpch --path /root/benchmarks/data --queries /root/benchmarks/queries
```

### Describe alternatives you've considered

Currently, I am using a patched version of datafusion with the hard-coded path replaced

### Additional context

_No response_

Contributor guide

Open the contributing guide

Research direction

Start at the TPC-H benchmark entry point and its get_query_sql function, then inspect how the existing --path option is handled. Add the requested --queries path so the benchmark command can find its SQL files there, and run the example tpch command to verify query loading.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust, sql
Domain
cli, data-engineering
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.