[FEA] Add pipelining to the `NDS-H-cpp` benchmarks
- Dominant language
- C++
- Stars
- 9.8k
- Forks
- 1.1k
- Avg merge
- 3d 6m
- Merged PRs (30d)
- 278
Description
**Is your feature request related to a problem? Please describe.**
In the libcudf microbenchmarks, the `NDS-H-cpp` benchmarks are a useful tool for studying GPU query performance.
They could also be used to study pipelining. An application can "pipeline" work on the GPU using 2 or more host threads to sequence calls to the libcudf public API. Pipelining is useful in IO-heavy workloads where one thread can be copying data to the GPU while another thread is running kernels over previously-copied data. Pipelining is needed to ensure that GPU compute is not left idle during copying steps.
**Describe the solution you'd like**
Claude and I wrote a simple concurrent benchmark for query 5 using PTDS. We could take this idea and update to use a CUDA stream pool. We would also want to consider how pipelining could be applied to other queries without modifying each query file.
```
void ndsh_q5_concurrent(nvbench::state& state)
{
// Generate the required parquet files in device buffers
double const scale_factor = state.get_float64("scale_factor");
int const num_threads = state.get_int64("num_threads");
int const runs_per_thread = state.get_int64("runs_per_thread");
std::unordered_map sources;
generate_parquet_data_sources(
scale_factor, {"customer", "orders", "lineitem", "supplier", "nation", "region"}, sources);
BS::thread_pool threads(num_threads);
state.exec(nvbench::exec_tag::sync, [&](nvbench::launch& launch) {
nvtxRangePushA(("ndsh_q5_concurrent " + std::to_string(num_threads) + " threads, " +
std::to_string(runs_per_thread) + " runs/thread, scale_factor=" +
std::to_string(scale_factor)).c_str());
auto query_func = [&](int index) {
nvtxRangePushA("ndsh_q5");
run_ndsh_q5(state, sources);
nvtxRangePop();
};
threads.pause();
threads.detach_sequence(0, num_threads * runs_per_thread, query_func);
threads.unpause();
threads.wait();
nvtxRangePop();
});
}
NVBENCH_BENCH(ndsh_q5_concurrent)
.set_name("ndsh_q5_concurrent")
.add_float64_axis("scale_factor", {0.01, 0.1, 1})
.add_int64_axis("num_threads", {2, 4})
.add_int64_axis("runs_per_thread", {1, 4});
```
The profiles show that query 5 is IO-bound and yet still has some bubbles where compute is running, but not IO. We should investigate why IO is blocking kernel work in some cases.

Contributor guide
Assessment
This issue has not been assessed yet.