NVIDIA / NVIDIA/cudf

[FEA] Add pipelining to the `NDS-H-cpp` benchmarks

Open
#18,206 0 comments 0 reactions 0 assignees View on GitHub
feature request good first issue libcudf
Dominant language
C++
Stars
9.8k
Forks
1.1k
Avg merge
3d 6m
Merged PRs (30d)
278

Description

**Is your feature request related to a problem? Please describe.**
In the libcudf microbenchmarks, the `NDS-H-cpp` benchmarks are a useful tool for studying GPU query performance.

They could also be used to study pipelining. An application can "pipeline" work on the GPU using 2 or more host threads to sequence calls to the libcudf public API. Pipelining is useful in IO-heavy workloads where one thread can be copying data to the GPU while another thread is running kernels over previously-copied data. Pipelining is needed to ensure that GPU compute is not left idle during copying steps.

**Describe the solution you'd like**
Claude and I wrote a simple concurrent benchmark for query 5 using PTDS. We could take this idea and update to use a CUDA stream pool. We would also want to consider how pipelining could be applied to other queries without modifying each query file.

```
void ndsh_q5_concurrent(nvbench::state& state)
{
// Generate the required parquet files in device buffers
double const scale_factor = state.get_float64("scale_factor");
int const num_threads = state.get_int64("num_threads");
int const runs_per_thread = state.get_int64("runs_per_thread");

std::unordered_map sources;
generate_parquet_data_sources(
scale_factor, {"customer", "orders", "lineitem", "supplier", "nation", "region"}, sources);

BS::thread_pool threads(num_threads);

state.exec(nvbench::exec_tag::sync, [&](nvbench::launch& launch) {
nvtxRangePushA(("ndsh_q5_concurrent " + std::to_string(num_threads) + " threads, " +
std::to_string(runs_per_thread) + " runs/thread, scale_factor=" +
std::to_string(scale_factor)).c_str());
auto query_func = [&](int index) {
nvtxRangePushA("ndsh_q5");
run_ndsh_q5(state, sources);
nvtxRangePop();
};

threads.pause();
threads.detach_sequence(0, num_threads * runs_per_thread, query_func);
threads.unpause();
threads.wait();
nvtxRangePop();
});
}

NVBENCH_BENCH(ndsh_q5_concurrent)
.set_name("ndsh_q5_concurrent")
.add_float64_axis("scale_factor", {0.01, 0.1, 1})
.add_int64_axis("num_threads", {2, 4})
.add_int64_axis("runs_per_thread", {1, 4});
```

The profiles show that query 5 is IO-bound and yet still has some bubbles where compute is running, but not IO. We should investigate why IO is blocking kernel work in some cases.
![Image](https://github.com/user-attachments/assets/a54f519e-b770-4c2e-91d4-e7899d09c427)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.