NVIDIA / NVIDIA/cudf

[FEA] Implement Memory-Aware Test Utility for Large-Scale Tests

Open
#21,759 0 comments 1 reaction 0 assignees View on GitHub
feature request libcudf
Dominant language
C++
Stars
9.8k
Forks
1.1k
Avg merge
3d 6m
Merged PRs (30d)
278

Description

**Is your feature request related to a problem? Please describe.**
Currently, large-scale tests—such as those involving large hash tables for joins—can require significant device memory (e.g., 20 GB+). In CI environments with limited GPU resources, these tests trigger `rmm` bad allocation errors. To prevent CI failures, we currently either manually add `GTEST_SKIP()` workarounds or simply remove the test for simplicity.

**Describe the solution you'd like**
Proposed by @ttnghia https://github.com/rapidsai/cudf/pull/21440#discussion_r2920565360, It may be helpful to have a standardized utility function within `cudf_test` that can query the available device memory. This utility would allow tests to conditionally run or skip based on the current hardware's capability.

Example usage:

```cpp
if (!cudf_test::has_available_memory(20_GB)) {
GTEST_SKIP() << "Skipping test: Requires 20GB of available device memory.";
}

```

**Describe alternatives you've considered**

* **Manual Skips:** Hard-coding `GTEST_SKIP()` as a temporary workaround. This is difficult to maintain and requires manual reverts to run tests locally.
* **Environment Variables:** Using a flag like `RUN_LARGE_TESTS=1`. This is less precise than a direct memory check because it doesn't account for the actual physical limits of the GPU being used.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.