[FEA] Implement Memory-Aware Test Utility for Large-Scale Tests
- Dominant language
- C++
- Stars
- 9.8k
- Forks
- 1.1k
- Avg merge
- 3d 6m
- Merged PRs (30d)
- 278
Description
**Is your feature request related to a problem? Please describe.**
Currently, large-scale tests—such as those involving large hash tables for joins—can require significant device memory (e.g., 20 GB+). In CI environments with limited GPU resources, these tests trigger `rmm` bad allocation errors. To prevent CI failures, we currently either manually add `GTEST_SKIP()` workarounds or simply remove the test for simplicity.
**Describe the solution you'd like**
Proposed by @ttnghia https://github.com/rapidsai/cudf/pull/21440#discussion_r2920565360, It may be helpful to have a standardized utility function within `cudf_test` that can query the available device memory. This utility would allow tests to conditionally run or skip based on the current hardware's capability.
Example usage:
```cpp
if (!cudf_test::has_available_memory(20_GB)) {
GTEST_SKIP() << "Skipping test: Requires 20GB of available device memory.";
}
```
**Describe alternatives you've considered**
* **Manual Skips:** Hard-coding `GTEST_SKIP()` as a temporary workaround. This is difficult to maintain and requires manual reverts to run tests locally.
* **Environment Variables:** Using a flag like `RUN_LARGE_TESTS=1`. This is less precise than a direct memory check because it doesn't account for the actual physical limits of the GPU being used.
Contributor guide
Assessment
This issue has not been assessed yet.