[BUG] Investigate the use of memory resources in Parquet stack
- Dominant language
- C++
- Stars
- 9.8k
- Forks
- 1.1k
- Avg merge
- 3d 6m
- Merged PRs (30d)
- 278
Description
## Description
In the Parquet reader/writer, we may be using the `_mr` (the memory resource with which the output device memory should be allocated) to also allocate internal memory/vectors and (highly unlikely) vice versa with the `cudf::get_current_device_resource_ref()` for output memory.
## Proposed fix
Create a new member variable called `internal_mr` or `temp_mr` initialized with `cudf::get_current_device_resource_ref()` or user provided temp_mr (as per proposed libcudf API design) and use that to allocate all internal memory (strictly no use of `cudf::get_current_device_resource_ref()` or not-providing an `mr`). xref https://github.com/rapidsai/cudf/issues/20780
## Additional context
Hmm, since this memory is not for the output table, we should actually use `cudf::get_current_device_resource_ref()` for all these. That said, I think this is likely a parquet-wide issue that we need to fix (in a separate PR of course).
_Originally posted by @mhaseeb123 in https://github.com/rapidsai/cudf/pull/21139#discussion_r2748444735_
Contributor guide
Assessment
This issue has not been assessed yet.