Memory-resource porting checklist for current resource free `CUDF_TEST_EXPECT_[COLUMNS/TABLES]_EQUAL` use
- Dominant language
- C++
- Stars
- 9.8k
- Forks
- 1.1k
- Avg merge
- 3d 6m
- Merged PRs (30d)
- 278
Description
Track migration to explicit `cudf::memory_resources`, separating returned allocations from temporary allocations and preventing fallback to the current device resource. Required for #20780.
### Completed
- [x] Memory-resource abstraction and test harness #23581
- `cudf::memory_resources`
- `BaseFixtureWithHarness`
- `cpp/include/cudf/utilities/memory_resource.hpp`
- `cpp/include/cudf_test/base_fixture.hpp`
- [x] Equality preprocessing and row hashing #23665
- Equality `preprocessed_table`, comparators, and hash preprocessing
- Lexicographic preprocessing remains separate and incomplete.
- [ ] Primary column-wrapper allocation paths https://github.com/NVIDIA/cudf/pull/23688
- Fixed-width, string, dictionary, and initializer resource forwarding
- `cpp/include/cudf_test/column_wrapper.hpp`
- Recursive list concatenation remains incomplete.
- [ ] Gather https://github.com/NVIDIA/cudf/pull/23690
- Public/detail gather and type-specific dispatch
- `cpp/include/cudf/detail/gather.*`
- `cpp/src/copying/gather.cu`
- [ ] Segmented gather https://github.com/NVIDIA/cudf/pull/23690
- List segmented-gather entry points and recursion
- `cpp/include/cudf/lists/gather.hpp`
- `cpp/src/lists/copying/segmented_gather.cu`
- [x] Dictionary encode/decode #23642
- Public/detail APIs, row hashing, gather, and output masks
- `cpp/include/cudf/dictionary/encode.hpp`
- `cpp/src/dictionary/{encode,decode}.cu`
### Remaining
- [ ] Generic device algorithms https://github.com/NVIDIA/cudf/pull/24084
- Port `transform_reduce`, `count_if`, `copy_if`, and related helpers.
- `cpp/include/cudf/detail/algorithms/`
- [ ] Validity-mask construction https://github.com/NVIDIA/cudf/pull/24085
- Port `detail::valid_if` to use output and temporary resources.
- `cpp/include/cudf/detail/valid_if.cuh`
- [ ] Null counting and sliced views
- Port `null_count`, `segmented_null_count`, `batch_count_set_bits`, `slice`, and `split`.
- Port list/struct `get_sliced_child`.
- `cpp/src/bitmask/null_mask.cu`
- `cpp/src/copying/{slice,split}.*`
- `cpp/src/lists/lists_column_view.cu`
- `cpp/src/structs/structs_column_view.cpp`
- [ ] Non-empty-null detection
- Port `has_nonempty_nulls` and `has_nonempty_null_rows`.
- Update test assertion callers.
- `cpp/src/copying/purge_nonempty_nulls.cu`
- `cpp/tests/utilities/column_utilities.cu`
- [ ] Concatenate
- Port public/detail column, table, and mask APIs.
- Port fixed-width, string, list, struct, and dictionary implementations.
- `cpp/src/copying/concatenate.cu`
- `cpp/src/{strings,lists,structs}/copying/concatenate.cu`
- `cpp/src/dictionary/detail/concatenate.cu`
- [ ] Recursive list column wrappers
- Pass the full resource pair into concatenate.
- Add harness coverage for nullable nested initializers.
- `cpp/include/cudf_test/column_wrapper.hpp`
- [ ] Lexicographic row preprocessing https://github.com/NVIDIA/cudf/pull/24087
- Port `preprocessed_table`, `self_comparator`, and `two_table_comparator`.
- `cpp/include/cudf/detail/row_operator/lexicographic.cuh`
- `cpp/src/row_operator/row_operators.cu`
- [ ] Dremel list preprocessing https://github.com/NVIDIA/cudf/pull/24086
- Port `get_dremel_data`, `get_comparator_data`, device vectors, and algorithm scratch.
- `cpp/include/cudf/lists/detail/dremel.hpp`
- `cpp/src/lists/dremel.cu`
- [ ] Ranking and sorted order https://github.com/NVIDIA/cudf/pull/24088
- Port `rank`, `sorted_order`, and `stable_sorted_order`.
- `cpp/include/cudf/{detail/,}sorting.hpp`
- `cpp/src/sort/`
- [ ] List offset normalization https://github.com/NVIDIA/cudf/pull/24089
- Port `lists::detail::get_normalized_offsets`.
- `cpp/src/lists/utilities.cu`
- [ ] Regression coverage
- Add resource-routing tests for concatenate dispatch.
- Add recursive nullable-list wrapper tests.
- Fix debug/failure assertion paths that pass a single MR and recapture current MR.
### Acceptance tests
```bash
./gtests/COPYING_TEST \
--gtest_filter='Gather*:SegmentedGather*:StructGather*:TypedStructGather*'
./gtests/DICTIONARY_TEST \
--gtest_filter='DictionaryEncode*:DictionaryDecode*'
./gtests/ROW_OPERATOR_TEST \
--gtest_filter='TypedTableView*:NaNTableView*:RowOperator*'
./gtests/UTILITIES_TEST \
--gtest_filter='*ColumnWrapperTest*'
```
Contributor guide
Research direction
Choose one unchecked migration area rather than the whole checklist, then read the listed headers and source files for that component. Run the relevant acceptance-test command, and consider the work done when allocations use explicit output and temporary resources without recapturing the current device resource, with the listed regression coverage passing.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- data
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100