[FEA] Improve cudf tests for hash_partition
- Dominant language
- C++
- Stars
- 9.8k
- Forks
- 1.1k
- Avg merge
- 3d 6m
- Merged PRs (30d)
- 278
Description
**Is your feature request related to a problem? Please describe.**
As `cudf::hash_partition` now can use a hash function other than Murmur3 or take a custom seed value, I think the tests should be improved for better coverage.
**Describe the solution you'd like**
As described in [this comment](https://github.com/rapidsai/cudf/pull/7771#issuecomment-811692022), two key properties of `cudf::hash_partition` are
1. Output partitions are disjoint; and
2. Rows having the same key are always assigned the same partition id, even when they appear in different dataframes (when the number of partitions is fixed).
We need to check these properties for some combinations of hash function types and seed values. Property 1 is checked in [this Python test](https://github.com/rapidsai/cudf/blob/b9415cae362882380e14b5baa6af49275021356e/python/cudf/cudf/tests/test_dataframe.py#L1106-L1135), but only with the default hash function and seed value, and I believe there are no tests for property 2 yet.
Additionally, we should remove [this test](https://github.com/rapidsai/cudf/blob/branch-0.19/python/cudf/cudf/tests/test_dataframe.py#L1181-L1200). The test is comparing a hard-coded value with the result of `hash_partition` for a test input, but this is over-constraining, as `hash_partition` in general has no guarantee on the order in which the keys appear in the output partitions. If we later decide to change the `row_hasher` in any way, this test will start to fail.
Contributor guide
Assessment
This issue has not been assessed yet.