[FEA] Hash values in nested columns
- Dominant language
- C++
- Stars
- 9.8k
- Forks
- 1.1k
- Avg merge
- 2d 22h
- Merged PRs (30d)
- 274
Description
Hashing categorical features to a fixed number of bins is a common preprocessing operation, particularly for tabular deep learning models where memory requirements scale with the number of bins. For extension of CuDF to nested columns, it would be helpful if calls to `Series.hash_values` hashed the values *in* each element's list and not the list itself. This would allow categorical hashing to extend to multi-hot categorical features.
As an example:
```
df = cudf.DataFrame({'a': [[0, 1, 2], [3, 4], [], [5], [6, 7, 8, 9]]})
df['a'].hash_values()
# not sure what this representation will
# print like but some like this
[[ 29140149, -247539971, 1683430573], [1098043756, 1851360991], [],
[100260016], [154726282, -1778135556, -1793932552, 246633392]]
df['a'].hash_values() % 4
[[1, 1, 1], [0, 3], [], [0], [2, 0, 0, 0]]
```
**Additional context**
Necessary for extension of [NVTabular HashBucket op](https://github.com/NVIDIA/NVTabular/blob/9f8216a89d565e00d8356ffef62f4437f3e2dee3/nvtabular/ops.py#L498) to multi-hot categorical data
Contributor guide
Research direction
Start by tracing the cuDF Series.hash_values entry point and how it handles nested list columns. Use the issue's multi-hot example and NVTabular HashBucket context to define expected per-element results, including empty lists; done means nested values are hashed individually and modulo binning works as shown.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, python
- Domain
- data
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100