NVIDIA / NVIDIA/cudf

[FEA] Check dtype requirements on multiindex codes

Open
#14,472 0 comments 0 reactions 0 assignees View on GitHub
feature request no-oom Python
Dominant language
C++
Stars
9.8k
Forks
1.1k
Avg merge
3d 6m
Merged PRs (30d)
278

Description

**Is your feature request related to a problem? Please describe.**

(Seen as part of a review of #14470).

Multiindex codes and levels are effectively a categorical encoding of the columns of the multiindex entries. The codes are used to index the levels. As such, they should probably have type equivalent to `cudf::size_type`. Currently, however, they are a int64. This is a larger memory footprint than necessary. Moreover, it (in some constructor circumstances) necessitates more copies than necessary.

**Describe the solution you'd like**

Use correct dtype. Since the public `codes` and `levels` properties wrap the results in pandas `FrozenList` objects to mimic the pandas API, it may be possible to just store the codes/levels pairs as `CategoricalColumn`s internally, rather than the current structure.

**Describe alternatives you've considered**

n/a

**Additional context**

n/a

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.