NVIDIA / NVIDIA/cudf

[FEA] Provide a way to specify the maximum allowable precision for integers/floats

Open
#10,558 2 comments 0 reactions 0 assignees View on GitHub
feature request Python
Dominant language
C++
Stars
9.8k
Forks
1.1k
Avg merge
3d 6m
Merged PRs (30d)
278

Description

**Is your feature request related to a problem? Please describe.**

GPU memory is a valuable resource, and using int64/float64 columns where int32/float32 would suffice means using 2x as much memory unnecessarily. As opposed to scientific computing, 32-bit data types (or lower) are sufficient for many data science applications.

Even only 32-bit data types as inputs, the resulting output can be a 64-bit type:

```python
>>> cudf.Series([1, 2, 3], dtype="int32") + cudf.Scalar(1, dtype="float32")
0 2.0
1 3.0
2 4.0
dtype: float64
```

(this is consistent with Pandas and NumPy)

**Describe the solution you'd like**

It would be nice to be able to specify a maximum bitwidth for integer/floating types. If an operation would result in a value greater than could be accommodated, simply overflowing would be acceptable.

This could be another use case for [cudf.config](https://github.com/rapidsai/cudf/issues/5311).

**Describe alternatives you've considered**

The user can carefully cast results back from 64bit to 32bit to reduce memory usage, but this is tedious and does not help with peak memory usage.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.