NVIDIA / NVIDIA/cudf

[QST] Is Series.value_counts supposed to behave differently between Pandas and CuDF ?

Open
#16,044 3 comments 0 reactions 0 assignees View on GitHub
0 - Backlog bug cudf.pandas doc Python
Dominant language
C++
Stars
9.8k
Forks
1.1k
Avg merge
3d 6m
Merged PRs (30d)
278

Description

From reading the [value_counts() docs](https://docs.rapids.ai/api/cudf/stable/user_guide/api_docs/api/cudf.series.value_counts), i gathered that the function would be working exactly the same as the pandas implementation, but upon using it, it became clear that the output is different in some cases, specifically when using the `bins` parameter:

- Input data is exactly the same between the 2 runs (Pandas and CuDF), which is a Series of 2734 numbers

```python
cudf_data = cudf.from_pandas(pandas_data)
```
- When running the function in pandas the result is the correct and desired output (counting through 5 intervals between the min and max)
```python
pandas_data.value_counts(bins=5, sort=False)
```
```
(-2038.538, 407707.479] 1493
(407707.479, 815414.958] 1240
(815414.958, 1223122.436] 0
(1223122.436, 1630829.915] 0
(1630829.915, 2038537.394] 1
Name: count, dtype: int64
```
- But when running the same data in CuDF it seems that the function doesn't guarantee the desired number of bins to be in the output
```python
cudf_data.value_counts(bins=5, sort=False)
```
```
(1630829.915, 2038537.394] 1
(407707.479, 815414.958] 1240
(-2038.538, 407707.479] 1493
Name: count, dtype: int64
```

1. From comparing the outputs, it looks like the CuDF implementation ignores the intervals where the count is 0.
2. The order of the output doesn't seem to be the same even though i've set `sort=False` and the input data is the same in order in both cases.

Is this by design, or should the 2 implementations produce the same output ?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.