[QST] Is Series.value_counts supposed to behave differently between Pandas and CuDF ?
- Dominant language
- C++
- Stars
- 9.8k
- Forks
- 1.1k
- Avg merge
- 3d 6m
- Merged PRs (30d)
- 278
Description
From reading the [value_counts() docs](https://docs.rapids.ai/api/cudf/stable/user_guide/api_docs/api/cudf.series.value_counts), i gathered that the function would be working exactly the same as the pandas implementation, but upon using it, it became clear that the output is different in some cases, specifically when using the `bins` parameter:
- Input data is exactly the same between the 2 runs (Pandas and CuDF), which is a Series of 2734 numbers
```python
cudf_data = cudf.from_pandas(pandas_data)
```
- When running the function in pandas the result is the correct and desired output (counting through 5 intervals between the min and max)
```python
pandas_data.value_counts(bins=5, sort=False)
```
```
(-2038.538, 407707.479] 1493
(407707.479, 815414.958] 1240
(815414.958, 1223122.436] 0
(1223122.436, 1630829.915] 0
(1630829.915, 2038537.394] 1
Name: count, dtype: int64
```
- But when running the same data in CuDF it seems that the function doesn't guarantee the desired number of bins to be in the output
```python
cudf_data.value_counts(bins=5, sort=False)
```
```
(1630829.915, 2038537.394] 1
(407707.479, 815414.958] 1240
(-2038.538, 407707.479] 1493
Name: count, dtype: int64
```
1. From comparing the outputs, it looks like the CuDF implementation ignores the intervals where the count is 0.
2. The order of the output doesn't seem to be the same even though i've set `sort=False` and the input data is the same in order in both cases.
Is this by design, or should the 2 implementations produce the same output ?
Contributor guide
Assessment
This issue has not been assessed yet.