NVIDIA / NVIDIA/cudf

[BUG] `TypeError` when broadcasting `Series` to `DataFrame` with non-numerical columns

Open
#13,731 1 comment 0 reactions 0 assignees View on GitHub
0 - Backlog bug Python
Dominant language
C++
Stars
9.8k
Forks
1.1k
Avg merge
3d 6m
Merged PRs (30d)
278

Description

**Describe the bug**
I am getting a `TypeError` when I try to subtract a `cudf.Series` (`s`) from a `cudf.Dataframe` (`df`) having non-numerical columns, even when `s` only contains elements that correspond to the numerical columns of `df`. Note that it is not an issue when `s` leaves out columns of `df`, unless those columns are numerical.

**Steps/Code to reproduce bug**
```python
In [1]: import cudf
In [2]: df = cudf.DataFrame({"a": ["dog"] * 10, "b": range(10)})
In [3]: s = df.mean(numeric_only=True)
In [4]: df - s
```
```
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
Cell In[4], line 1
----> 1 df - s

File /datasets/rzamora/miniconda3/envs/cudf_pandas_2/lib/python3.10/site-packages/cudf/core/mixins/mixin_factory.py:11, in _partialmethod..wrapper(self, *args2, **kwargs2)
10 def wrapper(self, *args2, **kwargs2):
---> 11 return method(self, *args1, *args2, **kwargs1, **kwargs2)

File /datasets/rzamora/miniconda3/envs/cudf_pandas_2/lib/python3.10/site-packages/cudf/core/indexed_frame.py:3475, in IndexedFrame._binaryop(self, other, op, fill_value, can_reindex, *args, **kwargs)
3471 if operands is NotImplemented:
3472 return NotImplemented
3474 return self._from_data(
-> 3475 ColumnAccessor(type(self)._colwise_binop(operands, op)),
3476 index=out_index,
3477 )

File /datasets/rzamora/miniconda3/envs/cudf_pandas_2/lib/python3.10/site-packages/nvtx/nvtx.py:101, in annotate.__call__..inner(*args, **kwargs)
98 @wraps(func)
99 def inner(*args, **kwargs):
100 libnvtx_push_range(self.attributes, self.domain.handle)
--> 101 result = func(*args, **kwargs)
102 libnvtx_pop_range(self.domain.handle)
103 return result

File /datasets/rzamora/miniconda3/envs/cudf_pandas_2/lib/python3.10/site-packages/cudf/core/frame.py:1761, in Frame._colwise_binop(cls, operands, fn)
1753 assert False, "At least one operand must be a column."
1755 # TODO: Disable logical and binary operators between columns that
1756 # are not numerical using the new binops mixin.
1758 outcol = (
1759 getattr(operator, fn)(right_column, left_column)
1760 if reflect
-> 1761 else getattr(operator, fn)(left_column, right_column)
1762 )
1764 if output_mask is not None:
1765 outcol = outcol.set_mask(output_mask)

TypeError: unsupported operand type(s) for -: 'StringColumn' and 'NoneType'
```

For `pandas`, I get the following output:

```
a b
0 NaN -4.5
1 NaN -3.5
2 NaN -2.5
3 NaN -1.5
4 NaN -0.5
5 NaN 0.5
6 NaN 1.5
7 NaN 2.5
8 NaN 3.5
9 NaN 4.5
```

**Expected behavior**
I expect the same behavior as pandas. That is, when broadcasting a `Series` to the columns of a `DataFrame` for a binop, missing columns in the `Series`' index should result in the corresponding column having all null values in the output. This seems to work fine in `cudf`, unless the missing column is non-numerical.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.