NVIDIA / NVIDIA/cudf

[BUG] Series setitem raises for safe cases in pandas compatibility mode

Open
#20,720 0 comments 0 reactions 1 assignee Claimed by @galipremsagar View on GitHub
bug cudf.pandas Python
Dominant language
C++
Stars
9.8k
Forks
1.1k
Avg merge
3d 6m
Merged PRs (30d)
278

Description

**Describe the bug**
Series setitem raises/converts dtypes to incorrect type if an empty slice is passed to iloc. See example below.

This is the cause of various failures, some of which are:
```
"tests/series/indexing/test_indexing.py::test_setitem_empty_indexer[indexer0-val0]",
"tests/series/indexing/test_indexing.py::test_setitem_empty_indexer[indexer0-val1]",
"tests/series/indexing/test_indexing.py::test_setitem_empty_indexer[indexer1-val0]",
"tests/series/indexing/test_indexing.py::test_setitem_empty_indexer[indexer1-val1]",
"tests/series/indexing/test_indexing.py::test_setitem_empty_indexer[indexer2-val0]",
"tests/series/indexing/test_indexing.py::test_setitem_empty_indexer[indexer2-val1]",
"tests/series/indexing/test_indexing.py::test_setitem_empty_indexer[indexer3-val0]",
"tests/series/indexing/test_indexing.py::test_setitem_empty_indexer[indexer3-val1]",
```
**Steps/Code to reproduce bug**
```python

In [1]: import pandas as pd

In [2]: df = pd.DataFrame({'a':[1, 2], 'b':"x"})

In [3]: df
Out[3]:
a b
0 1 x
1 2 x

In [4]: df.loc[[]] = 1.5

In [5]: df
Out[5]:
a b
0 1 x
1 2 x

In [6]: import cudf

In [7]: cudf.set_option("mode.pandas_compatible", True)

In [8]: gdf = cudf.from_pandas(df)

In [9]: gdf
Out[9]:
a b
0 1 x
1 2 x

In [10]: gdf.loc[[]] = 1.5
/raid/pgali/envs/cudfdev/lib/python3.13/site-packages/cudf/core/series.py:254: FutureWarning: Setting an item of incompatible dtype is deprecated and will raise in a future error of pandas. Value '1.5' has dtype incompatible with int64, please explicitly cast to a compatible dtype first.
warnings.warn(
d---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
Cell In[10], line 1
----> 1 gdf.loc[[]] = 1.5

File /raid/pgali/envs/cudfdev/lib/python3.13/site-packages/cudf/core/dataframe.py:161, in _DataFrameIndexer.__setitem__(self, key, value)
159 if not isinstance(key, tuple):
160 key = (key, slice(None))
--> 161 return self._setitem_tuple_arg(key, value)

File /raid/pgali/envs/cudfdev/lib/python3.13/site-packages/cudf/core/dataframe.py:381, in _DataFrameLocIndexer._setitem_tuple_arg(self, key, value)
377 self._frame[
378 columns_df._column_names[0]
379 ].loc._loc_to_iloc(key[0])
380 for col in columns_df._column_names:
--> 381 self._frame[col].loc[key[0]] = value
382 except KeyError:
383 if not is_scalar(key[0]):

File /raid/pgali/envs/cudfdev/lib/python3.13/site-packages/cudf/core/series.py:332, in _SeriesLocIndexer.__setitem__(self, key, value)
330 value = cudf.Series(value)
331 value = value._align_to_index(self._frame.index, how="right")
--> 332 self._frame.iloc[key] = value

File /raid/pgali/envs/cudfdev/lib/python3.13/site-packages/cudf/core/series.py:248, in _SeriesIlocIndexer.__setitem__(self, key, value)
244 else:
245 to_dtype = find_common_type(
246 (tmp_value.dtype, self._frame.dtype)
247 )
--> 248 tmp_value = tmp_value.astype(to_dtype)
249 if to_dtype != self._frame.dtype:
250 # Do not remove until pandas-3.0 support is added.
251 assert PANDAS_LT_300, (
252 "Need to drop after pandas-3.0 support is added."
253 )

File /raid/pgali/envs/cudfdev/lib/python3.13/site-packages/cudf/core/column/column.py:1886, in ColumnBase.astype(self, dtype, copy)
1880 if (
1881 cudf.get_option("mode.pandas_compatible")
1882 and isinstance(dtype, pd.ArrowDtype)
1883 and not cudf.api.types.is_string_dtype(dtype)
1884 ):
1885 raise TypeError(f"Unsupported dtype for astype: {dtype}")
-> 1886 result = self.as_string_column(dtype)
1887 else:
1888 result = self.as_numerical_column(dtype)

File /raid/pgali/envs/cudfdev/lib/python3.13/site-packages/cudf/core/column/numerical.py:487, in NumericalColumn.as_string_column(self, dtype)
481 col = self
482 if (
483 cudf.get_option("mode.pandas_compatible")
484 and isinstance(dtype, np.dtype)
485 and dtype.kind == "O"
486 ):
--> 487 raise ValueError(
488 "Cannot convert numerical column to string column "
489 "when dtype is an object dtype in pandas compatibility mode."
490 )
491 if len(self) == 0:
492 return cast(
493 cudf.core.column.StringColumn,
494 column_empty(0, dtype=CUDF_STRING_DTYPE),
495 )

ValueError: Cannot convert numerical column to string column when dtype is an object dtype in pandas compatibility mode.

In [11]: cudf.set_option("mode.pandas_compatible", False)

In [12]: gdf.loc[[]] = 1.5

In [13]: gdf
Out[13]:
a b
0 1.0 x
1 2.0 x
```

**Expected behavior**
A clear and concise description of what you expected to happen.

**Environment overview (please complete the following information)**
- Environment location: [Bare-metal, Docker, Cloud(specify cloud provider)]
- Method of cuDF install: [conda, Docker, or from source]
- If method of install is [Docker], provide `docker pull` & `docker run` commands used

**Environment details**
Please run and paste the output of the `cudf/print_env.sh` script here, to gather any other relevant environment details

**Additional context**
Add any other context about the problem here.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.