Unexpected behavior with Pyarrow string conversion
- Dominant language
- Python
- Stars
- 13.9k
- Forks
- 2k
- PR merge metrics
- No merged PRs in 30d
Description
**Describe the issue**:
**Minimal Complete Verifiable Example**:
```python
# Put your MCVE code here
import pandas as pd
import dask.dataframe as dd
df = dd.from_pandas(pd.DataFrame({"col1": ["a","a","a", "b", "c", "c"], "col2": [1,2,3,4,5,6], "col3": [43,23,12,32,65,76]}), npartitions=2)
df.groupby("col1").agg({"col2": list, "col3": "first"}).reset_index().compute().info()
```
**Anything else we need to know?**:
In this example, the column 'col2' is being converted to a string instead of an array (object), which makes future processing more difficult. Is there a way to ignore certain columns from auto string conversion? I am aware of the configuration setting dataframe.convert-string, but it applies to all columns.
##### Dataframe
##### Dataframe after groupby
**Environment**:
- Dask version: 2023.9.3
- Python version: 3.9.15
- Operating System: Centos 7
- Install method (conda, pip, source): conda
Contributor guide
Research direction
No source files or tests are named. Start by running the provided MCVE with the reported Dask 2023.9.3 and Python 3.9.15 environment, compare col2 before and after the groupby aggregation, and consider the issue done when list-valued output is preserved or the supported per-column conversion behavior is clearly established.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- pandas, python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100