aws / aws/amazon-redshift-python-driver
Missing support for PyArrow types in Pandas dataframes
Open
- Dominant language
- Python
- Stars
- 220
- Forks
- 86
- PR merge metrics
- No merged PRs in 30d
Description
Most of the data ingestion utilities (`pd.read_csv`, `pd.read_parquet`, `pd.read_sql`) since Pandas 2.2 include a `dtype_backend='pyarrow'` option to use PyArrow types instead of NumPy backend.
It would be awesome if there was support for this.
Contributor guide
Research direction
The issue names pd.read_csv, pd.read_parquet, and pd.read_sql but no repository files or tests. First locate the data-ingestion paths that construct Pandas dataframes and inspect current dtype handling; establish coverage for dtype_backend='pyarrow', then verify all three ingestion paths handle PyArrow-backed dataframes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- pandas, python
- Domain
- data
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100