[Python] supporting pandas sparse series in pyarrow
- Dominant language
- C++
- Stars
- 17.1k
- Forks
- 4.3k
- Avg merge
- 3d 13h
- Merged PRs (30d)
- 88
Description
I've seen that Pandas sparse series was not supported in pyarrow since it was planned to be deprecated. In Pandas 1.0.1 they released a stable version of sparse array and as far as I know it is not planned to be deprecated anymore. Are you planning to support sparse series in next versions of pyarrow ?
**Environment**: ubuntu 16/18
**Reporter**: [Michael Novitsky](https://issues.apache.org/jira/browse/ARROW-8679)
**Note**: *This issue was originally created as [ARROW-8679](https://issues.apache.org/jira/browse/ARROW-8679). Please see the [migration documentation](https://github.com/apache/arrow/issues/14542) for further details.*
Contributor guide
Research direction
Start with pyarrow's pandas integration and the linked migration documentation. The issue names no file, test, or entry point, so first clarify the intended sparse Series support and its compatibility scope; done requires an agreed behavior and verification path.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- pandas, python
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100