DataFrame API: Consider adding support for sparse data
Open
core
dataframe
dsl
improvement
P3
python
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 205
Description
pandas has support for [sparse data representation](https://pandas.pydata.org/pandas-docs/stable/user_guide/sparse.html). We don't currently support this in the Beam DataFrame API, but it may make sense to add.
Imported from Jira [BEAM-12425](https://issues.apache.org/jira/browse/BEAM-12425). Original Jira may contain additional context.
Reported by: bhulette.
Contributor guide
Research direction
No files, tests, or entry points are named in the issue. Review the Beam DataFrame API and the linked pandas sparse-data documentation, then use the original Jira context to define the supported sparse representations and tests that would demonstrate completed support.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- pandas, python
- Domain
- data
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100