mars-project / mars-project/mars
Optimize performance of GroupBy.nunique
Open
mod: dataframe
task: medium
type: enhancement
- Dominant language
- Python
- Stars
- 2.7k
- Forks
- 325
- PR merge metrics
- No merged PRs in 30d
Description
Now we implements GroupBy.nunique using GroupBy.transform, some optimizations could be applied to reduce intermediate data size.
Contributor guide
Research direction
Start by locating the GroupBy.nunique implementation and inspecting how it currently delegates to GroupBy.transform. Identify the intermediate data created by that path, then verify that the optimized behavior preserves existing results while reducing intermediate data size.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- pandas, python
- Domain
- data
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100