mars-project / mars-project/mars

Support full featured groupby function

Open
#1,036 0 comments 0 reactions 0 assignees View on GitHub
mod: dataframe type: feature
Dominant language
Python
Stars
2.7k
Forks
325
PR merge metrics
No merged PRs in 30d

Description

Currently df.groupby() in Mars only implements ``as_index``. ``sort`` is added but not implemented yet, while ``level`` is not implemented which is also useful. What's more, when ``GroupBy`` object is generated, keys and grouped data frames are serialized instead of ``GroupBy`` object itself, which may not be a universal solution for different scenarios of groupby.

``sort_index`` (#1037) may be useful when implementing this.

Subtasks:

* [x] Fix serialization issue of GroupBy objects (#1136)
* [x] Support groupby-getitem (#1136)
* [x] Support level option (#1136)
* [x] Support ``group_keys`` (#1136)
* [ ] Support ``observed``
* [x] Support ``sort`` (#2959)
* [x] Support args of cumulative functions in groupby (and integrate ``df.sum(level=x, **kwargs)``)
* [x] Support grouping by series (#1181)

Contributor guide

Open the contributing guide

Research direction

Start at Mars's df.groupby() implementation and compare the remaining observed option with the existing groupby options. Done means observed is supported for groupby; the issue names no specific file or test path, so those will need to be located first.

Written by the indexing model from the issue text.

Assessment

Tech stack
pandas, python
Domain
data
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.