performance regression when groupby acts on multiple columns
Open
- Dominant language
- C++
- Stars
- 1.9k
- Forks
- 164
- Avg merge
- 7h 31m
- Merged PRs (30d)
- 1
Description
For example, we have a frame which the shape of it is(34284584, 18),
when by only acts on one column or a few columns, like:
`df[:, {"A":min(f.A), by("B")}] or df[:, {"A":min(f.A), by(["B", "C"])}]`
It takes about seconds.
But when by acts on multiple columns, the time-consuming increases to the minute level, like:
`df[:, {"A":min(f.A), by(["B", "C", "D", "E", "F", "H", "G", "J", "K", "M")}]`
Performance is even weaker than single-threaded pandas.
Is there a performance problem with groupby based on multiple columns?
Thanks a lot!
Contributor guide
Assessment
This issue has not been assessed yet.