Pivot rendering in the Koltin notebook plugin is slow
- Dominant language
- Kotlin
- Stars
- 1.1k
- Forks
- 83
- Avg merge
- 4d 12h
- Merged PRs (30d)
- 30
Description
While working with a relatively big dataset (~10m rows), I observed that rendering the table for a pivot result is extremely slow—about 10 seconds for a 10m rows dataframe.
After a brief investigation, I found out that the problem is with converting the pivot object to a dataframe. That is implemented like that `pivot.frames().toDataFrame()`
TODO: investigate what is the bottleneck and optimize the Pivot->Dataframe conversion process
Contributor guide
Research direction
Start at the Kotlin notebook plugin path that renders pivot results and inspect the `pivot.frames().toDataFrame()` conversion. Profile the conversion with a dataset near 10m rows to identify the bottleneck; done means pivot-to-dataframe rendering is measurably faster, though the issue names no specific files or tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- kotlin
- Domain
- data, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100