Kotlin / Kotlin/dataframe

Pivot rendering in the Koltin notebook plugin is slow

Open
#674 0 comments 0 reactions 0 assignees View on GitHub
performance
Dominant language
Kotlin
Stars
1.1k
Forks
83
Avg merge
4d 12h
Merged PRs (30d)
30

Description

While working with a relatively big dataset (~10m rows), I observed that rendering the table for a pivot result is extremely slow—about 10 seconds for a 10m rows dataframe.
After a brief investigation, I found out that the problem is with converting the pivot object to a dataframe. That is implemented like that `pivot.frames().toDataFrame()`
TODO: investigate what is the bottleneck and optimize the Pivot->Dataframe conversion process

Contributor guide

Open the contributing guide

Research direction

Start at the Kotlin notebook plugin path that renders pivot results and inspect the `pivot.frames().toDataFrame()` conversion. Profile the conversion with a dataset near 10m rows to identify the bottleneck; done means pivot-to-dataframe rendering is measurably faster, though the issue names no specific files or tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
kotlin
Domain
data, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.