Improve speed for Notebook tables by using comms instead of executing Kotlin for pagination
Nobody has claimed this yet.
- Dominant language
- Kotlin
- Stars
- 1.1k
- Forks
- 83
- Avg merge
- 4d 12h
- Merged PRs (30d)
- 30
Description
From @ileasile:
Now we have Jupyter comms used by two production integrations, which are properly covered with tests.
So I suggest you to look into this task: https://youtrack.jetbrains.com/issue/KTNB-428/Research-the-comms-API-for-the-Kotlin-Jupyter-kernel-in-order-to-replace-the-execution-of-raw-codeA bit of context: now for pagination we execute real Kotlin code. We do it for each pagination request. It adds extra delay which is generally unwanted.
Also, responses include a lot of unwanted things such as table HTML.To solve this, DataFrame could register special comm target which could respond to (dataframeId, range) requests with JSONs containing solely the data needed.
I'm ready to collaborate on it
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the linked KTNB-428 research task and review the existing Jupyter comm integrations and their tests. Determine how pagination currently executes Kotlin and how a comm target can handle dataframeId and range requests; done means pagination receives JSON containing only the requested data.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- jupyter, kotlin
- Domain
- data, performance
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100