jupyterlab / jupyterlab/benchmarks

Proposal: Jupyter should be able to handle large notebooks

Open
#75 2 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Jupyter Notebook
Stars
12
Forks
16
PR merge metrics
No merged PRs in 30d

Description

### Problem

We should add a benchmark test and make changes so that 2k cell notebooks feels good to work with. In practice, I have seen some users make notebook in the 1k-ish range, so 2k is an arbitrary number that is bigger than that (maybe it should be 10k?).

We'd first need to define "feels good to work with" a bit more, which I'll state as something like:
1. Allows a user to interact with it in no more than 10s seconds
2. Clicking on cells within the notebook become interactive as fast as a 10 cell notebook
3. Characters typed in code cells are rendered as fast as a 10 cell notebook
4. Switching tabs should be no more than 20% slower than with a 10 cell notebook
5. Scrolling/jumping to a cell (e.g. via ToC) should be interactive in less than 500ms
6. It does not significantly interfere with the rest of the page (e.g. button clicks takes no more than 20% more than if the notebook was not on the page)

All the numbers and metrics above are just somewhere to get started. Happy to put in other metrics and/or change any of the numbers as I choose them somewhat arbitrarily as well. That being said, today none of these metrics pass.

__________________
### What is is like today?

Given this generated notebook (note, there is no output for any cell which makes this simpler than in the real world):
```
import json
import nbformat

NUM_CELLS = 2000

nb = nbformat.v4.new_notebook()
nb.metadata.kernelspec = {
"display_name": "Python 3",
"language": "python",
"name": "python3",
}
for n in range(NUM_CELLS):
nb.cells.append(nbformat.v4.new_code_cell("# cell {}".format(n + 1)))

with open(
"generated-{}cells.ipynb".format(NUM_CELLS),
"w",
) as f:
f.write(json.dumps(nb, indent=4))
```

In lab 3.1 I am finding the following performance when I open the above notebook and try to use it:
![image](https://user-images.githubusercontent.com/1813603/139144179-a44c706c-607e-470d-938e-063e36bca2ea.png)

Zooming in, all the work seems to be this codemirror pattern over and over again:
![image](https://user-images.githubusercontent.com/1813603/139144264-6b0d05ea-efd6-4eb2-ba9e-35477efdb437.png)

While we are working on things like https://github.com/jupyterlab/jupyterlab/issues/10370 and https://github.com/jupyterlab/lumino/pull/231 I thought it would be good to set a both a bit more defined goal and give everyone the same example to test against.

@blois, I'm curious how this notebook performs with the new colab virtualization (https://github.com/jupyterlab/benchmarks/issues/68#issuecomment-953156215).

What do others think?

CC those who have come to performance meetings as this size notebook was a topic of our first meeting. @fcollonval @sagemaster @echarles @Zsailer @jasongrout @afshin @ellisonbg @3coins @goanpeca

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by generating and opening the described 2,000-cell notebook in JupyterLab 3.1, then profile the repeated CodeMirror work during opening and interaction. Review the related JupyterLab issue and Lumino pull request, and clarify the performance metrics before implementing changes. Done means an agreed benchmark exists and the notebook meets the selected interaction, typing, tab-switching, and scrolling targets.

Written by the indexing model from the issue text.

Assessment

Tech stack
jupyter-notebook
Domain
performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.