ipyflow / ipyflow/ipyflow

Upstream only cell execution feature (cmd+k) overwrites untouched cell downstream dependencies

Open
#160 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1.3k
Forks
25
PR merge metrics
No merged PRs in 30d

Description

Hi,

i noticed a (for me) unexpected behaviour of the upstream only cell execution feature "(cmd+k shortcut).
It also overwrites (removes) any downstream dependencies of the executed cell. This leads to in my eyes unexpected behaviour because, if one then executes anything downstream it does not execute the correct upstream dependencies.

  1. Step 1 - As an example lets take the simpel Notebook:

0_start

Step 1 ipflow Notebook metadata.json

  1. Step 2 - Change the x= cell but do not execute it. Execute the z= cell with cmd+k:

1_z_cmd_k

Looks good but in the metadata you can already see that the upstream depency for the last cell with deps is removed.

Step 2 ipflow Notebook metadata.json

  1. Step 3 - Change the x= cell but do not execute it again. Execute the y= cell with cmd+k:

2_y_cmd_k

Looks also good but in the metadata you can see that the upstream depency for the z= cell is removed.

Step 3 ipflow Notebook metadata.json

  1. Step 4 - Change the x= cell but do not execute it again. Execute the z= cell with cmd+k:

3_z_cmd_k

This is now quite unexpected. I would assume that again every z upstream cell (x and y) get executed but only x is actually executed because due to Step 3 z is no longer a downstream dependency of y.

Step 4 ipflow Notebook metadata.json

Subsequent normal executions (cmd+enter) seem to repair that problem again one downstream dependency at a time. So the first normal execution and y= is executed and a second normal execution => the last deps cell is also executed.

This behaviour is to some degree understandable because one does not know if z cell still exists, if one only executes until y but is there some way to preserve such downstream deps metadata from previous executions or execute/build the dependency graph of the whole notebook also for partial (cmd + k) executions ?

Ipyflow Version: 0.0.200
Jupyter versions:

Selected Jupyter core packages...
IPython          : 8.22.2
ipykernel        : 6.29.3
ipywidgets       : 8.1.2
jupyter_client   : 8.6.1
jupyter_core     : 5.7.2
jupyter_server   : 2.13.0
jupyterlab       : 4.1.5
nbclient         : 0.10.0
nbconvert        : 7.16.3
nbformat         : 5.10.3
notebook         : 7.1.2
qtconsole        : 5.5.1
traitlets        : 5.14.2

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the four-step cmd+k sequence from the issue using the attached ipflow_reset.json metadata and the simple x, y, and z cells. Compare dependency metadata after each partial execution and then after normal execution. Done means downstream dependencies that were not executed remain represented correctly and later executions select the expected upstream cells.

Written by the indexing model from the issue text.

Assessment

Tech stack
jupyter, jupyter-notebook, python
Domain
developer-experience
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.