ipython / ipython/ipykernel

Kernel crashes randomly: `IndexError: pop from an empty deque`

未關閉
#1,250 2 則留言 1 個 reaction 已指派 0 人 在 GitHub 檢視
主要語言
Python
星號
734
分支
412
平均合併
1 天 5 小時
30 天內合併 PR
8

描述

## Description

Hello,
We encounter a random error which occurs from time to time on our shared jupyterhub instance.
After some time of running many jupyterlab notebooks, the kernel tends the crash always at the same cell of a notebook(which changes from a crash occurrence to another). Note that the same cell works in other runs, and that the error does not occur on the same cell or cell content after the jupyterhub is restarted and the error happens again.
Here is the stacktrace we get:
```
Traceback (most recent call last):
File "/srv/envs/mambaforge/envs/experiments-2024-06-03/lib/python3.9/runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/srv/envs/mambaforge/envs/experiments-2024-06-03/lib/python3.9/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/srv/envs/mambaforge/envs/experiments-2024-06-03/lib/python3.9/site-packages/ipykernel_launcher.py", line 18, in
app.launch_new_instance()
File "/srv/envs/mambaforge/envs/experiments-2024-06-03/lib/python3.9/site-packages/traitlets/config/application.py", line 1075, in launch_instance
app.start()
File "/srv/envs/mambaforge/envs/experiments-2024-06-03/lib/python3.9/site-packages/ipykernel/kernelapp.py", line 739, in start
self.io_loop.start()
File "/srv/envs/mambaforge/envs/experiments-2024-06-03/lib/python3.9/site-packages/tornado/platform/asyncio.py", line 205, in start
self.asyncio_loop.run_forever()
File "/srv/envs/mambaforge/envs/experiments-2024-06-03/lib/python3.9/asyncio/base_events.py", line 601, in run_forever
self._run_once()
File "/srv/envs/mambaforge/envs/experiments-2024-06-03/lib/python3.9/asyncio/base_events.py", line 1890, in _run_once
handle = self._ready.popleft()
IndexError: pop from an empty deque
```

To overcome the error, we need to restart the jupyterhub instance thus interrupting everybody's work.

## Reproduce

1. Open a notebook
2. Run a series of cells
3. A popup appears telling that the kernel has crashed and need to be restarted

## Expected behavior

The kernel does not crash and all cells are run successfully.

## Context

- Operating System and version: Debian 11 (bullseye)
- Browser and version: Chrome 125, firefox 125
- JupyterLab version: 4.1.2

For reference, here are all jupyter-related packages that are used to build our jupyterhub instance
```
jupyter-client==8.6.0 \
--hash=sha256:0642244bb83b4764ae60d07e010e15f0e2d275ec4e918a8f7b80fbbef3ca60c7 \
--hash=sha256:909c474dbe62582ae62b758bca86d6518c85234bdee2d908c778db6d72f39d99
# via
# ipykernel
# jupyter-server
# nbclient
jupyter-core==5.7.1 \
--hash=sha256:c65c82126453a723a2804aa52409930434598fd9d35091d63dfb919d2b765bb7 \
--hash=sha256:de61a9d7fc71240f688b2fb5ab659fbb56979458dc66a71decd098e03c79e218
# via
# ipykernel
# jupyter-client
# jupyter-server
# jupyterlab
# nbclient
# nbconvert
# nbformat
jupyter-events==0.9.0 \
--hash=sha256:81ad2e4bc710881ec274d31c6c50669d71bbaa5dd9d01e600b56faa85700d399 \
--hash=sha256:d853b3c10273ff9bc8bb8b30076d65e2c9685579db736873de6c2232dde148bf
# via jupyter-server
jupyter-lsp==2.2.2 \
--hash=sha256:256d24620542ae4bba04a50fc1f6ffe208093a07d8e697fea0a8d1b8ca1b7e5b \
--hash=sha256:3b95229e4168355a8c91928057c1621ac3510ba98b2a925e82ebd77f078b1aa5
# via
# jupyterlab
# jupyterlab-lsp
jupyter-resource-usage==1.0.1 \
--hash=sha256:d722ad32fc8bfaff3f81da4f8a2202c5e5258895d546399ff0e5ddf11f56bd8e \
--hash=sha256:ede723ebb63d531615d0da4f8769470c8ee9cc3fa18e6af9a12b70f11972cc61
# via -r requirements.txt
jupyter-server==2.12.5 \
--hash=sha256:0edb626c94baa22809be1323f9770cf1c00a952b17097592e40d03e6a3951689 \
--hash=sha256:184a0f82809a8522777cfb6b760ab6f4b1bb398664c5860a27cec696cb884923
# via
# jupyter-lsp
# jupyter-resource-usage
# jupyterlab
# jupyterlab-server
# notebook
# notebook-shim
jupyter-server-terminals==0.5.2 \
--hash=sha256:1b80c12765da979513c42c90215481bbc39bd8ae7c0350b4f85bc3eb58d0fa80 \
--hash=sha256:396b5ccc0881e550bf0ee7012c6ef1b53edbde69e67cab1d56e89711b46052e8
# via jupyter-server
jupyter-telemetry==0.1.0 \
--hash=sha256:1de3e423b23aa40ca4a4238d65c56dda544061ff5aedc3f7647220ed7e3b9589 \
--hash=sha256:445c613ae3df70d255fe3de202f936bba8b77b4055c43207edf22468ac875314
# via jupyterhub
jupyterhub==4.0.2 \
--hash=sha256:2f389e7d3067e1b11bb4091719048eedecee161039fd2e5b025d031f7ab23c62 \
--hash=sha256:d4e450eed8d90dfbcf0eca08f00f2093a0bce74dc51f7cfc0b7057f602341a1d
# via
# -r requirements.txt
# jupyterhub-traefik-proxy
# sudospawner
jupyterhub-traefik-proxy==1.1.0 \
--hash=sha256:6972d8eb788274d1d541c11cc10764d94a6829843bdb15873b60e19519bd4d5c \
--hash=sha256:9e67eb354165b9400fb5a01c4272b4c2073dbd81fbbcd4e766ea7a62fac12cde
# via -r requirements.txt
jupyterlab==4.1.2 \
--hash=sha256:5d6348b3ed4085181499f621b7dfb6eb0b1f57f3586857aadfc8e3bf4c4885f9 \
--hash=sha256:aa88193f03cf4d3555f6712f04d74112b5eb85edd7d222c588c7603a26d33c5b
# via
# -r requirements.txt
# jupyterlab-lsp
# notebook
jupyterlab-lsp==5.0.3 \
--hash=sha256:1a1c96f60202e49c28538f40b3ee19487d490a338bed88c95368073412689e22 \
--hash=sha256:a9c8a3a646494be484d7e3174ab1d2415ed60f4773cec8479aa5ba37e5f04bc9
# via -r requirements.txt
jupyterlab-pygments==0.3.0 \
--hash=sha256:721aca4d9029252b11cfa9d185e5b5af4d54772bb8072f9b7036f4170054d35d \
--hash=sha256:841a89020971da1d8693f1a99997aefc5dc424bb1b251fd6322462a1b8842780
# via nbconvert
jupyterlab-server==2.25.3 \
--hash=sha256:846f125a8a19656611df5b03e5912c8393cea6900859baa64fa515eb64a8dc40 \
--hash=sha256:c48862519fded9b418c71645d85a49b2f0ec50d032ba8316738e9276046088c1
# via
# jupyterlab
# notebook
jupyterlab-skip-traceback==5.0.0 \
--hash=sha256:a1091b639850ef3b614b534816f38647c41a2519b4bf47ca7e7585770fb3dd9e \
--hash=sha256:dd1acf9150c951150c6de24ebc6c3247bb7788971671158cc0915fd3310ed75b
# via -r requirements.txt
jupyterlab-widgets==3.0.10 \
```

Troubleshoot Output

Paste the output from running `jupyter troubleshoot` from the command line here.

You may want to sanitize the paths in the output.

Command Line Output

Paste the output from your command line running `jupyter lab` here, use `--debug` if possible.

Browser Output

Paste the output from your browser Javascript console here, if applicable.

貢獻指南

開啟貢獻指南

研究方向

Start with the traceback in Python's asyncio/base_events.py, then inspect the ipykernel entry points in ipykernel_launcher.py and kernelapp.py. Try to reproduce the intermittent crash by running notebook cells repeatedly in the listed JupyterLab 4.1.2 environment. Done means identifying why the kernel reaches an empty ready deque and preventing the kernel crash without requiring a JupyterHub restart.

由索引模型根據 Issue 內容生成。

評估

技術堆疊
jupyter-notebook, python
領域
backend
Issue 類型
缺陷
難度
4/5
預估耗時
3-5 天
活躍度
停滯
描述清晰度
需要釐清
新手友好度
25/100

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。