ipython / ipython/ipykernel

v6.3.0 introduced pkg_resources side effects

Đang mở
#824 1 bình luận 0 reaction 0 người được giao Xem trên GitHub
bug
Ngôn ngữ chính
Python
Star
734
Fork
412
Merge trung bình
1 ngày 5 giờ
Pull request đã merge (30 ngày)
8

Mô tả

`ipykernel` v6.3.0, I believe 470145ca7d103bdf6081bcfde09933ab81a56a5b specifically, introduced an indirect import on the `pkg_resources` module, via `debugpy.server.api`.

During this import, `debugpy` imports `pkg_resources` ([here](https://github.com/microsoft/debugpy/blob/b1ad4d66f1db9e2efac7d3df6a5fbb922d1435de/src/debugpy/_vendored/pydevd/pydevd_plugins/__init__.py#L5)).

The `pkg_resources` module has some [import-time side-effects](https://setuptools.pypa.io/en/latest/pkg_resources.html#workingset-objects) that cause problems in complex notebooks:
> It’s important to note that the global working_set object is initialized from sys.path **when pkg_resources is first imported**, but is only updated if you do all future sys.path manipulation via pkg_resources APIs.

I am filing this with `ipykernel` because I believe it's the place we start caring about the `sys.path` manipulations - it is somewhat common for notebooks to include `!pip install foo` steps, which impacts `sys.path` without using the `pkg_resources` APIs. However, I'd like to discuss where a fix _should_ live.

I'm part of the TensorFlow docs team, and we have an increasing number of notebooks that have a "restart runtime" requirement ([e.g.](https://colab.research.google.com/github/tensorflow/tfx/blob/master/docs/tutorials/data_validation/tfdv_basic.ipynb#scrollTo=s_NXX5GaSiZx)). I don't think fixing every notebook is the right solution, and we could apply a fix in our own infrastructure, but I would favour a fix at the root, if possible.

When we fix it in the notebook, we reload the module ([e.g.](https://colab.research.google.com/github/tensorflow/quantum/blob/master/docs/tutorials/mnist.ipynb#scrollTo=4Ql5PW-ACO0J&line=3&uniqifier=1)), however we should unload the module completely if we fix this before any notebook code, in case the notebook uses `pkg_resources`. If we were to do this, say, in `ipykernel`, would it impact the notebook kernel or other environment factors? e.g. in the use of pydevd or debugging?

In case it helps, [this notebook](https://gist.github.com/markmcd/04ee48ec7d4058c57ab6c40858cf4ddf) can be used to reproduce the failure. `ipykernel<6.3.0` works, but `ipykernel==6.3.0` (and at head) fail.

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.