v6.3.0 introduced pkg_resources side effects
- Ngôn ngữ chính
- Python
- Star
- 734
- Fork
- 412
- Merge trung bình
- 1 ngày 5 giờ
- Pull request đã merge (30 ngày)
- 8
Mô tả
`ipykernel` v6.3.0, I believe 470145ca7d103bdf6081bcfde09933ab81a56a5b specifically, introduced an indirect import on the `pkg_resources` module, via `debugpy.server.api`.
During this import, `debugpy` imports `pkg_resources` ([here](https://github.com/microsoft/debugpy/blob/b1ad4d66f1db9e2efac7d3df6a5fbb922d1435de/src/debugpy/_vendored/pydevd/pydevd_plugins/__init__.py#L5)).
The `pkg_resources` module has some [import-time side-effects](https://setuptools.pypa.io/en/latest/pkg_resources.html#workingset-objects) that cause problems in complex notebooks:
> It’s important to note that the global working_set object is initialized from sys.path **when pkg_resources is first imported**, but is only updated if you do all future sys.path manipulation via pkg_resources APIs.
I am filing this with `ipykernel` because I believe it's the place we start caring about the `sys.path` manipulations - it is somewhat common for notebooks to include `!pip install foo` steps, which impacts `sys.path` without using the `pkg_resources` APIs. However, I'd like to discuss where a fix _should_ live.
I'm part of the TensorFlow docs team, and we have an increasing number of notebooks that have a "restart runtime" requirement ([e.g.](https://colab.research.google.com/github/tensorflow/tfx/blob/master/docs/tutorials/data_validation/tfdv_basic.ipynb#scrollTo=s_NXX5GaSiZx)). I don't think fixing every notebook is the right solution, and we could apply a fix in our own infrastructure, but I would favour a fix at the root, if possible.
When we fix it in the notebook, we reload the module ([e.g.](https://colab.research.google.com/github/tensorflow/quantum/blob/master/docs/tutorials/mnist.ipynb#scrollTo=4Ql5PW-ACO0J&line=3&uniqifier=1)), however we should unload the module completely if we fix this before any notebook code, in case the notebook uses `pkg_resources`. If we were to do this, say, in `ipykernel`, would it impact the notebook kernel or other environment factors? e.g. in the use of pydevd or debugging?
In case it helps, [this notebook](https://gist.github.com/markmcd/04ee48ec7d4058c57ab6c40858cf4ddf) can be used to reproduce the failure. `ipykernel<6.3.0` works, but `ipykernel==6.3.0` (and at head) fail.
Hướng dẫn đóng góp
Đánh giá
Issue này chưa được đánh giá.