bazel-contrib / bazel-contrib/rules_python
Packages shadowed by other modules are no longer importable under bzlmod
- 主要言語
- Starlark
- スター
- 688
- フォーク
- 721
- 平均マージ
- 15時間 7分
- マージ済み PR(30日)
- 76
説明
# 🐞 bug report
### Affected Rule
The issue is caused by the rule: `py_binary`/`py_library` under bzlmod
### Is this a regression?
Yes, this works under the WORKSPACE model. It is only bzlmod that is broken.
### Description
When two Bazel workspaces (with one depending on the other) both define packages with the same name (`common`, `core`, `helpers`, `private`, `util`, ...) the outer one will shadow the inner one. Without bzlmod this has always been solvable by using absolute imports with the workspace name as the prefix. However, under bzlmod, these absolute imports no longer work: when a module is used as an outer repository the prefix is `_main` but when a module is used as a dependency repository it is ``. And even if that was fine, there are further problems caused by `local_path_override` because it suffixes `~override` onto the folders in the runfiles directory.
From my perspective the best solution would be to make the generated `py_binary` wrapper script read the generated `_repo_mapping` file and set up the path such that absolute imports work the same way they've always worked. One way to make that happen would be to have the wrapper script create a temporary directory, create symlinks with the correct names pointing to the real folders, and then add the temporary directory to the PYTHONPATH. I could imagine there is probably some cleaner way I can't think of.
## 🔬 Minimal Reproduction
I put an example here: https://github.com/aschleck/bzlmod-shadow-python-problem (with another writeup in `README.md`.)
This repository illustrates three situations:
* [shadowed_with_workspace](https://github.com/aschleck/bzlmod-shadow-python-problem/tree/master/shadowed_with_workspace) shows a workspace depending on another workspace where both define a package named `core`. In this case, the `outer` workspace imports `dep` absolutely but `dep`'s import of itself fails due to being shadowed.
* [with_workspace](https://github.com/aschleck/bzlmod-shadow-python-problem/tree/master/with_workspace) illustrates the obvious fix: `dep` can always import itself with its workspace name as a prefix (like `import dep.core.`. Another solution would be to use relative imports, but those get unwieldy and annoying in large repos.
* [with_bzlmod](https://github.com/aschleck/bzlmod-shadow-python-problem/tree/master/with_bzlmod) fails due to `rules_python` not respecting the `_repo_mapping` file for imports. The absolute import of `dep` from `outer` fails because the folder in the runfiles is actually `dep~override` since `local_path_override` was used. Even if it was named `dep`, the code in `dep` itself could no longer assume that `import dep` refers to itself (because when its rules are run directly it is `_main`, and so it would need to do `import _main.core` not `import dep.core`.)
## 🔥 Exception or Error
The error is what you'd expect: the import fails.
~/bzlmod-shadow-python-problem/with_bzlmod/outer $ bazel run //core
INFO: Analyzed target //core:core (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
Target //core:core up-to-date:
bazel-bin/core/core
INFO: Elapsed time: 0.212s, Critical Path: 0.00s
INFO: 1 process: 1 internal.
INFO: Build completed successfully, 1 total action
INFO: Running command line: bazel-bin/core/core
imported outer.core.outer
Traceback (most recent call last):
File "/home/april/.cache/bazel/_bazel_april/1d28658223745047b82756f6e8e88a66/execroot/_main/bazel-out/k8-fastbuild/bin/core/core.runfiles/_main/core/outer.py", line 3, in
import dep.core.interface
ModuleNotFoundError: No module named 'dep'
## 🌍 Your Environment
**Operating System:**
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=22.04
DISTRIB_CODENAME=jammy
DISTRIB_DESCRIPTION="Ubuntu 22.04.2 LTS"
**Output of `bazel version`:**
Bazelisk version: v1.14.0
Build label: 6.1.1
Build target: bazel-out/k8-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Wed Mar 15 15:44:56 2023 (1678895096)
Build timestamp: 1678895096
Build timestamp as int: 1678895096
**Rules_python version:**
0.20.0
**Anything else relevant?**
Thank you for maintaining these rules! It's been amazing to see how well this all works on the whole.
コントリビューションガイド
調査の方向性
まず bzlmod 下での py_binary と py_library の動作を確認し、次に生成された py_binary ラッパースクリプトと生成された _repo_mapping ファイルを調べます。リンクされている bzlmod-shadow-python-problem の再現を実行し、特にその README と with_bzlmod の例を確認して、WORKSPACE と bzlmod の動作を比較します。shadowed packages と local_path_override repositories に対して absolute imports が機能すれば完了です。
索引モデルが issue の本文から書いたものです。
評価
- 技術スタック
- python
- 領域
- build-system
- issue の種類
- バグ
- 難易度
- 5/5
- 見積もり時間
- 1週間以上
- 活発さ
- 停滞
- 明瞭さ
- おおむね明確
- 初心者へのやさしさ
- 25/100