bazel-contrib / bazel-contrib/rules_python
py_zipapp duplicates shared libraries
- 主要言語
- Starlark
- スター
- 688
- フォーク
- 721
- 平均マージ
- 15時間 7分
- マージ済み PR(30日)
- 76
説明
# 🐞 bug report
### Affected Rule
The issue is caused by the rule: `py_zipapp_binary`
With the following settings
```
common --@rules_python//python/config_settings:bootstrap_impl=script
common --@rules_python//python/config_settings:venvs_site_packages=yes
common --@rules_python//python/config_settings:venvs_use_declare_symlink=yes
```
### Is this a regression?
No
### Description
Shared libraries that shipped in pip packages are duplicated in the zipapp, once in the venv path and once in the wheel run files path. For native heavy packages, such as torch, this can significantly increase the size of the resulting zip file.
This is related to the closed #3439, but there the issue was packaging the run files using third-party rules.
The run file layout has both paths as symlinks pointing to the real file, but when building the zip file one is created as a `symlink` and the other as a `root_symlink`. These are added separately into the zip manifest and don't get deduped `tools/private/zipapp/zipper.py` and instead get byte-copied into the zip file.
I would expect these to be deduped, so that only one copied into the zip file and the other becomes a symlink to that.
## 🔬 Minimal Reproduction
```python
# BUILD.bazel
load("@rules_python//python:py_binary.bzl", "py_binary")
load("@rules_python//python:py_zipapp_binary.bzl", "py_zipapp_binary")
py_binary(
name = "demo",
srcs = ["demo.py"],
deps = ["@pypi//grpcio"],
)
py_zipapp_binary(name = "demo_zip", binary = ":demo")
```
```bash
$ bazel build //:demo_zip
$ unzip -l bazel-bin/demo_zip.pyz | grep cygrpc
2419632 1980-01-01 00:00 runfiles/rules_python++pip+pypi_311_grpcio_.../site-packages/grpc/_cython/cygrpc.cpython-311-x86_64-linux-gnu.so
2419632 1980-01-01 00:00 runfiles/_main//_demo.venv/lib/python3.11/site-packages/grpc/_cython/cygrpc.cpython-311-x86_64-linux-gnu.so
```
## 🔥 Exception or Error
No exceptions or errors
コントリビューションガイド
調査の方向性
tools/private/zipapp/zipper.py と issue 内の py_zipapp_binary reproduction から始めます。bazel build //:demo_zip を実行し、生成されたアーカイブを unzip -l で調べ、2 つの cygrpc パスを比較します。shared library が 1 回だけコピーされ、もう一方のパスが symlink として表現されれば完了です。
索引モデルが issue の本文から書いたものです。
評価
- 技術スタック
- python
- 領域
- build-system
- issue の種類
- バグ
- 難易度
- 3/5
- 見積もり時間
- 1〜2日
- 活発さ
- 静か
- 明瞭さ
- おおむね明確
- 初心者へのやさしさ
- 68/100