bazel-contrib / bazel-contrib/rules_python

py_zipapp duplicates shared libraries

Abierto
#3,857 0 comentarios 0 reacciones 0 asignados Ver en GitHub
Lenguaje dominante
Starlark
Estrellas
688
Forks
721
Merge medio
15 h 7 min
PR fusionados (30 d)
76

Descripción

# 🐞 bug report

### Affected Rule

The issue is caused by the rule: `py_zipapp_binary`

With the following settings

```
common --@rules_python//python/config_settings:bootstrap_impl=script
common --@rules_python//python/config_settings:venvs_site_packages=yes
common --@rules_python//python/config_settings:venvs_use_declare_symlink=yes
```

### Is this a regression?

No

### Description

Shared libraries that shipped in pip packages are duplicated in the zipapp, once in the venv path and once in the wheel run files path. For native heavy packages, such as torch, this can significantly increase the size of the resulting zip file.

This is related to the closed #3439, but there the issue was packaging the run files using third-party rules.

The run file layout has both paths as symlinks pointing to the real file, but when building the zip file one is created as a `symlink` and the other as a `root_symlink`. These are added separately into the zip manifest and don't get deduped `tools/private/zipapp/zipper.py` and instead get byte-copied into the zip file.

I would expect these to be deduped, so that only one copied into the zip file and the other becomes a symlink to that.

## 🔬 Minimal Reproduction

```python
# BUILD.bazel
load("@rules_python//python:py_binary.bzl", "py_binary")
load("@rules_python//python:py_zipapp_binary.bzl", "py_zipapp_binary")

py_binary(
name = "demo",
srcs = ["demo.py"],
deps = ["@pypi//grpcio"],
)

py_zipapp_binary(name = "demo_zip", binary = ":demo")
```

```bash
$ bazel build //:demo_zip
$ unzip -l bazel-bin/demo_zip.pyz | grep cygrpc
2419632 1980-01-01 00:00 runfiles/rules_python++pip+pypi_311_grpcio_.../site-packages/grpc/_cython/cygrpc.cpython-311-x86_64-linux-gnu.so
2419632 1980-01-01 00:00 runfiles/_main//_demo.venv/lib/python3.11/site-packages/grpc/_cython/cygrpc.cpython-311-x86_64-linux-gnu.so
```

## 🔥 Exception or Error

No exceptions or errors

Guía de contribución

Abrir la guía de contribución

Línea de trabajo

Comienza con tools/private/zipapp/zipper.py y la reproducción de py_zipapp_binary del issue. Ejecuta bazel build //:demo_zip, inspecciona el archivo resultante con unzip -l y compara las dos rutas de cygrpc. Se considera terminado cuando la shared library se copia una sola vez y la otra ruta se representa como un symlink.

Escrito por el modelo de indexación a partir del texto del issue.

Evaluación

Stack tecnológico
python
Área
build-system
Tipo de issue
Error
Dificultad
3/5
Tiempo estimado
1-2 días
Estado de actividad
Tranquilo
Claridad
Bastante claro
Aptitud para principiantes
68/100

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.