bazel-contrib / bazel-contrib/rules_python
Some questions around `experimental_index_url` behavior
- Vorherrschende Sprache
- Starlark
- Sterne
- 688
- Forks
- 721
- Ø Merge
- 15 Std. 7 Min.
- Gemergte PRs (30 T.)
- 76
Beschreibung
Opening a ticket to dump in all questions for triage, per suggestion of Slack message.
In this thread https://github.com/bazel-contrib/rules_python/issues/2949#issuecomment-3448011820 it was suggested that `experimental_index_url` would fix any issues I was seeing with `RULES_PYTHON_ENABLE_PIPSTAR=1`. That turned out to be true, but I saw a few things that I wanted clarification on.
### Setup
We use a "universal" lock-file generated via `uv` that looks like this:
```
triton==3.4.0 ; python_full_version < '3.13' and platform_machine == 'x86_64' and sys_platform == 'linux' \
...
```
and then in `pip.parse` we don't specify requirements_by_platform as a result:
```python
pip.parse(
enable_implicit_namespace_pkgs = True,
experimental_requirement_cycles = { ... bunch of cycles ... },
hub_name = "core_py_deps",
python_version = "3.12.3",
requirements_lock = "//:3rdparty_python_requirements.txt",
)
```
### Questions/Concerns from the switch
Switching allowed my `genquery` based tests to work alongside `pipstar`, but I was hoping to get clarification on the following:
* One of my tests had this line `r.Rlocation("rules_python++pip+core_py_deps_312_attrs/attrs-25.4.0-py3-none-any.whl")` which I had to switch to `r.Rlocation("rules_python++pip+core_py_deps_312_attrs_py3_none_any_adcf7e2a/attrs-25.4.0-py3-none-any.whl")`. I found the new path by manually `cquery`ing in CLI and am not sure why this is happening or what the idiomatic path forward is. Interestingly this new path works on both OSX and Linux.
* On the branch with `experimental_index_url`, I noticed it was downloading all the linux-only torch/cuda wheels. I don't *think* this happens normally, but I am not an expert. The build still works on Mac, but this is still a little concerning since those wheels are massive. Am I thinking about this the wrong way? I can confirm that in a different branch output_base, the `external` folder doesn't have non OSX wheels. But in the `experimental_index_url` branch, it does.
* `bazel query deps(...)` now fails because it is trying to install a Windows-only package (pywinpty). That package is in my universal lockfile with `os_name == 'nt'`
Beitragsleitfaden
Rechercherichtung
Start with the linked issue discussion and the pip.parse configuration using experimental_index_url, then compare the cquery and genquery behavior described for the universal uv lockfile. Investigate the generated Rlocation path and platform-specific wheel installation, including the pywinpty failure. Done means the behavior and an idiomatic configuration or documentation path are clarified for all three concerns.
Vom Indexierungsmodell aus dem Issue-Text verfasst.
Bewertung
- Tech-Stack
- python
- Bereich
- build-system
- Issue-Typ
- Bug
- Schwierigkeit
- 5/5
- Geschätzter Aufwand
- Über eine Woche
- Aktivitätsstatus
- Veraltet
- Klarheit
- Muss geklärt werden
- Anfängerfreundlichkeit
- 25/100