bazel-contrib / bazel-contrib/rules_python
Some questions around `experimental_index_url` behavior
- Dominant language
- Starlark
- Stars
- 688
- Forks
- 721
- Avg merge
- 15h 7m
- Merged PRs (30d)
- 76
Description
Opening a ticket to dump in all questions for triage, per suggestion of Slack message.
In this thread https://github.com/bazel-contrib/rules_python/issues/2949#issuecomment-3448011820 it was suggested that `experimental_index_url` would fix any issues I was seeing with `RULES_PYTHON_ENABLE_PIPSTAR=1`. That turned out to be true, but I saw a few things that I wanted clarification on.
### Setup
We use a "universal" lock-file generated via `uv` that looks like this:
```
triton==3.4.0 ; python_full_version < '3.13' and platform_machine == 'x86_64' and sys_platform == 'linux' \
...
```
and then in `pip.parse` we don't specify requirements_by_platform as a result:
```python
pip.parse(
enable_implicit_namespace_pkgs = True,
experimental_requirement_cycles = { ... bunch of cycles ... },
hub_name = "core_py_deps",
python_version = "3.12.3",
requirements_lock = "//:3rdparty_python_requirements.txt",
)
```
### Questions/Concerns from the switch
Switching allowed my `genquery` based tests to work alongside `pipstar`, but I was hoping to get clarification on the following:
* One of my tests had this line `r.Rlocation("rules_python++pip+core_py_deps_312_attrs/attrs-25.4.0-py3-none-any.whl")` which I had to switch to `r.Rlocation("rules_python++pip+core_py_deps_312_attrs_py3_none_any_adcf7e2a/attrs-25.4.0-py3-none-any.whl")`. I found the new path by manually `cquery`ing in CLI and am not sure why this is happening or what the idiomatic path forward is. Interestingly this new path works on both OSX and Linux.
* On the branch with `experimental_index_url`, I noticed it was downloading all the linux-only torch/cuda wheels. I don't *think* this happens normally, but I am not an expert. The build still works on Mac, but this is still a little concerning since those wheels are massive. Am I thinking about this the wrong way? I can confirm that in a different branch output_base, the `external` folder doesn't have non OSX wheels. But in the `experimental_index_url` branch, it does.
* `bazel query deps(...)` now fails because it is trying to install a Windows-only package (pywinpty). That package is in my universal lockfile with `os_name == 'nt'`
Contributor guide
Research direction
Start with the linked issue discussion and the pip.parse configuration using experimental_index_url, then compare the cquery and genquery behavior described for the universal uv lockfile. Investigate the generated Rlocation path and platform-specific wheel installation, including the pywinpty failure. Done means the behavior and an idiomatic configuration or documentation path are clarified for all three concerns.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- build-system
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100