bazel-contrib / bazel-contrib/rules_python

Some questions around `experimental_index_url` behavior

Open
#3,434 10 comments 0 reactions 0 assignees View on GitHub
need: discussion
Dominant language
Starlark
Stars
688
Forks
721
Avg merge
15h 7m
Merged PRs (30d)
76

Description

Opening a ticket to dump in all questions for triage, per suggestion of Slack message.

In this thread https://github.com/bazel-contrib/rules_python/issues/2949#issuecomment-3448011820 it was suggested that `experimental_index_url` would fix any issues I was seeing with `RULES_PYTHON_ENABLE_PIPSTAR=1`. That turned out to be true, but I saw a few things that I wanted clarification on.

### Setup

We use a "universal" lock-file generated via `uv` that looks like this:

```
triton==3.4.0 ; python_full_version < '3.13' and platform_machine == 'x86_64' and sys_platform == 'linux' \
...
```

and then in `pip.parse` we don't specify requirements_by_platform as a result:

```python
pip.parse(
enable_implicit_namespace_pkgs = True,
experimental_requirement_cycles = { ... bunch of cycles ... },
hub_name = "core_py_deps",
python_version = "3.12.3",
requirements_lock = "//:3rdparty_python_requirements.txt",
)
```

### Questions/Concerns from the switch

Switching allowed my `genquery` based tests to work alongside `pipstar`, but I was hoping to get clarification on the following:

* One of my tests had this line `r.Rlocation("rules_python++pip+core_py_deps_312_attrs/attrs-25.4.0-py3-none-any.whl")` which I had to switch to `r.Rlocation("rules_python++pip+core_py_deps_312_attrs_py3_none_any_adcf7e2a/attrs-25.4.0-py3-none-any.whl")`. I found the new path by manually `cquery`ing in CLI and am not sure why this is happening or what the idiomatic path forward is. Interestingly this new path works on both OSX and Linux.
* On the branch with `experimental_index_url`, I noticed it was downloading all the linux-only torch/cuda wheels. I don't *think* this happens normally, but I am not an expert. The build still works on Mac, but this is still a little concerning since those wheels are massive. Am I thinking about this the wrong way? I can confirm that in a different branch output_base, the `external` folder doesn't have non OSX wheels. But in the `experimental_index_url` branch, it does.
* `bazel query deps(...)` now fails because it is trying to install a Windows-only package (pywinpty). That package is in my universal lockfile with `os_name == 'nt'`

Contributor guide

Open the contributing guide

Research direction

Start with the linked issue discussion and the pip.parse configuration using experimental_index_url, then compare the cquery and genquery behavior described for the universal uv lockfile. Investigate the generated Rlocation path and platform-specific wheel installation, including the pywinpty failure. Done means the behavior and an idiomatic configuration or documentation path are clarified for all three concerns.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
build-system
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.