bazel-contrib / bazel-contrib/rules_python

Some questions around `experimental_index_url` behavior

Open
#3,434 10 comments 0 reactions 0 assignees View on GitHub
need: discussion
Dominant language
Starlark
Stars
688
Forks
721
Avg merge
14h 49m
Merged PRs (30d)
81

Description

Opening a ticket to dump in all questions for triage, per suggestion of Slack message.

In this thread https://github.com/bazel-contrib/rules_python/issues/2949#issuecomment-3448011820 it was suggested that `experimental_index_url` would fix any issues I was seeing with `RULES_PYTHON_ENABLE_PIPSTAR=1`. That turned out to be true, but I saw a few things that I wanted clarification on.

### Setup

We use a "universal" lock-file generated via `uv` that looks like this:

```
triton==3.4.0 ; python_full_version < '3.13' and platform_machine == 'x86_64' and sys_platform == 'linux' \
...
```

and then in `pip.parse` we don't specify requirements_by_platform as a result:

```python
pip.parse(
enable_implicit_namespace_pkgs = True,
experimental_requirement_cycles = { ... bunch of cycles ... },
hub_name = "core_py_deps",
python_version = "3.12.3",
requirements_lock = "//:3rdparty_python_requirements.txt",
)
```

### Questions/Concerns from the switch

Switching allowed my `genquery` based tests to work alongside `pipstar`, but I was hoping to get clarification on the following:

* One of my tests had this line `r.Rlocation("rules_python++pip+core_py_deps_312_attrs/attrs-25.4.0-py3-none-any.whl")` which I had to switch to `r.Rlocation("rules_python++pip+core_py_deps_312_attrs_py3_none_any_adcf7e2a/attrs-25.4.0-py3-none-any.whl")`. I found the new path by manually `cquery`ing in CLI and am not sure why this is happening or what the idiomatic path forward is. Interestingly this new path works on both OSX and Linux.
* On the branch with `experimental_index_url`, I noticed it was downloading all the linux-only torch/cuda wheels. I don't *think* this happens normally, but I am not an expert. The build still works on Mac, but this is still a little concerning since those wheels are massive. Am I thinking about this the wrong way? I can confirm that in a different branch output_base, the `external` folder doesn't have non OSX wheels. But in the `experimental_index_url` branch, it does.
* `bazel query deps(...)` now fails because it is trying to install a Windows-only package (pywinpty). That package is in my universal lockfile with `os_name == 'nt'`

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.