aspect-build / aspect-build/rules_py

[Bug]: avoid downloading foreign whl while running gazelle

Open
#1,271 3 comments 0 reactions 1 assignee Claimed by @xangcastle View on GitHub
bug
Dominant language
Starlark
Stars
145
Forks
97
Avg merge
1d 1h
Merged PRs (30d)
71

Description

### What happened?

Hello,
https://github.com/aspect-build/rules_py/blob/31e86164f67d0c224d93654f18465c13205de9eb/uv/private/whl_install/repository.bzl#L608

```python
if prebuilds:
gazelle_index_whl = prebuilds.values()[0] # Effectively random choice :shrug:
elif default_target:
```

this picks random wheel from the index, for example download a win32 whl on linux, which can lead to unnecessary downloads. Would it make more sense to try the host-architecture wheel first, since it is very likely already downloaded for execution?

I created a prototype something like

```python
def pick_gazelle_index_whl(repository_ctx, prebuilds):
"""Pick a wheel for Gazelle indexing without fetching foreign platforms.

Any wheel of the package works for indexing, but prefer an "any" wheel,
then a wheel matching the host platform: the build/test is very likely to
download that host-platform wheel already, so reusing it avoids fetching a
large foreign-platform wheel purely for indexing.
"""
bazel_os = repository_ctx.os.name
bazel_arch = repository_ctx.os.arch

# repository_ctx os:
# name = linux, arch = amd64
# name = linux, arch = aarch64
# name = "mac os x", arch = aarch64
#
# pip platform_tags (from uv.lock):
# [musllinux_1_2_x86_64]
# [linux_aarch64, manylinux_2_17_aarch64]
# [linux_x86_64, manylinux_2_17_x86_64]
# [win_amd64],
# [win_arm64]
# [macosx_10_9_x86_64],
# [macosx_11_0_universal2],
# [macosx_11_0_arm64]
#
#
# note: only linux amd64/aarch64 and osx aarch64 are handled here.
for whl, target in prebuilds.items():
parsed = parse_whl_name(whl)
for tag in parsed.platform_tags:
if tag == "any":
return target
elif (bazel_os == "linux") and any([tag.startswith(v) for v in ["linux", "manylinux", "musllinux"]]):
pip_arch = {"amd64": "x86_64"}.get(bazel_arch, bazel_arch)
if tag.endswith("_" + pip_arch):
return target
elif (bazel_os == "mac os x") and tag.startswith("macosx"):
if tag.endswith("universal2") or tag.endswith("arm64"):
return target
return prebuilds.values()[0]

```

### Version

Development (host) and target OS/architectures:

linux/x86_64

Output of `bazel --version`:

bazel 8.5.1

Version of the Aspect rules, or other relevant rules from your
`WORKSPACE` or `MODULE.bazel` file:

Language(s) and/or frameworks involved:

python

### How to reproduce

run gazelle_python_manifest on linux 86_64, it will download aarch64 osx/win32 whl files

### Any other information?

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.