bazel-contrib / bazel-contrib/rules_python

Invalid wheel metadata when a direct-URL requirement has an environment marker

Open
#4,158 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Starlark
Stars
688
Forks
721
Avg merge
15h 7m
Merged PRs (30d)
76

Description

# 🐞 bug report

### Affected Rule

The issue is caused by the rule: `py_wheel` (specifically `get_new_requirement_line` in `tools/wheelmaker.py`, and the `extra_requires` handling in `python/private/py_wheel.bzl`)

### Is this a regression?

No — the bug was introduced together with URL-requirement support in wheelmaker (PR #3569). Direct-URL requirements with environment markers have never produced valid wheel metadata.

### Description

When a `py_wheel` `requires` (or a line in `requires_file`) is a PEP 508 **direct-URL requirement with an environment marker**, e.g.

```
somepkg @ https://example.com/wheels/somepkg-1.0-none-none-linux_x86_64.whl ; sys_platform == 'linux'
```

wheelmaker re-serializes the requirement in [`get_new_requirement_line`](https://github.com/bazel-contrib/rules_python/blob/2.3.3/tools/wheelmaker.py#L377) as:

```python
base = f"Requires-Dist: {req.name}{req_extra_deps}{req_spec}" # ends with the URL
...
return f"{base}; {req.marker}" # no whitespace before ';'
```

producing a METADATA line without whitespace before the `;` marker separator:

```
Requires-Dist: somepkg @ https://example.com/wheels/somepkg-1.0-none-none-linux_x86_64.whl; sys_platform == 'linux'
```

That is **invalid PEP 508**. Per the spec, whitespace after a URL is mandatory since `;` is a valid URI character (`sub-delims`), so a strict parser greedily consumes `whl;` into the URL and then fails. `pip` rejects the wheel at install time:

```
ERROR: Requested urlmarker==0.1 from file:///.../urlmarker-0.1-py3-none-any.whl has invalid metadata: Expected semicolon (after URL and whitespace) or end
somepkg @ https://example.com/wheels/somepkg-1.0-none-none-linux_x86_64.whl; sys_platform == 'linux'
```

Note that wheelmaker strips extra whitespace via `Requirement(...)` round-tripping, so there is **no way to emit valid metadata for URL+marker requirements through `py_wheel`** without patching wheelmaker.

For non-URL requirements (`stim~=1.16.0; sys_platform != 'linux'`) the missing space happens to be harmless since the grammar allows `version_end wsp* ';'`, so this only breaks direct URLs.

## 🔬 Minimal Reproduction

```
# BUILD.bazel
load("@rules_python//python:packaging.bzl", "py_wheel")

py_wheel(
name = "wheel",
distribution = "urlmarker",
version = "0.1",
requires = [
"somepkg @ https://example.com/wheels/somepkg-1.0-none-none-linux_x86_64.whl ; sys_platform == 'linux'",
],
)
```

```bash
bazel build //:wheel
pip install --dry-run bazel-bin/urlmarker-0.1-py3-none-any.whl
# => ERROR: ... has invalid metadata: Expected semicolon (after URL and whitespace) or end
```

(The URL does not need to resolve — metadata validation fails before pip ever fetches it.)

## 🔥 Exception or Error



ERROR: Requested urlmarker==0.1 from file:///.../urlmarker-0.1-py3-none-any.whl has invalid metadata: Expected semicolon (after URL and whitespace) or end
somepkg @ https://example.com/wheels/somepkg-1.0-cp312-cp312-macosx_15_0_arm64.whl; sys_platform == 'darwin'

## Suggested fix

Emit a space before `;` in all branches of `get_new_requirement_line` — valid for every requirement type and required for URLs:

```diff
if req.marker:
if extra:
- return f"{base}; ({req.marker}) and {extra}"
+ return f"{base} ; ({req.marker}) and {extra}"
else:
- return f"{base}; {req.marker}"
+ return f"{base} ; {req.marker}"
elif extra:
- return f"{base}; {extra}"
+ return f"{base} ; {extra}"
```

(Note `extra` may itself be a marker expression such as `extra == 'cuda'`, and `base` may end in a URL here too when `extra_requires` entries use direct URLs — see below.)

## 🌍 Your Environment

**Operating System:**

  

Ubuntu (Linux x86_64)

**Output of `bazel version`:**

  

Bazelisk version: v1.29.0
Build label: 9.2.0
Build target: @@//src/main/java/com/google/devtools/build/lib/bazel:BazelServer
Build time: Mon Jul 13 18:15:04 2026 (1783966504)
Build timestamp: 1783966504
Build timestamp as int: 1783966504

**Rules_python version:**

  

2.3.3 (still present on main at the time of writing)

**Anything else relevant?**

1. The `extra_requires` path in [`python/private/py_wheel.bzl`](https://github.com/bazel-contrib/rules_python/blob/main/python/private/py_wheel.bzl#L474) has the same missing-space problem: it emits

```python
metadata_contents.append(
"Requires-Dist: %s; extra == '%s'" % (requirement, option),
)
```

If an extra's requirement is a direct URL (e.g. `"somepkg @ https://example.com/pkg.whl"` under `extra_requires = {"cuda": [...]}`), this produces `...pkg.whl; extra == 'cuda'` — the same invalid metadata. These lines pass through `get_new_requirement_line` in wheelmaker (which fixes them if the suggested fix above is applied, since the URL ends up in `base` and `extra == '...'` becomes `extra`), but only because `rpartition(";")` splits off the extra marker; the raw `"%s; extra == '%s'"` formatting should arguably also include the space for consistency.

2. `pip`/`uv` accept `name @ url; marker` in requirements *files*, which masks the issue during development — the failure only surfaces when installing the built wheel, because wheel metadata is validated strictly.

Contributor guide

Open the contributing guide

Research direction

Start in tools/wheelmaker.py at get_new_requirement_line and inspect the extra_requires handling in python/private/py_wheel.bzl. Reproduce the issue with the provided BUILD.bazel example and pip install --dry-run; done means generated wheel metadata accepts direct-URL requirements with environment markers, including extra_requires entries.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
build-system
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.