[FR] pip options in extra_requires

setuptools version

setuptools==77.0.3

Python version

Python 3.9

OS

Void Linux

Additional environment information

packaging==24.2

pip==25.0.1

pip-tools==7.4.1

Description

When hosting .whl locally or on a warehouse, besides pypi.org, how to inform pip where to search for those packages:

1. cli provided options --find-links --extra-index-url or --index-url

2. $HOME/.pip/pip-conf

[global]
extra-index-url = http://localhost:8080/simple/

3. Specified within each and every requirements.txt file

When .whl are hosted locally or on non-pypi.org warehouses, pip-compile will insert lines looking like,

--extra-index-url http://localhost:8080/simple/

setuptools.dist.Distribution._normalize_requires does not filter out those lines!

Then packaging.requirements.Requirement.parse does not recognize those lines as packages.

other reports

wreck#30

packaging#884

Expected behavior

setuptools.dist.Distribution._normalize_requires, filter out (sourced from config file) lines that start with --find-links --extra-index-url or --index-url.

Avoiding packaging.requirements.Requirement raising a packaging.requirements.InvalidRequirement exception.

How to Reproduce

1. setup pypiserver service locally, including $HOME/.pip/pip-conf file

[global]
extra-index-url = http://localhost:8080/simple/

1. pip-compile -o requirements.in requirements.txt

requirements.in

# available on pypi.org
packaging

# not available on pypi.org Available locally or on another remote warehouse
packagenotonpypiorg

requirements.txt

--extra-index-url http://localhost:8080/simple/

# available on pypi.org
packaging

# not available on pypi.org Available locally or on another remote warehouse
packagenotonpypiorg

2. pyproject.toml

[build-system]
requires = ["setuptools>=77.0.3", "wheel", "build"]
build-backend = "setuptools.build_meta"

[project]
name = "mypackage"
version = "0.0.1.dev0"
dynamic = [
    "dependencies",
]
[tool.setuptools.dynamic]
dependencies = { file = ["requirements.txt"] }

3. place a package, not available on pypi.org, within the configured folder holding .whl files

packagenotonpypiorg.whl

4. python -m build a project with requirement.txt containing a dependency only hosted locally.

build asks setuptools to search for packages. First http://localhost:8080/simple/ then pypi.org

setuptools.dist.Distribution._normalize_requires does not filter out warehouse location lines.

packaging.requirements.Requirement does not recognize those lines as packages.

In this example, a local pypiserver is configured. When python -m pip install runs, the --extra-index-url option is necessary to find locally hosted .whl (packages). Which is great for pip, not so great for python -m build.

Output

Traceback (most recent call last):
  File "/tmp/build-env-ve1j18i8/lib/python3.9/site-packages/packaging/requirements.py", line 36, in __init__
    parsed = _parse_requirement(requirement_string)
  File "/tmp/build-env-ve1j18i8/lib/python3.9/site-packages/packaging/_parser.py", line 62, in parse_requirement
    return _parse_requirement(Tokenizer(source, rules=DEFAULT_RULES))
  File "/tmp/build-env-ve1j18i8/lib/python3.9/site-packages/packaging/_parser.py", line 71, in _parse_requirement
    name_token = tokenizer.expect(
  File "/tmp/build-env-ve1j18i8/lib/python3.9/site-packages/packaging/_tokenizer.py", line 142, in expect
    raise self.raise_syntax_error(f"Expected {expected}")
  File "/tmp/build-env-ve1j18i8/lib/python3.9/site-packages/packaging/_tokenizer.py", line 167, in raise_syntax_error
    raise ParserSyntaxError(
packaging._tokenizer.ParserSyntaxError: Expected package name at the start of dependency specifier
    --extra-index-url http://localhost:8080/simple/
    ^
The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/mnt/sda1/dev_parent/sqa_cf/.venv/lib/python3.9/site-packages/pyproject_hooks/_in_process/_in_process.py", line 389, in <module>
    main()
  File "/mnt/sda1/dev_parent/sqa_cf/.venv/lib/python3.9/site-packages/pyproject_hooks/_in_process/_in_process.py", line 373, in main
    json_out["return_val"] = hook(**hook_input["kwargs"])
  File "/mnt/sda1/dev_parent/sqa_cf/.venv/lib/python3.9/site-packages/pyproject_hooks/_in_process/_in_process.py", line 317, in get_requires_for_build_sdist
    return hook(config_settings)
  File "/tmp/build-env-ve1j18i8/lib/python3.9/site-packages/setuptools/build_meta.py", line 337, in get_requires_for_build_sdist
    return self._get_build_requires(config_settings, requirements=[])
  File "/tmp/build-env-ve1j18i8/lib/python3.9/site-packages/setuptools/build_meta.py", line 304, in _get_build_requires
    self.run_setup()
  File "/tmp/build-env-ve1j18i8/lib/python3.9/site-packages/setuptools/build_meta.py", line 320, in run_setup
    exec(code, locals())
  File "<string>", line 1, in <module>
  File "/tmp/build-env-ve1j18i8/lib/python3.9/site-packages/setuptools/__init__.py", line 117, in setup
    return distutils.core.setup(**attrs)
  File "/tmp/build-env-ve1j18i8/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 160, in setup
    dist.parse_config_files()
  File "/tmp/build-env-ve1j18i8/lib/python3.9/site-packages/setuptools/dist.py", line 757, in parse_config_files
    pyprojecttoml.apply_configuration(self, filename, ignore_option_errors)
  File "/tmp/build-env-ve1j18i8/lib/python3.9/site-packages/setuptools/config/pyprojecttoml.py", line 73, in apply_configuration
    return _apply(dist, config, filepath)
  File "/tmp/build-env-ve1j18i8/lib/python3.9/site-packages/setuptools/config/_apply_pyprojecttoml.py", line 60, in apply
    dist._finalize_requires()
  File "/tmp/build-env-ve1j18i8/lib/python3.9/site-packages/setuptools/dist.py", line 383, in _finalize_requires
    self._normalize_requires()
  File "/tmp/build-env-ve1j18i8/lib/python3.9/site-packages/setuptools/dist.py", line 404, in _normalize_requires
    self.extras_require = dict_(
  File "/tmp/build-env-ve1j18i8/lib/python3.9/site-packages/setuptools/dist.py", line 405, in <genexpr>
    (k, list(map(str, _reqs.parse(v or [])))) for k, v in extras_require.items()
  File "/tmp/build-env-ve1j18i8/lib/python3.9/site-packages/packaging/requirements.py", line 38, in __init__
    raise InvalidRequirement(str(e)) from e
packaging.requirements.InvalidRequirement: Expected package name at the start of dependency specifier
    --extra-index-url http://localhost:8080/simple/

Start in setuptools/dist.py at Distribution._normalize_requires and reproduce the failure with the pyproject.toml and requirements.txt setup described in the issue. Verify that pip option lines such as --extra-index-url no longer reach packaging.requirements.Requirement and that python -m build succeeds with locally hosted dependencies.