pypa / pypa/setuptools

install doesn't handle extra marker

Open
#2,204 1 comment 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
2.9k
Forks
1.4k
Avg merge
1d 1h
Merged PRs (30d)
1

Description

PEP 508 grammar allows you to specify extra as a marker

package-name>=1.3; extra = "foo"

Which is expected to be installed if specified with [foo] extras. However this syntax is not supported on legacy (setuptools & egg based) installation. Minimal example:

from setuptools import setup
setup(
    name="pip-extra-test",
    version="0.1.0",
    install_requires=[
        'requests>=2.0,<2.10; extra == "foo"',
        'requests>=2.10,<2.20; extra != "foo"',
    ],
    extras_require={
        'foo': ['fire'],
        'bar': ['arrow'],
    }
)

then run pip install ".[foo]" or pip install ".[bar]".

Error message shows:

Traceback (most recent call last):
  File "/Users/jjong/.pyenv/versions/3.7.6/envs/pipResolverTest/lib/python3.7/site-packages/pip/_internal/cli/base_command.py", line 188, in _main
    status = self.run(options, args)
  File "/Users/jjong/.pyenv/versions/3.7.6/envs/pipResolverTest/lib/python3.7/site-packages/pip/_internal/cli/req_command.py", line 185, in wrapper
    return func(self, options, args)
  File "/Users/jjong/.pyenv/versions/3.7.6/envs/pipResolverTest/lib/python3.7/site-packages/pip/_internal/commands/install.py", line 333, in run
    reqs, check_supported_wheels=not options.target_dir
  File "/Users/jjong/.pyenv/versions/3.7.6/envs/pipResolverTest/lib/python3.7/site-packages/pip/_internal/resolution/legacy/resolver.py", line 179, in resolve
    discovered_reqs.extend(self._resolve_one(requirement_set, req))
  File "/Users/jjong/.pyenv/versions/3.7.6/envs/pipResolverTest/lib/python3.7/site-packages/pip/_internal/resolution/legacy/resolver.py", line 411, in _resolve_one
    set(req_to_install.extras) - set(dist.extras)
  File "/Users/jjong/.pyenv/versions/3.7.6/envs/pipResolverTest/lib/python3.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2978, in extras
    return [dep for dep in self._dep_map if dep]
  File "/Users/jjong/.pyenv/versions/3.7.6/envs/pipResolverTest/lib/python3.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2702, in _dep_map
    self.__dep_map = self._filter_extras(self._build_dep_map())
  File "/Users/jjong/.pyenv/versions/3.7.6/envs/pipResolverTest/lib/python3.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2717, in _filter_extras
    invalid_marker(marker)
  File "/Users/jjong/.pyenv/versions/3.7.6/envs/pipResolverTest/lib/python3.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 1360, in invalid_marker
    evaluate_marker(text)
  File "/Users/jjong/.pyenv/versions/3.7.6/envs/pipResolverTest/lib/python3.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 1378, in evaluate_marker
    return marker.evaluate()
  File "/Users/jjong/.pyenv/versions/3.7.6/envs/pipResolverTest/lib/python3.7/site-packages/pip/_vendor/packaging/markers.py", line 329, in evaluate
    return _evaluate_markers(self._markers, current_environment)
  File "/Users/jjong/.pyenv/versions/3.7.6/envs/pipResolverTest/lib/python3.7/site-packages/pip/_vendor/packaging/markers.py", line 245, in _evaluate_markers
    lhs_value = _get_env(environment, lhs.value)
  File "/Users/jjong/.pyenv/versions/3.7.6/envs/pipResolverTest/lib/python3.7/site-packages/pip/_vendor/packaging/markers.py", line 225, in _get_env
    "{0!r} does not exist in evaluation environment.".format(name)
pip._vendor.packaging.markers.UndefinedEnvironmentName: 'extra' does not exist in evaluation environment.

More information

On further debugging, I found my package generates egg-info/requires.txt of the following:

[:extra == "foo"]
requests>=2.0,<2.10

[:extra != "foo"]
requests>=2.10,<2.20

[bar]
arrow

[foo]
fire

Internally during building the initial dependency map, it tries to strip out markers if it is evaluated as true, but since default environment does not contain extra field (as it should be injected externally), evaluation of the marker fails.

Related to #2007 #1139 #887

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the generated egg-info/requires.txt example and the dependency-map and marker-evaluation paths linked in the report. Reproduce the legacy installation with the setuptools example and trace how the extra marker is evaluated. Done means the [foo] and [bar] installations handle the marker requirements without the UndefinedEnvironmentName error.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
build-system
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.