[BUG] easy_install doesn't handle sysconfig vars containing multiple space separated paths
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 2.9k
- Forks
- 1.4k
- Avg merge
- 1d 1h
- Merged PRs (30d)
- 1
Description
setuptools version
setuptools==68.2.2
Python version
Python3.11 (via pyenv)
OS
macos
Additional environment information
$ python3 --version
Python 3.11.2
$ python3 -m pip --version
pip 23.2.1 from /Users/karlicos/.pyenv/versions/3.11.2/lib/python3.11/site-packages/pip (python 3.11)
Description
I'm using python3.11 via pyenv (this is the only version of python I have apart from system python3.9).
While I've never encountered this issue before, and it involves pyenv, I debugged a bit and I believe it's a setuptools rather than pyenv issue.
When trying to install any editable package, e.g. python3 -m pip install --user -e /path/to/pkg, somehow it creates an extra directory ending with a space in ~/.pyenv/versions:
$ ls -1 ~/.pyenv/versions/
3.11.2
3.11.2
(hard to see obviously because it ends with space 🙃 )
I think this causes some issues further down the line for pyenv, but it's kinda irrelevant for this bug.
I tried making ~/.pyenv/versions directory read only and running setuptools with debug information:
$ DISTUTILS_DEBUG=true python3 -m pip install --user -e /path/to/pkg
...
Distribution.get_command_obj(): creating 'egg_info' command object
os.makedirs('/Users/karlicos/.pyenv/versions/3.11.2 /Users/karlicos/.pyenv/versions/3.11.2/lib /Users/karlicos/.pyenv/versions/3.11.2/lib/python3.11 /Users/karlicos/.pyenv/versions/3.11.2/lib/python3.11/lib-dynload', 0o700)
error: [Errno 13] Permission denied: '/Users/karlicos/.pyenv/versions/3.11.2 '
...
It seems that this happens here https://github.com/pypa/setuptools/blob/82f5c60762eb16d9da9b19dbfc38b497a2939716/setuptools/command/easy_install.py#L1366-L1374
The offending variable turned out to be DESTDIRS, which contained a bunch of paths separated by a space. There is also INCLDIRSTROMAKE in my sysconfig that contains multiple paths
python3 -c 'import sysconfig; print({k: v for k, v in sysconfig.get_config_vars().items() if isinstance(v, str) and v.startswith("/Users/karlicos/") and " " in v})'
{'DESTDIRS': '/Users/karlicos/.pyenv/versions/3.11.2 /Users/karlicos/.pyenv/versions/3.11.2/lib /Users/karlicos/.pyenv/versions/3.11.2/lib/python3.11 /Users/karlicos/.pyenv/versions/3.11.2/lib/python3.11/lib-dynload', 'INCLDIRSTOMAKE': '/Users/karlicos/.pyenv/versions/3.11.2/include /Users/karlicos/.pyenv/versions/3.11.2/include /Users/karlicos/.pyenv/versions/3.11.2/include/python3.11 /Users/karlicos/.pyenv/versions/3.11.2/include/python3.11'}
I'm not sure what exactly the aim of this function, but perhaps it needs to do some extra checks before trying to create dirs. I suppose it's possible that it's a misconfiguration on my side somehow, but I think the variable names suggest they may contain multiple paths.
possibly relevant issue: https://github.com/pypa/setuptools/issues/3063
Expected behavior
Package is installed as editable, no extra dirs are created
How to Reproduce
Not sure how to provide consistent steps to reproduce since it requires pyenv set up in the first place, and there is no Docker on macos to try it
Output
provided above
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at setuptools/command/easy_install.py around lines 1366-1374 and inspect how sysconfig variables such as DESTDIRS and INCLDIRSTOMAKE are handled. Reproduce with the reported editable-install command and DISTUTILS_DEBUG=true where possible; done means an editable package install does not create directories from space-separated path values.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- build-system
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100