pip 21.3+ in-tree-builds with setuptools, do not work with read-only source trees
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 2.9k
- Forks
- 1.4k
- Avg merge
- 1d 1h
- Merged PRs (30d)
- 1
Description
Description
We are using pip as part of our build pipeline within a project containing a mix of languages. In order to keep everything clean and reproducible, we mount the source code read-only in a docker container and run the build steps there.
Even it's a mix of languages and tools, this approach used to work fine until now, when pip as the only of all the tools involved suddenly tried to touch the source tree during build.
I have read though the discussion in https://github.com/pypa/pip/issues/7555 that led to this change, but I feel like the point that there might be good reasons not to touch the source directory didn't get enough attention:
- You want to keep source code and build artifacts separate
- You don't want to clutter your scm
- Files created within the docker container are usually owned by root. So in our setup, our source directory would be cluttered with files owned by root after the container exits
- You want to make sure that no build script accidentally modifies the source code at build time. This is a very important point especially in large projects where it's hard to find side-effects in build scripts
Expected behavior
All other major packaging tools I know offer an option to chose a build directory that is separate from the source dir. I would expect that pip works in a similar way.
pip version
21.3
Python version
3.10
OS
ubuntu
How to Reproduce
Create a python package, mount read-only into a docker container and try to let pip install it, e.g:
mkdir readonly
echo "from setuptools import setup\nsetup(name='hello', version='1.0.0')" > readonly/setup.py
docker run -ti --rm -v `pwd`/readonly:/src:ro python:3.10 /bin/bash -c "pip install --upgrade pip; pip install /src"
rm -rf readonly
Output
Building wheels for collected packages: hello
Building wheel for hello (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: /usr/local/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/src/setup.py'"'"'; __file__='"'"'/src/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-gx0vxrjn
cwd: /src/
Complete output (8 lines):
running bdist_wheel
running build
installing to build/bdist.linux-x86_64/wheel
running install
running install_egg_info
running egg_info
creating hello.egg-info
error: could not create 'hello.egg-info': Read-only file system
----------------------------------------
ERROR: Failed building wheel for hello
Running setup.py clean for hello
Failed to build hello
Code of Conduct
- I agree to follow the PSF Code of Conduct.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the failure with the provided Docker command, using the read-only /src mount and setup.py example. Trace the setuptools egg_info step that attempts to create hello.egg-info; done means pip can install the package while keeping the source tree read-only and build artifacts separate.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, python
- Domain
- build-system
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100