indygreg / indygreg/PyOxidizer
No way to build packages with setup.py requirements
- Dominant language
- Rust
- Stars
- 6.2k
- Forks
- 256
- PR merge metrics
- No merged PRs in 30d
Description
This is admittedly a non-standard use case, but it breaks PyOxidizer on our internal libraries. For integration purposes we have something like the following at the top of our internal library `setup.py` files:
```
from distgradle import GradleDistribution
setup(
distclass=GradleDistribution,
...
)
```
This works for us because our build system guarantees that `distgradle` will be installed in the venv before this `setup.py` is executed. However, this breaks PyOxidizer because even though I can add a `PipInstallSimple` rule to install `distgradle` before the `PipRequirementsFile` rule that pulls in all of our internal dependencies, IIUC PyOxidizer first builds dependencies in a temp venv because it might have to manipulate the built packages for static linking and such.
Thus `PipInstallSimple(package='/path/to/distgradle.whl')` will put distgradle in the target venv but it won't be available in the temp venv for our internal library build. For the same reason, adding `distgradle` to my `requirements.txt` also doesn't work.
While more rare, this can be a problem with PyPI packages, if they also have these kinds of implicit pre-install dependencies. I remember seeing such packages, but don't have an example handy. Another possible problem would be packages which build differently depending on what other packages are already installed (e.g. conditional dependencies, etc).
I was thinking about maybe adding a feature for "pre-installs" which would land in the temp venv before the rule would run. While ugly, it probably wouldn't be too terrible.
Another option would be to clone the target venv, or merge the target venv with the temp venv before you try to build the individual dependency. Merging and cloning venvs though is fraught with peril.
I'm open to other suggestions, and I'm motivated to help implement a solution (if it's not to complex).
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by tracing the PipInstallSimple and PipRequirementsFile rules and how they create and populate the temporary build virtual environment. Compare the proposed pre-install, cloning, and merging approaches, then define a focused design and tests showing that a setup.py dependency such as distgradle is available while the internal package is built.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, rust
- Domain
- build-system
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 28/100