AcademySoftwareFoundation / AcademySoftwareFoundation/rez
rez-bind python erroneously set some PYTHONPATH for python/lib and python/extra
- Dominant language
- Python
- Stars
- 1.1k
- Forks
- 369
- Avg merge
- 12d 3h
- Merged PRs (30d)
- 5
Description
Hi,
I discovered, already some days/week ago, that `rez-bind python` actually creates a package.py which contains instructions to add to PYTHONPATH the paths to 2 of the "main" libs directories of the python which was bound to. (it's the directories where the os and setuptools modules are available under the python which is bound to).
It (rez-bind) also has copied everything under these 2 paths into the package installation directory of that python.
We so end with :
`../packages/python/x.y.z/platform-linux/arch-x86_64/bin/python` -> link to the real python binary which was bound to.
But:
```
../packages/python/x.y.z/platform-linux/arch-x86_64/python/lib
../packages/python/x.y.z/platform-linux/arch-x86_64/python/extra
```
are now also in PYTHONPATH.
BUT, now anything which use a python under the hood/internally will have to use that PYTHONPATH containing directories to the selected (by rez-env) python own system libs/modules.
We can see that effect like this :
```
$ python -m site
sys.path = [
'/home/gregory',
'/home/gregory/rez/master-2.9.1/packages/python/2.6.6/platform-linux/arch-x86_64/os-CentOS-6.7/python/lib',
'/home/gregory/rez/master-2.9.1/packages/python/2.6.6/platform-linux/arch-x86_64/os-CentOS-6.7/python/extra',
'/usr/lib64/python26.zip',
'/usr/lib64/python2.6',
'/usr/lib64/python2.6/plat-linux2',
'/usr/lib64/python2.6/lib-tk',
'/usr/lib64/python2.6/lib-old',
'/usr/lib64/python2.6/lib-dynload',
'/home/gregory/.local/lib/python2.6/site-packages',
'/usr/lib64/python2.6/site-packages',
'/usr/lib64/python2.6/site-packages/PIL',
'/usr/lib64/python2.6/site-packages/gst-0.10',
'/usr/lib64/python2.6/site-packages/gtk-2.0',
'/usr/lib64/python2.6/site-packages/webkit-1.0',
'/usr/lib64/python2.6/site-packages/wx-2.8-gtk2-unicode',
'/usr/lib/python2.6/site-packages',
]
USER_BASE: '/home/gregory/.local' (exists)
USER_SITE: '/home/gregory/.local/lib/python2.6/site-packages' (exists)
ENABLE_USER_SITE: True
```
As you can see, given they are in PYTHONPATH, they take precedence over my .local (USER_SITE) directory.. which is a first not good.
But, if I would have rez-env with something using a python under hood, like nuke for instance, then Houston we have a (big) problem :
because nuke uses its own python, not necessarily compatible with the one/thoses I've bound to.
That is : nuke starts, it starts its own embedded python, which early import many important modules, like openssl, but then boum because the openssl for one python version isn't compatible with another. and given PYTHONPATH the python under nuke happens so to import the important modules from the python that was resolved in my rez-env call, not from his own python.
It seems to me you're trying, somehow, to do a _kind of_ virtualenv manually (but only a kind of) but the way it is actually triggers the problem(s) I mention / try to explain here. But PYTHONPATH is not done for that. well I mean putting a python system library/directory in PYTHONPATH is calling for problems.. as soon as you have to play with different python versions.
Actually, only doing the link to the python which is bound-to is enough to make it all available.
waiting answer / feedback or question ..
I can propose the PR (which is simply to not do the copy of these dirs and not add them to PYTHONPATH in the package.py which is generated) if you agree with this ..
Contributor guide
Research direction
Start with the `rez-bind python` entry point and inspect how its generated `package.py` copies the Python `lib` and `extra` directories and adds them to `PYTHONPATH`. Reproduce the issue with `python -m site`; done means the bound executable remains available without injecting those system-library paths or copying them into the package installation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- tooling
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100