python-poetry / python-poetry/poetry
Virtual environment name mapping - make more portable [with PR for solution]
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 34.3k
- Forks
- 2.5k
- Avg merge
- 2d 19h
- Merged PRs (30d)
- 30
Description
- I have searched the issues of this repo and believe that this is not a duplicate.
- I have searched the documentation and believe that my question is not covered.
Feature Request
Poetry, if allowed to automatically manage them, creates virtual environments with names roughly like project name-hash-python version. This design choice makes life simple in some cases, but adds a hidden state to the project that complicates things in some situations. Mainly, the hash depends on information outside the scope of the project (e.g. project folder path), which is a questionable design choice in my opinion. I think that adding a little flexibility to override this behaviour, or even change how to hash is generated / stored would make Poetry a better tool overall.
Just to mention a few cases where the current name mapping causes problems: #3459 and #4654. Briefly, the hash in the virtual env name seems to depend on the path where the project is located and perhaps other things. Rename the project folder - poetry will create a new virtual env and you'll need to reinstall all the packages from zero; plus the old env will linger forgotten. If you share workspace settings on multiple computers (stored in project folders, e.g. for VSCode), the env path will need to be updated each time you switch to another computer; similar in multi-user situations.
@sinoroc asks in #3459, how a design change would help Poetry? (S)He also mentions that the current design makes things simpler by not having to remember the virtual env's name, like you need to if you use venv or Conda. I think Poetry only makes things simpler if you use it the way it's supposed to be used, but doesn't have the flexibility to move beyond that. New users, who are not familiar with virtual environment management, might find the automated environment creation useful ... until they rename the project folder ... and then get a big surprise when their environment is gone without warning. Moreover, while managing workspace settings for VSCode (example above) has nothing to do with Poetry, easier integration with other tools is better for Poetry, isn't it?
In my case, I use pyenv (pyenv-win on Windows) to manage Python versions; manage virtual environments with venv, and let poetry manage packages inside virtual envs. However, the only thing I need venv for, is to get the virtual environment name I want - if only poetry would let me change the name mapping, I could drop venv from the pipeline.
OK, OK, so why not store the virtual environment in the project folder? Poetry can already do that, so problem solved, right? Well, not really. What if I'm moving between Linux and PC? Surely the packages cannot be shared. And what if I keep code on Google/One/other cloud drive and don't have gigabytes to store all the packages for the dependencies? Again, Poetry covers some common cases but it could do better!
I believe that the problem here is how the hash is generated. It depends on things that are outside the scope of the project, thus reducing portability / consistency even between different folders on the same PC. Besides being a design choice, I don't see why it cannot be done differently. While I'm not sure what the best solution is, here are some ideas to start with:
- More manual control over the virtual env names (as suggested in #3459). Adds some complexity, but makes advanced users happy by allowing them to do things their own way. Don't be Apple.
- Store hash in poetry.toml with other local settings. First time the enviroment is created, the hash can be generated randomly, and subsequent virtual environment accesses (including creation) would reuse it and combine it with python version etc. Basically, assign the project some kind of ID that can be shared between computers / users as they want. Instead of just the hash, virtual env names could be stored. No hidden state; no dependency on location; better portability; and this could be done completely automatically without user interaction maintaining current level of simplicity.
1 & 2 aren't mutually exclusive, in fact, I think the best outcome would be achieved by implementing both. Overall, I'm sure that there are several points to iron out, such as how to enable this behaviour (additional command line switches / configuration settings?), but, overall, I don't think these changes would have negative side effects.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the virtual-environment name mapping described in the issue and review poetry.toml, which is proposed for storing local settings. Compare the manual naming and stored-project-ID approaches; done would require an agreed portable design that addresses renamed projects and moving between platforms.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- cli, devtools
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100