Option of defining an environment script for starting engines (with SSHEngineSetLauncher)
- 主要語言
- Jupyter Notebook
- 星號
- 2.6k
- 分支
- 1k
- PR 合併指標
- 30 天內沒有已合併 PR
描述
We need to adapt the environment before starting the actual ipp _engine_ (through the `SSHEngineSetLauncher`). Adapting the environment via the `environment` parameter (in class `SSHLauncher`) is possible, but it's not very practical, since the launcher needs to know all the engine computer details in advance.
Calling a local environment script before starting the _engine_ would give much more flexibility and would easily allow starting the engine e.g. in a python virtual environment, conda environment, etc.
Although the general idea is quite simple the actual implement is not as easy. The problem: one cannot simply start a shell, next the environment script and then the actual command because the process handle (in python) would point to the shell and not the actual command. Hence, the retrieved process id would be incorrect. Using some system code it would be still possible to retrieve the true process id, but this is especially difficult in windows which is why I tested an alternative solution:
Starting a shell, outputting all environment variable, running the environment script and outputting all environment variable again. A bit of python code analyses the output and collects the environment variable changes (especially added and/or changed env.vars). I have implemented a small test script that works under linux and windows.
The collected changes can then be easily passed to the `env` parameter of `subprocess.Popen` function.
Does this sound like an appropriate idea and solution?
As before I'm happy to provide the implementation, tests, documentation, etc.
貢獻指南
研究方向
先閱讀 SSHEngineSetLauncher 和 SSHLauncher,接著檢查 issue 中描述的小型跨平台測試腳本,以及 subprocess.Popen 的環境處理。完成的標準是:環境腳本能夠在 Linux 和 Windows 上準備 engine 環境,同時不讓 Python 程序控制代碼指向 shell,並包含實作測試和文件。
由索引模型根據 Issue 內容生成。
評估
- 技術堆疊
- python, shell
- 領域
- distributed-systems
- Issue 類型
- 功能
- 難度
- 4/5
- 預估耗時
- 3-5 天
- 活躍度
- 冷清
- 描述清晰度
- 基本清楚
- 新手友好度
- 48/100