SPython incompatible with windows. MLCube fails at import time
Open
@sergey-serebryakov is already working on this.
Since Feb 13, 2023.
- Dominant language
- Python
- Stars
- 160
- Forks
- 31
- PR merge metrics
- No merged PRs in 30d
Description
Trying to run a docker MLCube on windows fails because of unsupported imports. This happens because spython, the singularity python API, tries to import pwd, which doesn't exist on Windows. Here's the traceback:
$ mlcube run --mlcube=C:\Users\aleja\.medperf\localhost_8000\cubes\1\mlcube.yaml --task=prepare --platform=docker data_path="C:\Users\aleja\.medperf\localhost_8000\demo\137950e4e7b8de3baa4a9982c3ebea6a52bd33d3\mock_chexpert" labels_path="C:\Users\aleja\.medperf\localhost_8000\demo\137950e4e7b8de3baa4a9982c3ebea6a52bd33d3\mock_chexpert" output_path="C:\Users\aleja\.medperf\localhost_8000\data\tmp_26312981032654\data" Ptasks.prepare.parameters.input.data_path.opts="ro" Ptasks.prepare.parameters.input.labels_path.opts="ro"
Traceback (most recent call last):
File "d:\users\aleja\anaconda3\envs\factored\lib\runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "d:\users\aleja\anaconda3\envs\factored\lib\runpy.py", line 87, in _run_code
exec(code, run_globals)
File "D:\Users\aleja\anaconda3\envs\factored\Scripts\mlcube.exe\__main__.py", line 7, in <module>
File "d:\users\aleja\anaconda3\envs\factored\lib\site-packages\click\core.py", line 829, in __call__
return self.main(*args, **kwargs)
File "d:\users\aleja\anaconda3\envs\factored\lib\site-packages\click\core.py", line 782, in main
rv = self.invoke(ctx)
File "d:\users\aleja\anaconda3\envs\factored\lib\site-packages\click\core.py", line 1256, in invoke
Command.invoke(self, ctx)
File "d:\users\aleja\anaconda3\envs\factored\lib\site-packages\click\core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "d:\users\aleja\anaconda3\envs\factored\lib\site-packages\click\core.py", line 610, in invoke
return callback(*args, **kwargs)
File "d:\users\aleja\anaconda3\envs\factored\lib\site-packages\mlcube\__main__.py", line 119, in cli
_ = SystemSettings().update_installed_runners()
File "d:\users\aleja\anaconda3\envs\factored\lib\site-packages\mlcube\system_settings.py", line 111, in update_installed_runners
installed_runners: t.Dict = Platform.get_installed_runners()
File "d:\users\aleja\anaconda3\envs\factored\lib\site-packages\mlcube\platform.py", line 60, in get_installed_runners
runner_cls: t.Type[Runner] = get_runner_class()
File "d:\users\aleja\anaconda3\envs\factored\lib\site-packages\mlcube_singularity\__init__.py", line 40, in get_runner_class
from mlcube_singularity.singularity_run import SingularityRun
File "d:\users\aleja\anaconda3\envs\factored\lib\site-packages\mlcube_singularity\singularity_run.py", line 6, in <module>
from spython.utils.terminal import (
File "d:\users\aleja\anaconda3\envs\factored\lib\site-packages\spython\utils\__init__.py", line 5, in <module>
from .terminal import (
File "d:\users\aleja\anaconda3\envs\factored\lib\site-packages\spython\utils\terminal.py", line 12, in <module>
import pwd
ModuleNotFoundError: No module named 'pwd'
This issue disappears when uninstall mlcube-singularity. It would be useful to provide some hints to the user if this error happens. Is singularity even supported on windows without WSL? If not, raising an error when installing mlcube-singularity could be another approach
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.