dask / dask/distributed

self.connection.run("cmd /c ver") causing problems

Open
#5,411 6 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.7k
Forks
778
Avg merge
2h 50m
Merged PRs (30d)
3

Description

**What happened**:
SSHCluster() launch fails on Windows Server 2019 system. Turning on debug logs shows it fails at ln#179 (for scheduler), and when that gets 'fixed' (by me), it fails at ln#98 (for worker)

```
<< providing asyncssh.set_debug_level(2) detail log here>>
[INFO] 2021-10-12 18:20:35,479 logging.py:82 [conn=2, chan=1] Requesting new SSH session
[INFO] 2021-10-12 18:20:35,481 logging.py:82 [conn=2, chan=1] Command: cmd /c ver
...
<< providing asyncssh.set_debug_level(3) detail log here>>
distributed.deploy.ssh - INFO - File "..\lib\runpy.py", line 188, in _run_module_as_main
distributed.deploy.ssh - INFO - mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
distributed.deploy.ssh - INFO - File "..\lib\runpy.py", line 111, in _get_module_details
distributed.deploy.ssh - INFO - __import__(pkg_name)
distributed.deploy.ssh - INFO - File "..\lib\site-packages\distributed\__init__.py", line 24, in
distributed.deploy.ssh - INFO - from .deploy import Adaptive, LocalCluster, SpecCluster, SSHCluster
distributed.deploy.ssh - INFO - File "..\lib\site-packages\distributed\deploy\__init__.py", line 7, in
distributed.deploy.ssh - INFO - from .ssh import SSHCluster
distributed.deploy.ssh - INFO - File "..\lib\site-packages\distributed\deploy\ssh.py", line 98
distributed.deploy.ssh - INFO - result = await self.connection.run(cmd /c "ver")
distributed.deploy.ssh - INFO - ^
distributed.deploy.ssh - INFO - SyntaxError: invalid syntax
...
[INFO] 2021-10-12 18:20:35,502 logging.py:82 [conn=2, chan=1] Received exit status 1
[INFO] 2021-10-12 18:20:35,503 logging.py:82 [conn=2, chan=1] Received channel close
[INFO] 2021-10-12 18:20:35,505 logging.py:82 [conn=2, chan=1] Channel closed
...
...
Exception: Scheduler failed to set DASK_INTERNAL_INHERIT_CONFIG variable
```

`result = await self.connection.run("cmd /c ver")`

**What you expected to happen**:
indicated scheduler+workers ought to launch via SSHCluster().

**Minimal Complete Verifiable Example**:
Reproduced this error on multiple Windows 2019 server systems.

```python
Changing ln#98 and ln#179 results in successful establishment of desired dask scheduler and workers.
`result = await self.connection.run("ver")`
```

**Anything else we need to know?**:
However, it introduces additional issue in that subsequent conda env changes fail (due to size mismatch), and also versionmismatch warning/errors start popping because the appropriate env doesn't load right. Consequently, can't view dashboard via bokeh and such.

**Environment**:
OS: Windows Server 2019 v1809
Python: 3.9.7
Dask Distributed: 2021.9.1
Asyncssh: 2.7.1
Python version: 3.9.7
Install method: conda

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.