microsoft / microsoft/vscode-remote-release

Investigate using local server on remoteListenOnSocket

Open
#6,086 44 comments 15 reactions 1 assignee View on GitHub

Nobody has claimed this yet.

debt ssh
Dominant language
Dockerfile
Stars
4.2k
Forks
470
Avg merge
1d 1h
Merged PRs (30d)
1

Description

I was trying to get onto the second layer of an HPC after SSH-ing with some Slurm command (srun), as was generally described in #1722. I tried the newest RemoteCommand from the Pre-release version of Remote SSH (following instructions in this issue), but it seems not passing the RemoteCommand when connecting.

  • VSCode Version: 1.63
  • Local OS Version: Windows 10 21H2 (Release Preview)
  • Remote OS Version: CentOS Linux release 7.9.2009
  • Remote Extension/Connection Type: SSH
Logs 
[11:47:37.895] Log Level: 2
[11:47:37.897] remote-ssh@0.71.2021121615
[11:47:37.897] win32 x64
[11:47:37.915] SSH Resolver called for "ssh-remote+some.host.name", attempt 1
[11:47:37.915] "remote.SSH.useLocalServer": false
[11:47:37.916] "remote.SSH.showLoginTerminal": false
[11:47:37.916] "remote.SSH.remotePlatform": {"instance-1":"linux","some.host.name":"linux"}
[11:47:37.916] "remote.SSH.path": undefined
[11:47:37.916] "remote.SSH.configFile": undefined
[11:47:37.916] "remote.SSH.useFlock": true
[11:47:37.916] "remote.SSH.lockfilesInTmp": false
[11:47:37.916] "remote.SSH.localServerDownload": auto
[11:47:37.916] "remote.SSH.remoteServerListenOnSocket": true
[11:47:37.917] "remote.SSH.showLoginTerminal": false
[11:47:37.917] "remote.SSH.defaultExtensions": []
[11:47:37.917] "remote.SSH.loglevel": 2
[11:47:37.917] "remote.SSH.enableDynamicForwarding": true
[11:47:37.917] "remote.SSH.enableRemoteCommand": true
[11:47:37.917] "remote.SSH.serverPickPortsFromRange": {}
[11:47:37.917] "remote.SSH.serverInstallPath": {}
[11:47:37.918] SSH Resolver called for host: some.host.name
[11:47:37.918] Setting up SSH remote "some.host.name"
[11:47:37.922] Using commit id "899d46d82c4c95423fb7e10e68eba52050e30ba3" and quality "stable" for server
[11:47:37.925] Install and start server if needed
[11:47:37.927] Checking ssh with "ssh -V"
[11:47:37.961] > OpenSSH_for_Windows_8.1p1, LibreSSL 3.0.2
(HERE) [11:47:37.964] Running script with connection command: ssh -T -D 8501 -o RemoteCommand=none "some.host.name" bash
[11:47:37.966] Terminal shell path: C:\Windows\System32\cmd.exe
[11:47:38.158] > ]0;C:\Windows\System32\cmd.exe 
[11:47:38.158] Got some output, clearing connection timeout
[11:47:38.409] > P
[11:47:38.446] > Password: 
[11:47:38.447] Showing password prompt
[11:47:43.428] Got password response
[11:47:43.429] "install" wrote data to terminal: "*******************"
[11:47:43.446] > 
[11:47:58.917] > 91a30fe8ff8e: running
[11:47:59.003] > Acquiring lock on /home/yeh803/.vscode-server/bin/899d46d82c4c95423fb7e10e68eba5
> 2050e30ba3/vscode-remote-lock.yeh803.899d46d82c4c95423fb7e10e68eba52050e30ba3   
[11:47:59.041] > Found existing installation at /home/yeh803/.vscode-server/bin/899d46d82c4c95423
> fb7e10e68eba52050e30ba3...
[11:47:59.075] > Checking /home/yeh803/.vscode-server/.899d46d82c4c95423fb7e10e68eba52050e30ba3.l
> og and /home/yeh803/.vscode-server/.899d46d82c4c95423fb7e10e68eba52050e30ba3.pid
>  for a running server
[11:47:59.084] > Looking for server with pid: 31076
[11:47:59.173] > Starting server with command... /home/yeh803/.vscode-server/bin/899d46d82c4c9542
> 3fb7e10e68eba52050e30ba3/server.sh --start-server --host=127.0.0.1 --enable-remo
> te-auto-shutdown   --socket-path=/run/user/184812/vscode-ssh-remote-server-sock-
> 1640018879  &> "/home/yeh803/.vscode-server/.899d46d82c4c95423fb7e10e68eba52050e
> 30ba3.log" < /dev/null
> printenv:
>     MANPATH=/n/app/lmod/lmod/share/man::
>     XDG_SESSION_ID=20830
>     rvm_bin_path=/usr/local/rvm/bin
>     GEM_HOME=/usr/local/rvm/gems/ruby-2.4.9      
>     SHELL=/bin/bash
>     IRBRC=/usr/local/rvm/rubies/ruby-2.4.9/.irbrc
>     MODULEPATH_ROOT=/n/app/lmod/lmod/modulefiles 
>     SSH_CLIENT=73.61.9.36 9459 22
>     CONDA_SHLVL=1
>     CONDA_PROMPT_MODIFIER=(base)
>     LMOD_PKG=/n/app/lmod/lmod
[11:47:59.181] > 
>     MY_RUBY_HOME=/usr/local/rvm/rubies/ruby-2.4.9
>     LMOD_VERSION=7.5.17
>     QT_GRAPHICSSYSTEM_CHECKED=1
>     USER=yeh803
>     LMOD_sys=Linux
>     CONDA_EXE=/n/app/conda2/bin/conda
>     rvm_path=/usr/local/rvm
>     VSCODE_AGENT_FOLDER=/home/yeh803/.vscode-server
>     _CE_CONDA=
>     rvm_prefix=/usr/local
>     MAIL=/var/mail/yeh803
>     PATH=/n/app/conda2/bin:/n/app/conda2/condabin:/usr/local/rvm/gems/ruby-2.4.9
> /bin:/usr/local/rvm/gems/ruby-2.4.9@global/bin:/usr/local/rvm/rubies/ruby-2.4.9/
> bin:/n/cluster/bin:/opt/singularity/bin:/usr/local/bin:/usr/bin:/opt/puppetlabs/
> bin:/usr/local/rvm/bin
>     SLURM_CONF=/etc/slurm/slurm.conf
>     CONDA_PREFIX=/n/app/conda2
>     LMOD_SETTARG_CMD=:
>     PWD=/home/yeh803
>     LANG=en_US.UTF-8
>     MODULEPATH=/n/app/lmod/lmod/modulefiles/Linux:/n/app/lmod/lmod/modulefiles/C
> ore
>     TZ=America/New_York
>     LMOD_CMD=/n/app/lmod/lmod/libexec/lmod
>     _CE_M=
>     HMS_CLUSTER=o2
>     KRB5CCNAME=FILE:/tmp/krb5cc_184812_zPUZlA
>     rvm_version=1.29.10 (latest)
>     SHLVL=3
>     HOME=/home/yeh803
>     BASH_ENV=/n/app/lmod/lmod/init/bash
>     SSH_ORIGINAL_COMMAND=bash
>     CONDA_PYTHON_EXE=/n/app/conda2/bin/python
>     LOGNAME=yeh803
>     GEM_PATH=/usr/local/rvm/gems/ruby-2.4.9:/usr/local/rvm/gems/ruby-2.4.9@globa
> l
>     SSH_CONNECTION=73.61.9.36 9459 134.174.159.26 22
>     MODULESHOME=/n/app/lmod/lmod
>     CONDA_DEFAULT_ENV=base
>     LESSOPEN=||/usr/bin/lesspipe.sh %s
>     SHARED_DATABASES=/n/groups/shared_databases
>     LMOD_FULL_SETTARG_SUPPORT=no
>     XDG_RUNTIME_DIR=/run/user/184812
>     LMOD_DIR=/n/app/lmod/lmod/libexec
>     RUBY_VERSION=ruby-2.4.9
>     BASH_FUNC_module()=() {  eval $($LMOD_CMD bash "$@") && eval $(${LMOD_SETTAR
> G_CMD:-:} -s sh)
>     }
>     BASH_FUNC_ml()=() {  eval $($LMOD_DIR/ml_cmd "$@")
>     }
>     _=/usr/bin/printenv
[11:47:59.188] > 
[11:47:59.207] > Spawned remote server: 104641
> Waiting for server log...
[11:47:59.223] > 
[11:47:59.274] > Waiting for server log...
[11:47:59.374] > Waiting for server log...
[11:47:59.471] > Waiting for server log...
[11:47:59.571] > Waiting for server log...
[11:47:59.673] > Waiting for server log...
[11:47:59.773] > Waiting for server log...
[11:47:59.874] > Waiting for server log...
[11:47:59.972] > Waiting for server log...
[11:48:00.070] > Waiting for server log...
[11:48:00.172] > Waiting for server log...
[11:48:00.273] > Waiting for server log...
[11:48:00.373] > Waiting for server log...
[11:48:00.474] > Waiting for server log...
[11:48:00.575] > Waiting for server log...
[11:48:00.673] > Waiting for server log...
[11:48:00.709] > Waiting for server log...
[11:48:00.771] > Waiting for server log...
[11:48:00.881] >  
> *
> * Reminder: You may only use this software with Visual Studio family products,  
> * as described in the license (https://go.microsoft.com/fwlink/?linkid=2077057)
> *
> 
[11:48:00.906] > 
[11:48:00.974] > 91a30fe8ff8e: start
> SSH_AUTH_SOCK====
> DISPLAY====
> webUiAccessToken====
> listeningOn==/run/user/184812/vscode-ssh-remote-server-sock-1640018879==        
> osReleaseId==centos==
> arch==x86_64==
> tmpDir==/run/user/184812==
> platform==linux==
> unpackResult====
> didLocalDownload==0==
> downloadTime====
> installTime====
> extInstallTime====
> serverStartTime==1695==
> connectionToken==aa11111a-1aa1-111a-a111-11111a111a1a==
> 91a30fe8ff8e: end
[11:48:00.976] Received install output: 
SSH_AUTH_SOCK====
DISPLAY====
webUiAccessToken====
listeningOn==/run/user/184812/vscode-ssh-remote-server-sock-1640018879==        
osReleaseId==centos==
arch==x86_64==
tmpDir==/run/user/184812==
platform==linux==
unpackResult====
didLocalDownload==0==
downloadTime====
installTime====
extInstallTime====
serverStartTime==1695==
connectionToken==aa11111a-1aa1-111a-a111-11111a111a1a==
[11:48:00.977] Remote server is listening on /run/user/184812/vscode-ssh-remote-server-sock-1640018879
[11:48:00.977] Parsed server configuration: {"serverConfiguration":{"remoteListeningOn":{"socketPath":"/run/user/184812/vscode-ssh-remote-server-sock-1640018879"},"osReleaseId":"centos","arch":"x86_64","webUiAccessToken":"","sshAuthSock":"","display":"","tmpDir":"/run/user/184812","platform":"linux","connectionToken":"aa11111a-1aa1-111a-a111-11111a111a1a"},"serverStartTime":1695,"installUnpackCode":""}
(HERE) [11:48:00.979] Running script with connection command: ssh -T -L 127.0.0.1:8507:/run/user/184812/vscode-ssh-remote-server-sock-1640018879 -o RemoteCommand=none "some.host.name" bash
[11:48:00.980] Spawning tunnel with: type "C:\Users\Yep\AppData\Local\Temp\vscode-linux-multi-line-command-some.host.name-223392234.sh" | ssh -T -L 127.0.0.1:8507:/run/user/184812/vscode-ssh-remote-server-sock-1640018879 -o RemoteCommand=none "some.host.name" bash
[11:48:00.981] Terminal shell path: C:\Windows\System32\cmd.exe
[11:48:00.995] > 
[11:48:01.171] > ]0;C:\Windows\System32\cmd.exe 
[11:48:01.538] > P
[11:48:01.822] > Password: 
[11:48:01.823] Showing password prompt
[11:48:06.095] Got password response
[11:48:06.095] "SSH Tunnel" wrote data to terminal: "*******************"
[11:48:06.127] > 
[11:48:14.043] > SSH_AUTH_SOCK=
> Connected to SSH Host - Please do not close this terminal
[11:48:14.044] Spawned SSH tunnel between local port 8507 and remote target /run/user/184812/vscode-ssh-remote-server-sock-1640018879
[11:48:14.044] Waiting for ssh tunnel to be ready
[11:48:14.045] Tunneled /run/user/184812/vscode-ssh-remote-server-sock-1640018879 to local port 8507
[11:48:14.045] Resolved "ssh-remote+some.host.name" to "127.0.0.1:8507"
[11:48:14.048] ------

Steps to Reproduce:

  1. Add "remote.SSH.enableRemoteCommand": true, "remote.SSH.useLocalServer": true, "remote.SSH.remoteServerListenOnSocket": true to settings.json. "remote.SSH.remoteServerListenOnSocket": true is a must because otherwise would result in Failed to set up socket for dynamic port forward to remote port 45824: Socket closed.

  2. SSH to some.host.name with the following ssh config:

Host some.host.name
    User some.user.name
    // ForwardAgent yes  // Seems this does not matter
    RemoteCommand srun -t 0-12:00 --pty -p gpu --gres=gpu:1 /bin/bash

Does this issue occur when you try this locally?: It's a connection issue.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.