PowerShell / PowerShell/Win32-OpenSSH

OpenSSH for Windows hangs while writing to STDERR asynchronously

Open
#1,427 2 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
No language data
Stars
8.3k
Forks
819
PR merge metrics
No merged PRs in 30d

Description

Troubleshooting steps
https://github.com/PowerShell/Win32-OpenSSH/wiki/Troubleshooting-Steps

Terminal issue? please go through wiki
https://github.com/PowerShell/Win32-OpenSSH/wiki/TTY-PTY-support-in-Windows-OpenSSH

Please answer the following

"OpenSSH for Windows" version
7.9p1, 8.0p1.

Server OperatingSystem
Debian rodete rodete (x86-64).

Client OperatingSystem
Windows 10, Windows Server 2016.

What is failing
Reproduced reliably with OpenSSH 7.9p1, 8.0p1:

  1. We have an SSH client (named "test runner") written in Go running on Linux.
  2. Test runner SSH'es into Windows 10 and Server 2016 machines (OpenSSH server) to run commands which will create another SSH session connecting to other Linux machines. In short, the SSH sessions propagate like this: test runner (Linux) => Win (OpenSSH) => device (Linux).
  3. Test runner gets indefinitely blocked while trying to read from STDERR in the Windows SSH session since there is no output to STDERR.

I collected the call tack of ssh.exe on Windows:

        [External Code]
        ssh.exe!wait_for_multiple_objects_enhanced(unsigned long nCount, void * const * lpHandles, unsigned long dwMilliseconds, int bAlertable) Line 96        C
        ssh.exe!wait_for_any_event(void * * events, int num_events, unsigned long milli_seconds) Line 289       C
        ssh.exe!fileio_write(w32_io * pio, const void * buf, unsigned __int64 max_bytes) Line 749       C
        ssh.exe!w32_write(int fd, const void * buf, unsigned __int64 max) Line 562      C
        ssh.exe!do_log(LogLevel level, const char * fmt, char * args) Line 468  C
        ssh.exe!error(const char * fmt, ...) Line 170   C
        ssh.exe!check_host_key(char * hostname, sockaddr * hostaddr, unsigned short port, sshkey * host_key, int readonly, char * * user_hostfiles, unsigned int num_user_hostfiles, char * * system_hostfiles, unsigned int num_system_hostfiles) Line 1165    C
        ssh.exe!verify_host_key(char * host, sockaddr * hostaddr, sshkey * host_key) Line 1272  C
        ssh.exe!verify_host_key_callback(sshkey * hostkey, ssh * ssh) Line 98   C
        ssh.exe!kex_verify_host_key(ssh * ssh, sshkey * server_host_key) Line 1071      C
        ssh.exe!input_kex_gen_reply(int type, unsigned int seq, ssh * ssh) Line 160     C
        ssh.exe!ssh_dispatch_run(ssh * ssh, int mode, volatile int * done) Line 114     C
        ssh.exe!ssh_kex2(ssh * ssh, char * host, sockaddr * port, unsigned short) Line 224      C
        ssh.exe!ssh_login(ssh * ssh, Sensitive * sensitive, const char *) Line 1320     C
        ssh.exe!main(int ac, char * * av) Line 1499     C
        ssh.exe!wmain(int argc, wchar_t * * wargv) Line 61      C
        [External Code] 

For some reason the asyn write WriteFileEx gets stuck writing to STDERR and the main thread hangs waiting for the write to be completed.

I tried forcing the STDERR write to be sync and the problem goes away, i.e. in w32_write function, insert the following code:

if (fd == STDERR_FILENO) {
	fd_table.w32_ios[fd]->type = NONSOCK_SYNC_FD;
}

Expected output
The SSH sessions complete without hanging.

Actual output
The Windows SSH session hangs while writing to STDERR asynchronously.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with contrib/win32/win32compat/fileio.c around fileio_write and contrib/win32/win32compat/w32fd.c around w32_write, then reproduce the nested SSH-session case described in the issue. Compare asynchronous STDERR handling with the reported synchronous workaround. Done means Windows SSH sessions complete without hanging while writing to STDERR.

Written by the indexing model from the issue text.

Assessment

Tech stack
c
Domain
cli, operating-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.