containers / containers/gvisor-tap-vsock

win-sshproxy: named pipe connection not closed after SSH channel EOF causes non-Go API clients to hang

Open
#620 0 comments 1 reaction 0 assignees View on GitHub
Dominant language
Go
Stars
435
Forks
102
Avg merge
1d 15h
Merged PRs (30d)
16

Description

## Problem

When `win-sshproxy` forwards traffic between a Windows named pipe (`\.\pipe\docker_engine`) and an SSH tunnel to Podman's Unix socket, the named pipe connection is not properly closed after the remote SSH side sends EOF. This causes non-Go API clients (Node.js, .NET, etc.) to hang indefinitely when using Podman's exec or attach APIs on Windows.

## Root Cause Analysis

In `pkg/sshclient/ssh_forwarder.go`, the `forward()` function handles bidirectional data relay:

```go
func forward(src io.ReadCloser, dest CloseWriteStream, complete *sync.WaitGroup) {
defer complete.Done()
_, _ = io.Copy(dest, src)
_ = dest.CloseWrite() // ← Problem is here
}
```

When the SSH channel sends EOF (e.g., Podman closes a hijacked exec connection):
1. `io.Copy(pipe, ssh)` returns
2. `pipe.CloseWrite()` is called
3. go-winio's `CloseWrite()` implementation writes a **zero-byte message** to the named pipe to signal EOF
4. **Non-Go clients don't interpret zero-byte messages as EOF** — Node.js (libuv), .NET `NamedPipeClientStream`, and other clients simply ignore it
5. The pipe connection stays open, the client never receives an `end`/`close` event
6. Both `forward()` goroutines never complete (the pipe→ssh direction blocks on read), so `src.Close()` and `dest.Close()` in the cleanup goroutine are never called either

## Impact

This causes **connection leaks** that eventually exhaust all named pipe instances, making Podman completely unresponsive on Windows. This is the root cause of [containers/podman#24798](https://github.com/containers/podman/issues/24798).

Affected users: anyone using Podman on Windows with non-Go API clients, including:
- **testcontainers-node** (Node.js) — `container.exec()` hangs forever
- **testcontainers-dotnet** (.NET) — `NamedPipeClientStream` hangs
- **Docker SDK clients** — any client connecting via `\.\pipe\docker_engine`

## Reproduction

**Environment:** Podman 5.7.1, Windows 11, WSL2, win-sshproxy via `npipe:////./pipe/docker_engine`

```bash
# 1. Start a container
podman run -d --name exec-test alpine sleep 300

# 2. Create exec instance via named pipe (works fine)
node -e "
const http = require('http');
const body = JSON.stringify({Cmd:['/bin/sh','-c','echo hello'],AttachStdout:true,AttachStderr:true});
http.request({socketPath:'//./pipe/docker_engine',path:'/v1.41/containers/exec-test/exec',method:'POST',
headers:{'Content-Type':'application/json'}}, res => {
let d=''; res.on('data',c=>d+=c); res.on('end',()=>console.log(res.statusCode,d));
}).end(body);
"
# Output: 201 {"Id":""}

# 3. Start exec via named pipe — stream never closes
node -e "
const http = require('http');
const body = JSON.stringify({Detach:false,Tty:false});
http.request({socketPath:'//./pipe/docker_engine',path:'/v1.41/exec//start',method:'POST',
headers:{'Content-Type':'application/json'}}, res => {
console.log('status:', res.statusCode);
res.on('data', c => console.log('data:', JSON.stringify(c.toString())));
res.on('end', () => console.log('END — stream closed'));
}).setTimeout(10000, function() { console.error('TIMEOUT — stream never closed'); this.destroy(); })
.end(body);
"
# Output:
# status: 200
# data: "...hello\n"
# TIMEOUT — stream never closed ← BUG: 'end' event never fires

# 4. Same request via direct SSH tunnel — stream closes correctly
ssh -f -N -L 12345:/run/podman/podman.sock root@127.0.0.1 -p 60022 -i -o StrictHostKeyChecking=no
# (repeat exec create + start against http://127.0.0.1:12345 instead of named pipe)
# Output:
# status: 200
# data: "...hello\n"
# END — stream closed ← Works correctly, bypassing win-sshproxy
```

The comparison between step 3 (via win-sshproxy, hangs) and step 4 (via direct SSH tunnel, works) confirms the issue is in win-sshproxy's named pipe handling.

## Suggested Fix

When the SSH channel sends EOF on a hijacked/exec connection, `CloseWrite()` alone is insufficient for non-Go clients. The full `Close()` should be called on the pipe connection to properly signal EOF to all client implementations.

One approach: detect when `io.Copy` returns due to EOF on the SSH side and immediately close the pipe connection, rather than relying on `CloseWrite()` which only works with Go's `go-winio` reader.

## Related Issues

- [containers/podman#24798](https://github.com/containers/podman/issues/24798) — "Podman Becomes Unresponsive During Test Execution on Windows" (downstream symptom)
- [containers/podman#6853](https://github.com/containers/podman/issues/6853) — Original exec stream closure issue (fixed for direct API access, but not through win-sshproxy)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start in pkg/sshclient/ssh_forwarder.go at forward() and reproduce the named-pipe exec stream with the Node.js request described in the issue. Compare its behavior with the direct SSH tunnel; done means the non-Go client receives the stream end/close event and repeated connections do not remain open.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, node.js
Domain
networking
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.