Given Channel's flow control and lack of explicit thread safety, how do I read a command's stderr and stdout without deadlocking?
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 9.9k
- Forks
- 2.1k
- PR merge metrics
- No merged PRs in 30d
Description
Suppose I want to:
- Execute some arbitrary command, where I don't necessarily know anything about how it will use stdout and stderr.
- Block until the command completes.
- Return its status code as an
int, stdout output as abytes, and stderr output as a separatebytes.
client.exec_command(command) gives us separate stdout and stderr file-like objects. But, how do I safely read from them in a way that won't risk deadlocking?
Specifically, since a Channel seems to encapsulate both a stderr and stdout stream, I assume that both streams share flow control. [Edit: This assumption appears true at the protocol level, at least.] So, I'm concerned about:
stdout.read()hanging if there's too much data on stderr waiting to be read.stderr.read()hanging if there's too much data on stdout waiting to be read.
The unrelated recv_exit_status() docs suggest running Channel.recv() in the background to work around a separate but similar deadlock hazard:
In some situations, receiving remote output larger than the current
Transportor session’swindow_size[...] will causerecv_exit_statusto hang indefinitely if it is called prior to a sufficiently largeChannel.recv(or if there are no threads callingChannel.recvin the background).In these cases, ensuring that
recv_exit_statusis called afterChannel.recv(or, again, using threads) can avoid the hang.
Which maybe hints, indirectly, that the expected way to accomplish this is something like this:
stdout = None
stderr = None
def read_stdout():
nonlocal stdout
stdout = stdout_stream.read()
stdout_thread = threading.Thread(target=read_stdout)
def read_stderr():
nonlocal stderr
stderr = stderr_stream.read()
stderr_thread = threading.Thread(target=read_stderr)
stdout_thread.start()
stderr_thread.start()
stdout_thread.join()
stderr_thread.join()
status = stdout.channel.recv_exit_status()
Edit: Fixed some variable names.
But, other than that recv_exit_status() warning, it doesn't seem like the docs mention threading at all. There's no explicit guarantee that it's safe to read() from complementary stdout/stderr streams from concurrent threads. And it seems like a bad idea to assume thread safety, so I'm at an impasse.
What am I missing?
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with client.exec_command, the stdout and stderr file-like objects, and the Channel.recv_exit_status documentation. Reproduce commands that produce substantial output on both streams, then determine what concurrency and thread-safety guarantees the documentation should state; done means a decided, documented safe usage pattern.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- networking
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100