openwisp / openwisp/openwisp-controller
[bug] SSH exec_command throws away partial output and leaks channel on timeout
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 773
- Forks
- 315
- Avg merge
- 2d 16h
- Merged PRs (30d)
- 14
Description
Describe the bug
When exec_command hits its SSH command timeout, any output the remote process sent before the timeout is silently discarded. The channel also remains open on the server side.
Three separate problems at openwisp_controller/connection/connectors/ssh.py:
-
Partial output lost (lines 205-210). After
status_event.wait()returnsFalse, the method raisesCommandTimeoutExceptionwithout ever reading stdout or stderr:if not stdout.channel.status_event.wait(timeout=...): log_message = f"Command timed out after {timeout} seconds." logger.info(log_message) raise CommandTimeoutException(log_message) -
Channel leak (same block). No
channel.close()is called, so the server-side session for the timed-out command is never released until the SSH transport is torn down. -
Traceback lost (lines 196-197).
except socket.timeout: raise socket.timeout()creates a new exception instead of using bareraise, discarding the original traceback.
=> Not sure if this is intended or not ? if so I would add a comment next to it...
Steps To Reproduce
- Set up an SSH device connection with a remote host.
- Call
connector_instance.exec_command("echo partial_output && sleep 60", timeout=0.5). - Observe that
CommandTimeoutExceptionis raised with no.outputattribute —"partial_output"is lost. - Similar concerns for stderr...
Expected behavior
- If the remote process managed to produce any output before the timeout, that output should be available to the caller (e.g., as the message or an attribute of the exception).
- The SSH channel used for the timed-out command should be closed so the server can reclaim resources.
- The original traceback should likely be preserved when
socket.timeoutpropagates.
System Information:
- OS: any
- Python Version: any (3.x)
- OpenWISP Controller version: master (v1.2.x)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in openwisp_controller/connection/connectors/ssh.py at exec_command and inspect the timeout handling around the referenced lines. Reproduce the issue with echo partial_output && sleep 60 and a 0.5-second timeout, then verify that partial stdout/stderr remains available, the timed-out channel is closed, and the original socket.timeout traceback is preserved.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, networking
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100