fsspec / fsspec/filesystem_spec
How to properly close (SSH) file systems?
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.4k
- Forks
- 490
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 38
Description
I did open an SSH file system and used it successfully:
o = fsspec.open("ssh://user@127.0.0.1")
o.fs.listdir("/")
Then, I lost the SSH connection because I lost WLAN connectivity because I entered suspend mode on my notebook. After that, I got this error on o.fs.listdir:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/user/.local/lib/python3.12/site-packages/fsspec/spec.py", line 1593, in listdir
return self.ls(path, detail=detail, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/.local/lib/python3.12/site-packages/fsspec/implementations/sftp.py", line 127, in ls
stats = [self._decode_stat(stat, path) for stat in self.ftp.listdir_iter(path)]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/paramiko/sftp_client.py", line 278, in listdir_iter
t, msg = self._request(CMD_OPENDIR, path)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/paramiko/sftp_client.py", line 821, in _request
num = self._async_request(type(None), t, *arg)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/paramiko/sftp_client.py", line 846, in _async_request
self._send_packet(t, msg)
File "/usr/lib/python3/dist-packages/paramiko/sftp.py", line 198, in _send_packet
self._write_all(out)
File "/usr/lib/python3/dist-packages/paramiko/sftp.py", line 162, in _write_all
n = self.sock.send(out)
^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/paramiko/channel.py", line 801, in send
return self._send(s, m)
^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/paramiko/channel.py", line 1198, in _send
raise socket.error("Socket is closed")
OSError: Socket is closed
I was not able to get around this error without closing and restarting the Python interactive interpreter itself. Note that this error takes a while. Directly after loosing connectivity, or enabling VPN, the listdir call hangs multiple minutes before I loose patience. I tried unsuccessfully:
o.close()o.fs.close() # no attribute "close"- Simply reopening the same SSH connection:
fsspec.open("ssh://user@127.0.0.1").fs.listdir('/') o.fs.client.close(). This fixes the hang, but now I get theSocket is closederror, which I also got after ~30 minutes (timeout?), with no way to reopen the same connection again.- quit ipython3 and reopen it: works fine
I am especially stumped as to why the last one did not work. It seems that SSH connections are somehow cached ad some layer.
I also looked into the paramiko API specification and found this:
close()
Close this SSHClient and its underlying Transport.
This should be called anytime you are done using the client object.
Warning
Paramiko registers garbage collection hooks that will try to automatically close connections for you, but this is not presently reliable. Failure to explicitly close your client after use may lead to end-of-process hangs!
The fsspec SFTP implementation never calls this close method on the Client. As far as I can see, the client is only opened, never closed.
I would expect all other remote file systems to have similar issues, i.e., how can I ensure that the connection is properly closed?
Edit: I noticed that there is some kind of caching going on. This would explain why simply trying to reopen the SSH mount did not work.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with fsspec/implementations/sftp.py, especially the client-opening code and the caching referenced in the issue. Reproduce the lost-connectivity sequence, then inspect how filesystem and client lifecycle methods are exposed. Done means the affected SSH filesystem can be explicitly closed and reopened after a broken connection without restarting the interpreter; check for existing lifecycle tests before adding coverage.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, networking
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100