fsspec / fsspec/filesystem_spec

fsspec.asyn.sync shoud check the liveness of IO thread to avoid deadlocks

Open
#1,723 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1.4k
Forks
490
Avg merge
2d 3h
Merged PRs (30d)
38

Description

fsspec.asyn creates and runs an event loop used by async file system implementations as the default event loop.

However, this module does not explicitly close the event loop. As a result, when a Python interpreter enters the shutdown sequence, we experience a specific period during which the event loop is still marked as "running", but the IO thread running the loop has already stopped.

import fsspec.asyn

class Dummy:
    def __init__(self):
        self.loop = fsspec.asyn.get_loop()

    def __del__(self):
        print(fsspec.asyn.loop[0])
        print(fsspec.asyn.iothread[0])

dummy = None

if __name__ == "__main__":
    dummy = Dummy()
<_UnixSelectorEventLoop running=True closed=False debug=False> # Event loop is marked as "running"
<Thread(fsspecIO, stopped daemon 140283887175232)> # IO thread has stopped -> call of sync() with the loop will lead to a deadlock

This period is dangerous as it can lead to unexpected deadlocks, particularly if there are unclosed files that potentially trigger file system access when they are garbage-collected at the final moment of the interpreter shutdown, as I reported in #1685.

To mitigate this risk, we should add a liveness check of the IO thread in fsspec.asyn.sync() widely used by async file system implementations to run async functions synchronously.

def sync(loop, func, *args, timeout=None, **kwargs):

    if loop is None or loop.is_closed():
        raise RuntimeError("Loop is not running")
    if iothread[0] and loop == get_loop() and not iothread[0].is_alive():  # New
        raise RuntimeError("fsspec IO thread has already stopped.")

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with fsspec.asyn.sync() and inspect how get_loop() and iothread are used. Reproduce the interpreter-shutdown scenario described in the issue and verify that calling sync() after the IO thread stops raises the intended RuntimeError instead of deadlocking.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.