fsspec / fsspec/filesystem_spec

Keyboard interrupt does not work properly with ipython and TqdmCallback

Open
#1,667 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1.4k
Forks
490
Avg merge
2d 3h
Merged PRs (30d)
38

Description

Hi there,

It looks like keyboard interrupt in ipython does not work properly when downloading files using TqdmCallback.
I noticed it while downloading a ~1GB file:

URL = ...

import fsspec
fs = fsspec.filesystem("http", asynchronous=False)
fs.get_file(URL, "test-file", callback=fsspec.callbacks.TqdmCallback())

When I hit Ctrl+C, the download doesn't stop and continue in async mode. The traceback I get shows this:

---------------------------------------------------------------------------
KeyboardInterrupt                         Traceback (most recent call last)
Cell In[3], line 3
      1 import fsspec
      2 fs = fsspec.filesystem("http", asynchronous=False)
----> 3 fs.get_file(URL, "test-file", callback=fsspec.callbacks.TqdmCallback())

File ~/miniforge3/envs/fsspec/lib/python3.12/site-packages/fsspec/asyn.py:118, in sync_wrapper.<locals>.wrapper(*args, **kwargs)
    115 @functools.wraps(func)
    116 def wrapper(*args, **kwargs):
    117     self = obj or args[0]
--> 118     return sync(self.loop, func, *args, **kwargs)

File ~/miniforge3/envs/fsspec/lib/python3.12/site-packages/fsspec/asyn.py:91, in sync(loop, func, timeout, *args, **kwargs)
     88 asyncio.run_coroutine_threadsafe(_runner(event, coro, result, timeout), loop)
     89 while True:
     90     # this loops allows thread to get interrupted
---> 91     if event.wait(1):
     92         break
     93     if timeout is not None:

File ~/miniforge3/envs/fsspec/lib/python3.12/threading.py:655, in Event.wait(self, timeout)
    653 signaled = self._flag
    654 if not signaled:
--> 655     signaled = self._cond.wait(timeout)
    656 return signaled

File ~/miniforge3/envs/fsspec/lib/python3.12/threading.py:359, in Condition.wait(self, timeout)
    357 else:
    358     if timeout > 0:
--> 359         gotit = waiter.acquire(True, timeout)
    360     else:
    361         gotit = waiter.acquire(False)

KeyboardInterrupt:

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the sync entry point in fsspec/asyn.py, especially sync_wrapper and sync, then trace how fsspec.callbacks.TqdmCallback is used during fs.get_file. Reproduce the Ctrl+C case in IPython with the provided download example; done means the interrupt stops the download instead of allowing it to continue in async mode.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.