fsspec / fsspec/filesystem_spec
Keyboard interrupt does not work properly with ipython and TqdmCallback
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.4k
- Forks
- 490
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 38
Description
Hi there,
It looks like keyboard interrupt in ipython does not work properly when downloading files using TqdmCallback.
I noticed it while downloading a ~1GB file:
URL = ...
import fsspec
fs = fsspec.filesystem("http", asynchronous=False)
fs.get_file(URL, "test-file", callback=fsspec.callbacks.TqdmCallback())
When I hit Ctrl+C, the download doesn't stop and continue in async mode. The traceback I get shows this:
---------------------------------------------------------------------------
KeyboardInterrupt Traceback (most recent call last)
Cell In[3], line 3
1 import fsspec
2 fs = fsspec.filesystem("http", asynchronous=False)
----> 3 fs.get_file(URL, "test-file", callback=fsspec.callbacks.TqdmCallback())
File ~/miniforge3/envs/fsspec/lib/python3.12/site-packages/fsspec/asyn.py:118, in sync_wrapper.<locals>.wrapper(*args, **kwargs)
115 @functools.wraps(func)
116 def wrapper(*args, **kwargs):
117 self = obj or args[0]
--> 118 return sync(self.loop, func, *args, **kwargs)
File ~/miniforge3/envs/fsspec/lib/python3.12/site-packages/fsspec/asyn.py:91, in sync(loop, func, timeout, *args, **kwargs)
88 asyncio.run_coroutine_threadsafe(_runner(event, coro, result, timeout), loop)
89 while True:
90 # this loops allows thread to get interrupted
---> 91 if event.wait(1):
92 break
93 if timeout is not None:
File ~/miniforge3/envs/fsspec/lib/python3.12/threading.py:655, in Event.wait(self, timeout)
653 signaled = self._flag
654 if not signaled:
--> 655 signaled = self._cond.wait(timeout)
656 return signaled
File ~/miniforge3/envs/fsspec/lib/python3.12/threading.py:359, in Condition.wait(self, timeout)
357 else:
358 if timeout > 0:
--> 359 gotit = waiter.acquire(True, timeout)
360 else:
361 gotit = waiter.acquire(False)
KeyboardInterrupt:
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the sync entry point in fsspec/asyn.py, especially sync_wrapper and sync, then trace how fsspec.callbacks.TqdmCallback is used during fs.get_file. Reproduce the Ctrl+C case in IPython with the provided download example; done means the interrupt stops the download instead of allowing it to continue in async mode.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100