Wrong exception is raised at keyboard interrupt and application never finishes
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 9
- Forks
- 3
- PR merge metrics
- No merged PRs in 30d
Description
When a keyboard interrupt is sent while the libzim creator has already been created and is procesing entries, wrong exception is raised and the scraper never finishes and stays stuck.
^C[devdocs2zim::2024-09-10 07:12:10,459] ERROR:Traceback (most recent call last):
File "libzim/libzim.pyx", line 98, in libzim.string_cy_call_fct
File "libzim/libzim.pyx", line 84, in libzim.call_method
AttributeError: 'StaticItem' object has no attribute 'get_mimetype'
Traceback (most recent call last):
File "/home/benoit/Repos/openzim/devdocs/src/devdocs2zim/entrypoint.py", line 90, in main
).run()
^^^^^
File "/home/benoit/Repos/openzim/devdocs/src/devdocs2zim/generator.py", line 366, in run
self.generate_zim(
File "/home/benoit/Repos/openzim/devdocs/src/devdocs2zim/generator.py", line 423, in generate_zim
self.add_zim_contents(
File "/home/benoit/Repos/openzim/devdocs/src/devdocs2zim/generator.py", line 499, in add_zim_contents
creator.add_item_for( # type: ignore
File "/home/benoit/Repos/openzim/devdocs/.hatch/devdocs2zim/lib/python3.12/site-packages/zimscraperlib/zim/creator.py", line 367, in add_item_for
self.add_item(
File "/home/benoit/Repos/openzim/devdocs/.hatch/devdocs2zim/lib/python3.12/site-packages/zimscraperlib/zim/creator.py", line 404, in add_item
raise exc
File "/home/benoit/Repos/openzim/devdocs/.hatch/devdocs2zim/lib/python3.12/site-packages/zimscraperlib/zim/creator.py", line 401, in add_item
super().add_item(item)
File "libzim/libzim.pyx", line 358, in libzim._Creator.add_item
RuntimeError: Traceback (most recent call last):
File "libzim/libzim.pyx", line 98, in libzim.string_cy_call_fct
File "libzim/libzim.pyx", line 84, in libzim.call_method
AttributeError: 'StaticItem' object has no attribute 'get_mimetype'
I strongly suspect this is linked to both wrong exception handling in pylibzim and/or a problem freeing C++ resources.
I'll open an issue in pylibzim, because I strongly suspect this is an upstream issue.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in src/devdocs2zim/entrypoint.py at main, then trace generator.py through run, generate_zim, and add_zim_contents to the creator.add_item_for call shown in the traceback. Reproduce the keyboard interrupt and determine whether the incorrect exception or resource cleanup is handled here or in the suspected upstream pylibzim dependency. Done means interruption raises the correct exception and the scraper exits instead of remaining stuck.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100