Import Documents: crash on bad URL
Open
Nobody has claimed this yet.
bug
help wanted
snack
- Dominant language
- Python
- Stars
- 134
- Forks
- 86
- Avg merge
- 6d 5h
- Merged PRs (30d)
- 1
Description
Describe the bug
--------------------------- RuntimeError Exception ----------------------------
Traceback (most recent call last):
File "/Users/ajda/orange/orange3-text/orangecontrib/text/import_documents.py", line 244, in _read_files
return await asyncio.gather(*req)
File "/Users/ajda/orange/orange3-text/orangecontrib/text/import_documents.py", line 256, in _read_file
return UrlProxyReader._parse_response(response)
File "/Users/ajda/orange/orange3-text/orangecontrib/text/import_documents.py", line 273, in _parse_response
text_data.ext,
AttributeError: 'NoneType' object has no attribute 'ext'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/ajda/orange/orange3-text/orangecontrib/text/widgets/owimportdocuments.py", line 611, in __onRunFinished
corpus, errors, lemmas, pos, ner, is_conllu = task.future.result()
File "/Users/ajda/.pyenv-x86/versions/3.9.10/lib/python3.9/concurrent/futures/_base.py", line 439, in result
return self.__get_result()
File "/Users/ajda/.pyenv-x86/versions/3.9.10/lib/python3.9/concurrent/futures/_base.py", line 391, in __get_result
raise self._exception
File "/Users/ajda/orange/orange3/Orange/widgets/utils/concurrent.py", line 175, in run
result = func(*args, **kwargs)
File "/Users/ajda/orange/orange3-text/orangecontrib/text/widgets/owimportdocuments.py", line 589, in run_text_scan_task_interupt
return task.run()
File "/Users/ajda/orange/orange3-text/orangecontrib/text/import_documents.py", line 399, in run
self._text_data, errors_text, tokens, pos, ner, conllu = self._read_text_data(
File "/Users/ajda/orange/orange3-text/orangecontrib/text/import_documents.py", line 443, in _read_text_data
results = UrlProxyReader().read_files(paths, callback)
File "/Users/ajda/orange/orange3-text/orangecontrib/text/import_documents.py", line 238, in read_files
return asyncio.run(UrlProxyReader._read_files(urls, callback))
File "/Users/ajda/.pyenv-x86/versions/3.9.10/lib/python3.9/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "/Users/ajda/.pyenv-x86/versions/3.9.10/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
return future.result()
File "/Users/ajda/orange/orange3-text/orangecontrib/text/import_documents.py", line 244, in _read_files
return await asyncio.gather(*req)
File "/Users/ajda/.pyenv-x86/versions/3.9.10/envs/py3.9/lib/python3.9/site-packages/httpx/_client.py", line 1975, in __aexit__
await self._transport.__aexit__(exc_type, exc_value, traceback)
File "/Users/ajda/.pyenv-x86/versions/3.9.10/envs/py3.9/lib/python3.9/site-packages/httpx/_transports/default.py", line 332, in __aexit__
await self._pool.__aexit__(exc_type, exc_value, traceback)
File "/Users/ajda/.pyenv-x86/versions/3.9.10/envs/py3.9/lib/python3.9/site-packages/httpcore/_async/connection_pool.py", line 326, in __aexit__
await self.aclose()
File "/Users/ajda/.pyenv-x86/versions/3.9.10/envs/py3.9/lib/python3.9/site-packages/httpcore/_async/connection_pool.py", line 312, in aclose
raise RuntimeError(
RuntimeError: The connection pool was closed while 1 HTTP requests/responses were still in-flight.
-------------------------------------------------------------------------------
To Reproduce
Steps to reproduce the behavior:
- Import Documents
- Pass a bad URL, I entered http://file.biolab.si/text-semantics/, which is a folder of many subfolders.
Expected behavior
Error of some sort is reported.
Orange version:
3.32.dev
Text add-on version:
1.7.dev
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in orangecontrib/text/import_documents.py at UrlProxyReader._read_files, _read_file, and _parse_response, then trace how errors reach __onRunFinished in orangecontrib/text/widgets/owimportdocuments.py. Reproduce the issue through Import Documents with the bad folder URL, and verify that an error is reported without either AttributeError or the connection-pool RuntimeError.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 45/100