openml / openml/OpenML

IncompleteRead, ProtocolError and ChunkedEncodingError on task 168297 (dataset 41081)

Open
#755 10 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
PHP
Stars
755
Forks
128
PR merge metrics
No merged PRs in 30d

Description

Running task=oml.tasks.get_task(168297) generates the following error:

---------------------------------------------------------------------------
IncompleteRead                            Traceback (most recent call last)
c:\users\illia\appdata\local\conda\conda\envs\tensorflow-gpu\lib\site-packages\urllib3\response.py in _error_catcher(self)
    301             try:
--> 302                 yield
    303 

c:\users\illia\appdata\local\conda\conda\envs\tensorflow-gpu\lib\site-packages\urllib3\response.py in read_chunked(self, amt, decode_content)
    600                     break
--> 601                 chunk = self._handle_chunk(amt)
    602                 decoded = self._decode(chunk, decode_content=decode_content,

c:\users\illia\appdata\local\conda\conda\envs\tensorflow-gpu\lib\site-packages\urllib3\response.py in _handle_chunk(self, amt)
    566             returned_chunk = self._fp._safe_read(self.chunk_left)
--> 567             self._fp._safe_read(2)  # Toss the CRLF at the end of the chunk.
    568             self.chunk_left = None

c:\users\illia\appdata\local\conda\conda\envs\tensorflow-gpu\lib\http\client.py in _safe_read(self, amt)
    613             if not chunk:
--> 614                 raise IncompleteRead(b''.join(s), amt)
    615             s.append(chunk)

IncompleteRead: IncompleteRead(0 bytes read, 2 more expected)

During handling of the above exception, another exception occurred:

ProtocolError                             Traceback (most recent call last)
c:\users\illia\appdata\local\conda\conda\envs\tensorflow-gpu\lib\site-packages\requests\models.py in generate()
    744                 try:
--> 745                     for chunk in self.raw.stream(chunk_size, decode_content=True):
    746                         yield chunk

c:\users\illia\appdata\local\conda\conda\envs\tensorflow-gpu\lib\site-packages\urllib3\response.py in stream(self, amt, decode_content)
    431         if self.chunked and self.supports_chunked_reads():
--> 432             for line in self.read_chunked(amt, decode_content=decode_content):
    433                 yield line

c:\users\illia\appdata\local\conda\conda\envs\tensorflow-gpu\lib\site-packages\urllib3\response.py in read_chunked(self, amt, decode_content)
    625             if self._original_response:
--> 626                 self._original_response.close()

c:\users\illia\appdata\local\conda\conda\envs\tensorflow-gpu\lib\contextlib.py in __exit__(self, type, value, traceback)
     99             try:
--> 100                 self.gen.throw(type, value, traceback)
    101                 raise RuntimeError("generator didn't stop after throw()")

c:\users\illia\appdata\local\conda\conda\envs\tensorflow-gpu\lib\site-packages\urllib3\response.py in _error_catcher(self)
    319                 # This includes IncompleteRead.
--> 320                 raise ProtocolError('Connection broken: %r' % e, e)
    321 

ProtocolError: ('Connection broken: IncompleteRead(0 bytes read, 2 more expected)', IncompleteRead(0 bytes read, 2 more expected))

During handling of the above exception, another exception occurred:

ChunkedEncodingError                      Traceback (most recent call last)
<ipython-input-2-c5e38289fce1> in <module>()
      6 #cifar10_task=oml.tasks.get_task(167124)
      7 #cifar10small_task=oml.tasks.get_task(167133)
----> 8 svhn_task=task=oml.tasks.get_task(168297)
      9 #olivettifaces_task=task=oml.tasks.get_task(168299)
     10 #umistfacescropped_task=task=oml.tasks.get_task(168300)

D:\irfan\kuliah\TUE\Q7\thesis\openml-python\openml\tasks\functions.py in get_task(task_id)
    257         except Exception as e:
    258             _remove_task_cache_dir(tid_cache_dir)
--> 259             raise e
    260 
    261     return task

D:\irfan\kuliah\TUE\Q7\thesis\openml-python\openml\tasks\functions.py in get_task(task_id)
    250         try:
    251             task = _get_task_description(task_id)
--> 252             dataset = get_dataset(task.dataset_id)
    253             class_labels = dataset.retrieve_class_labels(task.target_name)
    254             task.class_labels = class_labels

D:\irfan\kuliah\TUE\Q7\thesis\openml-python\openml\datasets\functions.py in get_dataset(dataset_id)
    297         except Exception as e:
    298             _remove_dataset_cache_dir(did_cache_dir)
--> 299             raise e
    300 
    301         dataset = _create_dataset_from_description(

D:\irfan\kuliah\TUE\Q7\thesis\openml-python\openml\datasets\functions.py in get_dataset(dataset_id)
    292         try:
    293             description = _get_dataset_description(did_cache_dir, dataset_id)
--> 294             arff_file = _get_dataset_arff(did_cache_dir, description)
    295             features = _get_dataset_features(did_cache_dir, dataset_id)
    296             qualities = _get_dataset_qualities(did_cache_dir, dataset_id)

D:\irfan\kuliah\TUE\Q7\thesis\openml-python\openml\datasets\functions.py in _get_dataset_arff(did_cache_dir, description)
    380 
    381     url = description['oml:url']
--> 382     arff_string = _read_url(url)
    383     md5 = hashlib.md5()
    384     md5.update(arff_string.encode('utf8'))

D:\irfan\kuliah\TUE\Q7\thesis\openml-python\openml\_api_calls.py in _read_url(url, data)
    111     if len(data) == 0 or (len(data) == 1 and 'api_key' in data):
    112         # do a GET
--> 113         response = requests.get(url, params=data)
    114     else: # an actual post request
    115         # Using requests.post sets header 'Accept-encoding' automatically to

c:\users\illia\appdata\local\conda\conda\envs\tensorflow-gpu\lib\site-packages\requests\api.py in get(url, params, **kwargs)
     70 
     71     kwargs.setdefault('allow_redirects', True)
---> 72     return request('get', url, params=params, **kwargs)
     73 
     74 

c:\users\illia\appdata\local\conda\conda\envs\tensorflow-gpu\lib\site-packages\requests\api.py in request(method, url, **kwargs)
     56     # cases, and look like a memory leak in others.
     57     with sessions.Session() as session:
---> 58         return session.request(method=method, url=url, **kwargs)
     59 
     60 

c:\users\illia\appdata\local\conda\conda\envs\tensorflow-gpu\lib\site-packages\requests\sessions.py in request(self, method, url, params, data, headers, cookies, files, auth, timeout, allow_redirects, proxies, hooks, stream, verify, cert, json)
    506         }
    507         send_kwargs.update(settings)
--> 508         resp = self.send(prep, **send_kwargs)
    509 
    510         return resp

c:\users\illia\appdata\local\conda\conda\envs\tensorflow-gpu\lib\site-packages\requests\sessions.py in send(self, request, **kwargs)
    656 
    657         if not stream:
--> 658             r.content
    659 
    660         return r

c:\users\illia\appdata\local\conda\conda\envs\tensorflow-gpu\lib\site-packages\requests\models.py in content(self)
    821                 self._content = None
    822             else:
--> 823                 self._content = bytes().join(self.iter_content(CONTENT_CHUNK_SIZE)) or bytes()
    824 
    825         self._content_consumed = True

c:\users\illia\appdata\local\conda\conda\envs\tensorflow-gpu\lib\site-packages\requests\models.py in generate()
    746                         yield chunk
    747                 except ProtocolError as e:
--> 748                     raise ChunkedEncodingError(e)
    749                 except DecodeError as e:
    750                     raise ContentDecodingError(e)

ChunkedEncodingError: ('Connection broken: IncompleteRead(0 bytes read, 2 more expected)', IncompleteRead(0 bytes read, 2 more expected))

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce oml.tasks.get_task(168297) and trace the failure through openml/tasks/functions.py, openml/datasets/functions.py, and openml/_api_calls.py, especially _get_dataset_arff and _read_url. Determine whether the incomplete dataset download is handled by the client or depends on the remote response; done means the failure behavior or recovery path is reproducible and verified for this task.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.