dask / dask/dask-examples

Occasional failure in HTTP bytes

Open
#85 7 comments 0 reactions 1 assignee Claimed by @martindurant View on GitHub
help wanted
Dominant language
Jupyter Notebook
Stars
387
Forks
225
PR merge metrics
No merged PRs in 30d

Description

When running CI in this project I sometimes run across the following error:

```python-traceback
~/miniconda/envs/test/lib/python3.7/site-packages/dask/bag/core.py in reify()
1603 def reify(seq):
1604 if isinstance(seq, Iterator):
-> 1605 seq = list(seq)
1606 if seq and isinstance(seq[0], Iterator):
1607 seq = list(map(list, seq))
~/miniconda/envs/test/lib/python3.7/site-packages/dask/bag/core.py in map_chunk()
1769 yield f(**k)
1770 else:
-> 1771 for a in zip(*args):
1772 yield f(*a)
1773
~/miniconda/envs/test/lib/python3.7/site-packages/dask/bag/text.py in file_to_blocks()
103 def file_to_blocks(lazy_file):
104 with lazy_file as f:
--> 105 for line in f:
106 yield line
107
~/miniconda/envs/test/lib/python3.7/site-packages/dask/bytes/http.py in read()
247 # EOF (python files don't error, just return no data)
248 return b''
--> 249 self. _fetch(self.loc, end)
250 data = self.cache[self.loc - self.start:end - self.start]
251 self.loc = end
~/miniconda/envs/test/lib/python3.7/site-packages/dask/bytes/http.py in _fetch()
258 self.start = start
259 self.end = end + self.blocksize
--> 260 self.cache = self._fetch_range(start, self.end)
261 elif start < self.start:
262 if self.end - end > self.blocksize:
~/miniconda/envs/test/lib/python3.7/site-packages/dask/bytes/http.py in _fetch_range()
320 if cl <= end - start:
321 # data size OK
--> 322 return r.content
323 else:
324 raise ValueError('Got more bytes (%i) than requested (%i)' % (
~/miniconda/envs/test/lib/python3.7/site-packages/requests/models.py in content()
826 self._content = None
827 else:
--> 828 self._content = b''.join(self.iter_content(CONTENT_CHUNK_SIZE)) or b''
829
830 self._content_consumed = True
~/miniconda/envs/test/lib/python3.7/site-packages/requests/models.py in generate()
751 yield chunk
752 except ProtocolError as e:
--> 753 raise ChunkedEncodingError(e)
754 except DecodeError as e:
755 raise ContentDecodingError(e)
ChunkedEncodingError: ('Connection broken: OSError("(104, \'ECONNRESET\')")', OSError("(104, 'ECONNRESET')"))
ChunkedEncodingError: ('Connection broken: OSError("(104, \'ECONNRESET\')")', OSError("(104, 'ECONNRESET')"))
You can ignore this error by setting the following in conf.py:
nbsphinx_allow_errors = True
Notebook error:
CellExecutionError in applications/json-data-on-the-web.ipynb:
------------------
df.spec.value_counts().nlargest(20).to_frame().compute()
------------------

```

@martindurant , this seems to be in your general domain. Do you have any suggestions on what might be happening here?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.