Upload/Download bug with Azure Blobs
- Lingua principale
- Python
- Stelle
- 2.1k
- Fork
- 931
- Merge medio
- 1g 2h
- PR unite (30g)
- 4
Descrizione
## Summary
We are facing some issues interacting with Azure Blobs. Whether it is for uploading or downloading, it happens quite often that we get an error like below. This problem frequency increases dramatically with the size of the file we are interacting. For a large file ~100Go, it is almost impossible to run anything without encountering this. Is it something known?
## Detailed Information
Libcloud version: apache-libcloud==3.1.0
python version: 3.7.4
OS: Ubuntu
Within the dask docker image
Stacktrace:
`[2020-07-17 12:25:46] ERROR - prefect.TaskRunner | Unexpected error: ChunkedEncodingError(ProtocolError('Connection broken: OSError("(104, \'ECONNRESET\')")', OSError("(104, 'ECONNRESET')")))
Traceback (most recent call last):
File "/opt/conda/lib/python3.7/site-packages/urllib3/contrib/pyopenssl.py", line 297, in recv_into
return self.connection.recv_into(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/OpenSSL/SSL.py", line 1822, in recv_into
self._raise_ssl_error(self._ssl, result)
File "/opt/conda/lib/python3.7/site-packages/OpenSSL/SSL.py", line 1639, in _raise_ssl_error
raise SysCallError(errno, errorcode.get(errno))
OpenSSL.SSL.SysCallError: (104, 'ECONNRESET')
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/conda/lib/python3.7/site-packages/urllib3/response.py", line 360, in _error_catcher
yield
File "/opt/conda/lib/python3.7/site-packages/urllib3/response.py", line 442, in read
data = self._fp.read(amt)
File "/opt/conda/lib/python3.7/http/client.py", line 457, in read
n = self.readinto(b)
File "/opt/conda/lib/python3.7/http/client.py", line 501, in readinto
n = self.fp.readinto(b)
File "/opt/conda/lib/python3.7/socket.py", line 589, in readinto
return self._sock.recv_into(b)
File "/opt/conda/lib/python3.7/site-packages/urllib3/contrib/pyopenssl.py", line 302, in recv_into
raise SocketError(str(e))
OSError: (104, 'ECONNRESET')
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/conda/lib/python3.7/site-packages/requests/models.py", line 750, in generate
for chunk in self.raw.stream(chunk_size, decode_content=True):
File "/opt/conda/lib/python3.7/site-packages/urllib3/response.py", line 494, in stream
data = self.read(amt=amt, decode_content=decode_content)
File "/opt/conda/lib/python3.7/site-packages/urllib3/response.py", line 459, in read
raise IncompleteRead(self._fp_bytes_read, self.length_remaining)
File "/opt/conda/lib/python3.7/contextlib.py", line 130, in __exit__
self.gen.throw(type, value, traceback)
File "/opt/conda/lib/python3.7/site-packages/urllib3/response.py", line 378, in _error_catcher
raise ProtocolError('Connection broken: %r' % e, e)
urllib3.exceptions.ProtocolError: ('Connection broken: OSError("(104, \'ECONNRESET\')")', OSError("(104, 'ECONNRESET')"))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/conda/lib/python3.7/site-packages/prefect/engine/runner.py", line 48, in inner
new_state = method(self, state, *args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/prefect/engine/task_runner.py", line 943, in get_task_run_state
self.task.run, timeout=self.task.timeout, **raw_inputs
File "/opt/conda/lib/python3.7/site-packages/prefect/utilities/executors.py", line 182, in timeout_handler
return fn(*args, **kwargs)
File "xxxxx, line 127, in run
video_file.name
File "xxxxx", line 116, in xxxx
for k, line in enumerate(xxxx):
File "/opt/conda/lib/python3.7/site-packages/libcloud/utils/files.py", line 69, in read_in_chunks
chunk = b(get_data(*args))
File "/opt/conda/lib/python3.7/site-packages/requests/models.py", line 753, in generate
raise ChunkedEncodingError(e)
requests.exceptions.ChunkedEncodingError: ('Connection broken: OSError("(104, \'ECONNRESET\')")', OSError("(104, 'ECONNRESET')"))`
Guida per i contributori
Apri la guida per i contributori
Direzione di ricerca
Start with libcloud/utils/files.py at read_in_chunks and trace how Azure blob uploads and downloads use requests while reproducing the ECONNRESET failure with a large file. Compare the observed behavior with the stack trace in requests/models.py; done means identifying the failure boundary and documenting the cause or limitation, since the issue does not specify a proposed fix.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Valutazione
- Stack tecnologico
- azure, python
- Ambito
- cloud
- Tipo di issue
- Bug
- Difficoltà
- 4/5
- Tempo stimato
- 3-5 giorni
- Stato di attività
- Ferma
- Chiarezza
- Da chiarire
- Idoneità per principianti
- 25/100