dropbox / dropbox/dropbox-sdk-python

ReadTimeout

Aberta
#155 15 comentários 0 reações 0 responsáveis Ver no GitHub
bug
Linguagem predominante
Python
Estrelas
983
Forks
330
Merge médio
3min
PRs com merge (30d)
10

Descrição

There is an error while retrieving list of files from folder with almost 100k files. But all files were retrieved successfully.
```
Traceback (most recent call last):
File ".pyenv/versions/marketing-ua-reports/lib/python3.6/site-packages/urllib3/connectionpool.py", line 386, in _make_request
six.raise_from(e, None)
File "", line 2, in raise_from
File ".pyenv/versions/marketing-ua-reports/lib/python3.6/site-packages/urllib3/connectionpool.py", line 382, in _make_request
httplib_response = conn.getresponse()
File ".pyenv/versions/3.6.4/lib/python3.6/http/client.py", line 1331, in getresponse
response.begin()
File ".pyenv/versions/3.6.4/lib/python3.6/http/client.py", line 297, in begin
version, status, reason = self._read_status()
File ".pyenv/versions/3.6.4/lib/python3.6/http/client.py", line 258, in _read_status
line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
File ".pyenv/versions/3.6.4/lib/python3.6/socket.py", line 586, in readinto
return self._sock.recv_into(b)
File ".pyenv/versions/marketing-ua-reports/lib/python3.6/site-packages/urllib3/contrib/pyopenssl.py", line 291, in recv_into
raise timeout('The read operation timed out')
socket.timeout: The read operation timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File ".pyenv/versions/marketing-ua-reports/lib/python3.6/site-packages/requests/adapters.py", line 445, in send
timeout=timeout
File ".pyenv/versions/marketing-ua-reports/lib/python3.6/site-packages/urllib3/connectionpool.py", line 649, in urlopen
_stacktrace=sys.exc_info()[2])
File ".pyenv/versions/marketing-ua-reports/lib/python3.6/site-packages/urllib3/util/retry.py", line 357, in increment
raise six.reraise(type(error), error, _stacktrace)
File ".pyenv/versions/marketing-ua-reports/lib/python3.6/site-packages/urllib3/packages/six.py", line 686, in reraise
raise value
File ".pyenv/versions/marketing-ua-reports/lib/python3.6/site-packages/urllib3/connectionpool.py", line 600, in urlopen
chunked=chunked)
File ".pyenv/versions/marketing-ua-reports/lib/python3.6/site-packages/urllib3/connectionpool.py", line 388, in _make_request
self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
File ".pyenv/versions/marketing-ua-reports/lib/python3.6/site-packages/urllib3/connectionpool.py", line 308, in _raise_timeout
raise ReadTimeoutError(self, url, "Read timed out. (read timeout=%s)" % timeout_value)
urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='api.dropboxapi.com', port=443): Read timed out. (read timeout=30)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "apps/creatives_manager/processors/base.py", line 144, in _get_files_from_dropbox
result = self.dropbox.files_list_folder_continue(result.cursor)
File ".pyenv/versions/marketing-ua-reports/lib/python3.6/site-packages/dropbox/base.py", line 1583, in files_list_folder_continue
None,
File ".pyenv/versions/marketing-ua-reports/lib/python3.6/site-packages/dropbox/dropbox.py", line 274, in request
timeout=timeout)
File ".pyenv/versions/marketing-ua-reports/lib/python3.6/site-packages/dropbox/dropbox.py", line 365, in request_json_string_with_retry
timeout=timeout)
File ".pyenv/versions/marketing-ua-reports/lib/python3.6/site-packages/dropbox/dropbox.py", line 449, in request_json_string
timeout=timeout,
File ".pyenv/versions/marketing-ua-reports/lib/python3.6/site-packages/requests/sessions.py", line 559, in post
return self.request('POST', url, data=data, json=json, **kwargs)
File ".pyenv/versions/marketing-ua-reports/lib/python3.6/site-packages/requests/sessions.py", line 512, in request
resp = self.send(prep, **send_kwargs)
File ".pyenv/versions/marketing-ua-reports/lib/python3.6/site-packages/requests/sessions.py", line 622, in send
r = adapter.send(request, **kwargs)
File ".pyenv/versions/marketing-ua-reports/lib/python3.6/site-packages/requests/adapters.py", line 526, in send
raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='api.dropboxapi.com', port=443): Read timed out. (read timeout=30)
```

Repeating of call `self.dropbox.files_list_folder_continue(result.cursor)` and enlarging timeout don't help.

Guia de contribuição

Abrir o guia de contribuição

Direção de pesquisa

Comece em apps/creatives_manager/processors/base.py, em _get_files_from_dropbox, e inspecione a chamada files_list_folder_continue(result.cursor) mostrada no traceback. Reproduza a listagem com uma pasta contendo quase 100k arquivos e determine por que a continuação atinge o tempo limite apesar de ser concluída; considera-se concluído quando o comportamento for corrigido ou seu tratamento esperado estiver claramente estabelecido.

Escrita pelo modelo de indexação a partir do texto da issue.

Avaliação

Stack de tecnologia
python
Domínio
api
Tipo de issue
Bug
Dificuldade
4/5
Tempo estimado
3-5 dias
Status de atividade
Estagnada
Clareza
Precisa de esclarecimento
Facilidade para iniciantes
30/100

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.