GoogleCloudPlatform / GoogleCloudPlatform/gsutil

gsutil crashes on reset connection

Open
#865 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
918
Forks
335
PR merge metrics
No merged PRs in 30d

Description

Sample stack trace from build automation:

Traceback (most recent call last):
File "chromium/src/third_party/depot_tools/external_bin/gsutil/gsutil_4.30/gsutil/gsutil", line 22, in
gsutil.RunMain()
File "chromium/src/third_party/depot_tools/external_bin/gsutil/gsutil_4.30/gsutil/gsutil.py", line 115, in RunMain
sys.exit(gslib.__main__.main())
File "chromium/src/third_party/depot_tools/external_bin/gsutil/gsutil_4.30/gsutil/gslib/__main__.py", line 385, in main
perf_trace_token=perf_trace_token, user_project=user_project)
File "chromium/src/third_party/depot_tools/external_bin/gsutil/gsutil_4.30/gsutil/gslib/__main__.py", line 571, in _RunNamedCommandAndHandleExceptions
user_project=user_project)
File "chromium/src/third_party/depot_tools/external_bin/gsutil/gsutil_4.30/gsutil/gslib/command_runner.py", line 319, in RunNamedCommand
return_code = command_inst.RunCommand()
File "chromium/src/third_party/depot_tools/external_bin/gsutil/gsutil_4.30/gsutil/gslib/commands/cp.py", line 1115, in RunCommand
cat_out_fd=cat_out_fd)
File "chromium/src/third_party/depot_tools/external_bin/gsutil/gsutil_4.30/gsutil/gslib/cat_helper.py", line 140, in CatUrlStrings
provider=storage_url.scheme)
File "chromium/src/third_party/depot_tools/external_bin/gsutil/gsutil_4.30/gsutil/gslib/cloud_api_delegator.py", line 277, in GetObjectMedia
decryption_tuple=decryption_tuple)
File "chromium/src/third_party/depot_tools/external_bin/gsutil/gsutil_4.30/gsutil/gslib/gcs_json_api.py", line 998, in GetObjectMedia
decryption_tuple=decryption_tuple)
File "chromium/src/third_party/depot_tools/external_bin/gsutil/gsutil_4.30/gsutil/gslib/gcs_json_api.py", line 1087, in _PerformDownload
additional_headers=additional_headers, use_chunks=False)
File "chromium/src/third_party/depot_tools/external_bin/gsutil/gsutil_4.30/gsutil/third_party/apitools/apitools/base/py/transfer.py", line 524, in StreamMedia
additional_headers=additional_headers)
File "chromium/src/third_party/depot_tools/external_bin/gsutil/gsutil_4.30/gsutil/third_party/apitools/apitools/base/py/transfer.py", line 413, in __GetChunk
retries=self.num_retries)
File "chromium/src/third_party/depot_tools/external_bin/gsutil/gsutil_4.30/gsutil/third_party/apitools/apitools/base/py/http_wrapper.py", line 356, in MakeRequest
max_retry_wait, total_wait_sec))
File "chromium/src/third_party/depot_tools/external_bin/gsutil/gsutil_4.30/gsutil/gslib/util.py", line 1777, in RetriesInDataTransferHandler
http_wrapper.RethrowExceptionHandler(retry_args)
File "chromium/src/third_party/depot_tools/external_bin/gsutil/gsutil_4.30/gsutil/third_party/apitools/apitools/base/py/http_wrapper.py", line 346, in MakeRequest
check_response_func=check_response_func)
File "chromium/src/third_party/depot_tools/external_bin/gsutil/gsutil_4.30/gsutil/third_party/apitools/apitools/base/py/http_wrapper.py", line 396, in _MakeRequestNoRetry
redirections=redirections, connection_type=connection_type)
File "chromium/src/third_party/depot_tools/external_bin/gsutil/gsutil_4.30/gsutil/gslib/gcs_json_media.py", line 502, in NewRequest
connection_type=connection_type)
File "chromium/src/third_party/depot_tools/external_bin/gsutil/gsutil_4.30/gsutil/third_party/oauth2client/oauth2client/transport.py", line 175, in new_request
redirections, connection_type)
File "chromium/src/third_party/depot_tools/external_bin/gsutil/gsutil_4.30/gsutil/third_party/oauth2client/oauth2client/transport.py", line 282, in request
connection_type=connection_type)
File "chromium/src/third_party/depot_tools/external_bin/gsutil/gsutil_4.30/gsutil/third_party/httplib2/python2/httplib2/__init__.py", line 1659, in request
(response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey)
File "chromium/src/third_party/depot_tools/external_bin/gsutil/gsutil_4.30/gsutil/gslib/gcs_json_media.py", line 409, in OverrideRequest
headers)
File "chromium/src/third_party/depot_tools/external_bin/gsutil/gsutil_4.30/gsutil/gslib/gcs_json_media.py", line 641, in _conn_request
new_data = http_stream.read(TRANSFER_BUFFER_SIZE)
File "chromium/src/third_party/depot_tools/external_bin/gsutil/gsutil_4.30/gsutil/gslib/gcs_json_media.py", line 361, in read
data = orig_read_func(amt)
File "/usr/lib/python2.7/httplib.py", line 597, in read
s = self.fp.read(amt)
File "/usr/lib/python2.7/socket.py", line 384, in read
data = self._sock.recv(left)
File "/usr/lib/python2.7/ssl.py", line 772, in recv
return self.read(buflen)
File "/usr/lib/python2.7/ssl.py", line 659, in read
v = self._sslobj.read(len)
socket.error: [Errno 104] Connection reset by peer

Contributor guide

Open the contributing guide

Research direction

Start with gslib/gcs_json_media.py around _conn_request and read, then follow the retry path through apitools/base/py/http_wrapper.py and gslib/util.py. Reproduce the build-automation download scenario from the stack trace and determine how a reset connection should be handled; done means the command no longer crashes unexpectedly in that case.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli, networking
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.