GoogleCloudPlatform / GoogleCloudPlatform/gsutil
"gsutil cp gs://[...] -" fails with "httplib.BadStatusLine"
- Dominant language
- Python
- Stars
- 918
- Forks
- 335
- PR merge metrics
- No merged PRs in 30d
Description
While streaming a large (multi-TB) file to stdout using `gsutil cp gs://[...] -` it usually crashes with the following exception after a couple hours:
```
File "/snap/google-cloud-sdk/126/platform/gsutil/gsutil", line 21, in
gsutil.RunMain()
File "/snap/google-cloud-sdk/126/platform/gsutil/gsutil.py", line 124, in RunMain
sys.exit(gslib.__main__.main())
File "/snap/google-cloud-sdk/126/platform/gsutil/gslib/__main__.py", line 433, in main
user_project=user_project)
File "/snap/google-cloud-sdk/126/platform/gsutil/gslib/__main__.py", line 762, in _RunNamedCommandAndHandleExceptions
_HandleUnknownFailure(e)
File "/snap/google-cloud-sdk/126/platform/gsutil/gslib/__main__.py", line 628, in _RunNamedCommandAndHandleExceptions
user_project=user_project)
File "/snap/google-cloud-sdk/126/platform/gsutil/gslib/command_runner.py", line 411, in RunNamedCommand
return_code = command_inst.RunCommand()
File "/snap/google-cloud-sdk/126/platform/gsutil/gslib/commands/cp.py", line 1145, in RunCommand
cat_out_fd=cat_out_fd)
File "/snap/google-cloud-sdk/126/platform/gsutil/gslib/utils/cat_helper.py", line 152, in CatUrlStrings
provider=storage_url.scheme)
File "/snap/google-cloud-sdk/126/platform/gsutil/gslib/cloud_api_delegator.py", line 353, in GetObjectMedia
decryption_tuple=decryption_tuple)
File "/snap/google-cloud-sdk/126/platform/gsutil/gslib/gcs_json_api.py", line 1205, in GetObjectMedia
decryption_tuple=decryption_tuple)
File "/snap/google-cloud-sdk/126/platform/gsutil/gslib/gcs_json_api.py", line 1342, in _PerformDownload
use_chunks=False)
File "/snap/google-cloud-sdk/126/platform/gsutil/third_party/apitools/apitools/base/py/transfer.py", line 536, in StreamMedia
additional_headers=additional_headers)
File "/snap/google-cloud-sdk/126/platform/gsutil/third_party/apitools/apitools/base/py/transfer.py", line 415, in __GetChunk
retries=self.num_retries)
File "/snap/google-cloud-sdk/126/platform/gsutil/third_party/apitools/apitools/base/py/http_wrapper.py", line 356, in MakeRequest
max_retry_wait, total_wait_sec))
File "/snap/google-cloud-sdk/126/platform/gsutil/gslib/utils/retry_util.py", line 84, in RetriesInDataTransferHandler
http_wrapper.RethrowExceptionHandler(retry_args)
File "/snap/google-cloud-sdk/126/platform/gsutil/third_party/apitools/apitools/base/py/http_wrapper.py", line 346, in MakeRequest
check_response_func=check_response_func)
File "/snap/google-cloud-sdk/126/platform/gsutil/third_party/apitools/apitools/base/py/http_wrapper.py", line 396, in _MakeRequestNoRetry
redirections=redirections, connection_type=connection_type)
File "/snap/google-cloud-sdk/126/platform/gsutil/gslib/gcs_json_media.py", line 546, in NewRequest
connection_type=connection_type)
File "/snap/google-cloud-sdk/126/platform/gsutil/third_party/oauth2client/oauth2client/transport.py", line 175, in new_request
redirections, connection_type)
File "/snap/google-cloud-sdk/126/platform/gsutil/third_party/oauth2client/oauth2client/transport.py", line 282, in request
connection_type=connection_type)
File "/snap/google-cloud-sdk/126/platform/gsutil/third_party/httplib2/python2/httplib2/__init__.py", line 1694, in request
(response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey)
File "/snap/google-cloud-sdk/126/platform/gsutil/gslib/gcs_json_media.py", line 453, in OverrideRequest
headers)
File "/snap/google-cloud-sdk/126/platform/gsutil/gslib/gcs_json_media.py", line 668, in _conn_request
response = conn.getresponse()
File "/snap/google-cloud-sdk/126/platform/gsutil/gslib/gcs_json_media.py", line 369, in getresponse
orig_response = http_client.HTTPConnection.getresponse(self)
File "/snap/google-cloud-sdk/126/usr/lib/python2.7/httplib.py", line 1152, in getresponse
response.begin()
File "/snap/google-cloud-sdk/126/usr/lib/python2.7/httplib.py", line 463, in begin
version, status, reason = self._read_status()
File "/snap/google-cloud-sdk/126/usr/lib/python2.7/httplib.py", line 427, in _read_status
raise BadStatusLine(line)
httplib.BadStatusLine: ''
```
My version information is as follows:
```
$ gcloud --version
Google Cloud SDK 288.0.0
alpha 2020.04.03
beta 2020.04.03 bq 2.0.56
core 2020.04.03
gsutil 4.49
kubectl 2020.04.03
```
Contributor guide
Assessment
This issue has not been assessed yet.