GoogleCloudPlatform / GoogleCloudPlatform/gsutil
TypeError in gsutil cp and perfdiag due to gs_port in boto configuration
- Dominant language
- Python
- Stars
- 918
- Forks
- 335
- PR merge metrics
- No merged PRs in 30d
Description
The current boto configuration causes gsutil cp and gsutil perfdiag to raise a TypeError exception.
```
[Credentials]
gs_host = xx
gs_port = 8021
gs_json_host = xx
gs_json_port = 8020
```
```
Traceback (most recent call last):
File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/boto_translation.py", line 655, in _PerformSimpleDownload
key.get_contents_to_file(download_stream,
File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/vendored/boto/boto/gs/key.py", line 234, in get_contents_to_file
self.get_file(fp, headers, cb, num_cb, torrent=torrent,
File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/vendored/boto/boto/gs/key.py", line 174, in get_file
self._get_file_internal(fp, headers=headers, cb=cb, num_cb=num_cb,
File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/vendored/boto/boto/s3/key.py", line 1544, in _get_file_internal
self.open('r', headers, query_args=query_args,
File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/vendored/boto/boto/s3/key.py", line 365, in open
self.open_read(headers=headers, query_args=query_args,
File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/vendored/boto/boto/gs/key.py", line 164, in open_read
super(Key, self).open_read(headers=headers, query_args=query_args,
File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/vendored/boto/boto/s3/key.py", line 313, in open_read
self.resp = self.bucket.connection.make_request(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/vendored/boto/boto/s3/connection.py", line 668, in make_request
host = self.calling_format.build_host(self.server_name(), bucket)
^^^^^^^^^^^^^^^^^^
File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/vendored/boto/boto/connection.py", line 661, in server_name
signature_host = '%s:%d' % (self.host, port)
~~~~~~~~^~~~~~~~~~~~~~~~~~~
TypeError: %d format: a real number is required, not str
```
The issue originates from the `gs_port` setting in the boto configuration. In `gslib/boto_translation.py`, the `_AddCustomEndpointToKey` function retrieves the port using `config.get`, which returns a string. This leads to a **TypeError** when an integer is expected.
https://github.com/GoogleCloudPlatform/gsutil/blob/141e2a098f56e3ba4bcebb1a5cf89947efece4a4/gslib/boto_translation.py#L131C1-L141C72
```
def _AddCustomEndpointToKey(key):
"""Update Boto Key object with user config's custom endpoint."""
user_setting_to_key_attribute = {
'gs_host': 'host',
'gs_port': 'port',
'gs_host_header': 'host_header',
}
for user_setting, key_attribute in user_setting_to_key_attribute.items():
user_setting_value = config.get('Credentials', user_setting, None)
if user_setting_value is not None:
setattr(key.bucket.connection, key_attribute, user_setting_value)
```
I believe replacing `config.get` with the `config.getint` function will fix the issue (for `gs_port` only!)
For example, `config.getint` is used for the same `gs_port` here: https://github.com/GoogleCloudPlatform/gsutil/blob/141e2a098f56e3ba4bcebb1a5cf89947efece4a4/gslib/commands/perfdiag.py#L521
If you think this is the right way to fix the problem, I can make a PR
Contributor guide
Assessment
This issue has not been assessed yet.