GoogleCloudPlatform / GoogleCloudPlatform/gsutil

gsutil config -e throws an exception on python3.5

Open
#1,078 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
918
Forks
335
PR merge metrics
No merged PRs in 30d

Description

Recently we updated from gcloud 300 to 304, and started seeing the following exception when trying to use `gsutil config -e`:

```
$ gsutil config -e
This command will create a boto config file at /home/user/.boto
containing your credentials, based on your responses to the following
questions.
What is the full path to your private key file? key.json
Traceback (most recent call last):
File "/usr/lib/google-cloud-sdk/platform/gsutil/gsutil", line 21, in
gsutil.RunMain()
File "/usr/lib/google-cloud-sdk/platform/gsutil/gsutil.py", line 123, in RunMain
sys.exit(gslib.__main__.main())
File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/__main__.py", line 438, in main
user_project=user_project)
File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/__main__.py", line 767, in _RunNamedCommandAndHandleExceptions
_HandleUnknownFailure(e)
File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/__main__.py", line 633, in _RunNamedCommandAndHandleExceptions
user_project=user_project)
File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/command_runner.py", line 411, in RunNamedCommand
return_code = command_inst.RunCommand()
File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/commands/config.py", line 1211, in RunCommand
configure_auth=configure_auth)
File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/commands/config.py", line 844, in _WriteBotoConfigFile
json.loads(key_file_fp.read())
File "/usr/lib/python3.5/json/__init__.py", line 312, in loads
s.__class__.__name__))
TypeError: the JSON object must be str, not 'bytes'
```

I believe `json.loads(key_file_fp.read())` should be `json.loads(key_file_fp.read().decode('utf-8'))` here.

I am aware that as of gcloud 301, python3 is preferred for gsutil, and there is a `CLOUDSDK_GSUTIL_PYTHON` env var to control which version of python is used for gsutil. I think fixing this minor issue would be helpful to avoid this exception on new and upgraded installs using the defaults, though, especially in light of the upcoming deprecation of python2 support.

Contributor guide

Open the contributing guide

Research direction

Start in gslib/commands/config.py at _WriteBotoConfigFile, using the gsutil config -e traceback and the Python 3.5 json.loads failure as the reproduction. Verify the behavior with a service-account key file and confirm that gsutil config -e completes successfully under Python 3.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.