gcp_compute_instance throws Module error when trying to deconstruct "service_account_contents"

_From @rangapv on Jul 08, 2020 09:13_

##### SUMMARY

When trying to create a GCP instance in Ansible using the module "service_account_contents:" the playbook fails with the . For "service_account_contents" , i tried many different options
once as a variable(string myvar22 - defined in global_vars directory) with JSON ;key:value pairs ::::::: the error is the same as Module error , service_account_contents: "{{ myvar22 | string }}"
then tried with service_account_contents: "{{ JSON file | string }}" ::::::: the error is the same as Module error
then again tried with service_account_contents: "{{ path to json file with key:values }}" ::::::: the error is the same as Module error
then again tried with lookup, service_account_contents: "lookup('file','path to json file with key:values')" ::::::: the error is the same as Module error
ERROR MESSAGE:(PARTIAL LIST)
Module error...

"msg": "MODULE FAILURE\nSee stdout/stderr for the exact error",
"rc": 1

End of Error Message.

##### ISSUE TYPE
- Bug Report

##### COMPONENT NAME

Module name: "gcp_compute_instance"
the parameter "service_account_contents: is not deconstructing the supplied json file
##### ANSIBLE VERSION

```paste below
$ ansible --version
ansible 2.9.7
config file = /etc/ansible/ansible.cfg
configured module search path = ['/home/rangapv08/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /usr/local/lib/python3.5/dist-packages/ansible
executable location = /usr/local/bin/ansible
python version = 3.5.2 (default, Apr 16 2020, 17:47:17) [GCC 5.4.0 20160609]
```

##### CONFIGURATION

```paste below
DEFAULT_REMOTE_USER(/etc/ansible/ansible.cfg) = rangapv07
HOST_KEY_CHECKING(/etc/ansible/ansible.cfg) = False
```

##### OS / ENVIRONMENT

$ cat /etc/*-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=16.04
DISTRIB_CODENAME=xenial
DISTRIB_DESCRIPTION="Ubuntu 16.04.6 LTS"
NAME="Ubuntu"
VERSION="16.04.6 LTS (Xenial Xerus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.04.6 LTS"
VERSION_ID="16.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
VERSION_CODENAME=xenial
UBUNTU_CODENAME=xenial

##### STEPS TO REPRODUCE

ansible=playbook gcp.yaml -vvv

```yaml
- name: Create instance(s)
hosts: localhost
connection: local
gather_facts: no

tasks:

- debug:
msg: The Value os myvar22 is {{ myvar22 | string }}
- name: Launch instances
gcp_compute_instance:
name: kubenode32
auth_kind: serviceaccount
machine_type: "{{ machine_type }}"
disks:
- auto_delete: 'true'
boot: 'true'
initialize_params:
disk_size_gb: 10
source_image: "{{ image }}"
service_account_file: "{{ credentials_file }}"
service_account_contents: "{{ myvar22 }}"
project: "{{ project_id }}"
state: present
zone: "{{ zone }}"
scopes:
- storage-full
- cloud-platform
register: gce
```

##### EXPECTED RESULTS

The Playbook should run to completion with no errors, and a new GCP instance needs to be created

##### ACTUAL RESULTS

```paste below
$ ansible-playbook ./gcp.yaml -vvvv
ansible-playbook 2.9.7
config file = /etc/ansible/ansible.cfg
configured module search path = ['/home/rangapv08/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /usr/local/lib/python3.5/dist-packages/ansible
executable location = /usr/local/bin/ansible-playbook
python version = 3.5.2 (default, Apr 16 2020, 17:47:17) [GCC 5.4.0 20160609]
Using /etc/ansible/ansible.cfg as config file
setting up inventory plugins
host_list declined parsing /etc/ansible/hosts as it did not pass its verify_file() method
script declined parsing /etc/ansible/hosts as it did not pass its verify_file() method
auto declined parsing /etc/ansible/hosts as it did not pass its verify_file() method
Parsed /etc/ansible/hosts inventory source with ini plugin
[WARNING]: Found both group and host with same name: l1
Loading callback plugin default of type stdout, v2.0 from /usr/local/lib/python3.5/dist-packages/ansible/plugins/callback/default.py

PLAYBOOK: gcp.yaml *************************************************************************************************
Positional arguments: ./gcp.yaml
remote_user: rangapv76
inventory: ('/etc/ansible/hosts',)
become_method: sudo
tags: ('all',)
forks: 5
verbosity: 4
connection: smart
timeout: 10
1 plays in ./gcp.yaml
PLAY [Create instance(s)] ******************************************************************************************
META: ran handlers

TASK [debug] *******************************************************************************************************
task path: /home/rangapv08/myansible/gcp.yaml:8
ok: [localhost] => {
"msg": "The Value os myvar22 is { \"private_key_id\": \"a92e78b18244fff535879d8cb7dcf4b65bb2385e\" }"
}

TASK [Launch instances] ********************************************************************************************
task path: /home/rangapv08/myansible/gcp.yaml:10
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: rangapv08
<127.0.0.1> EXEC /bin/sh -c 'echo ~rangapv08 && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/rangapv08/.ansible/tmp `"&& mkdir /home/rangapv08/.ansible/tmp/ansible-tmp-1594197593.8106487-11724-166051136371677 && echo ansible-tmp-1594197593.8106487-11724-166051136371677="` echo /home/rangapv08/.ansible/tmp/ansible-tmp-1594197593.8106487-11724-166051136371677 `" ) && sleep 0'
Using module file /usr/local/lib/python3.5/dist-packages/ansible/modules/cloud/google/gcp_compute_instance.py
<127.0.0.1> PUT /home/rangapv08/.ansible/tmp/ansible-local-11716jigud28e/tmpey0lja_y TO /home/rangapv08/.ansible/tmp/ansible-tmp-1594197593.8106487-11724-166051136371677/AnsiballZ_gcp_compute_instance.py
<127.0.0.1> EXEC /bin/sh -c 'chmod u+x /home/rangapv08/.ansible/tmp/ansible-tmp-1594197593.8106487-11724-166051136371677/ /home/rangapv08/.ansible/tmp/ansible-tmp-1594197593.8106487-11724-166051136371677/AnsiballZ_gcp_compute_instance.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '/usr/bin/python3 /home/rangapv08/.ansible/tmp/ansible-tmp-1594197593.8106487-11724-166051136371677/AnsiballZ_gcp_compute_instance.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c 'rm -f -r /home/rangapv08/.ansible/tmp/ansible-tmp-1594197593.8106487-11724-166051136371677/ > /dev/null 2>&1 && sleep 0'
The full traceback is:
Traceback (most recent call last):
File "/home/rangapv08/.ansible/tmp/ansible-tmp-1594197593.8106487-11724-166051136371677/AnsiballZ_gcp_compute_instance.py", line 102, in
_ansiballz_main()
File "/home/rangapv08/.ansible/tmp/ansible-tmp-1594197593.8106487-11724-166051136371677/AnsiballZ_gcp_compute_instance.py", line 94, in _ansiballz_main
invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
File "/home/rangapv08/.ansible/tmp/ansible-tmp-1594197593.8106487-11724-166051136371677/AnsiballZ_gcp_compute_instance.py", line 40, in invoke_module
runpy.run_module(mod_name='ansible.modules.cloud.google.gcp_compute_instance', init_globals=None, run_name='__main__', alter_sys=True)
File "/usr/lib/python3.5/runpy.py", line 196, in run_module
return _run_module_code(code, init_globals, run_name, mod_spec)
File "/usr/lib/python3.5/runpy.py", line 96, in _run_module_code
mod_name, mod_spec, pkg_name, script_name)
File "/usr/lib/python3.5/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/tmp/ansible_gcp_compute_instance_payload_057hypv2/ansible_gcp_compute_instance_payload.zip/ansible/modules/cloud/google/gcp_compute_instance.py", line 1739, in
File "/tmp/ansible_gcp_compute_instance_payload_057hypv2/ansible_gcp_compute_instance_payload.zip/ansible/modules/cloud/google/gcp_compute_instance.py", line 1056, in main
File "/tmp/ansible_gcp_compute_instance_payload_057hypv2/ansible_gcp_compute_instance_payload.zip/ansible/modules/cloud/google/gcp_compute_instance.py", line 1159, in fetch_resource
File "/tmp/ansible_gcp_compute_instance_payload_057hypv2/ansible_gcp_compute_instance_payload.zip/ansible/module_utils/gcp_utils.py", line 85, in get
File "/tmp/ansible_gcp_compute_instance_payload_057hypv2/ansible_gcp_compute_instance_payload.zip/ansible/module_utils/gcp_utils.py", line 150, in full_get
File "/tmp/ansible_gcp_compute_instance_payload_057hypv2/ansible_gcp_compute_instance_payload.zip/ansible/module_utils/gcp_utils.py", line 195, in session
File "/tmp/ansible_gcp_compute_instance_payload_057hypv2/ansible_gcp_compute_instance_payload.zip/ansible/module_utils/gcp_utils.py", line 222, in _credentials
File "/home/rangapv08/.local/lib/python3.5/site-packages/google/oauth2/service_account.py", line 226, in from_service_account_file
filename, require=["client_email", "token_uri"]
File "/home/rangapv08/.local/lib/python3.5/site-packages/google/auth/_service_account_info.py", line 74, in from_filename
return data, from_dict(data, require=require)
File "/home/rangapv08/.local/lib/python3.5/site-packages/google/auth/_service_account_info.py", line 55, in from_dict
signer = crypt.RSASigner.from_service_account_info(data)
File "/home/rangapv08/.local/lib/python3.5/site-packages/google/auth/crypt/base.py", line 114, in from_service_account_info
info[_JSON_FILE_PRIVATE_KEY], info.get(_JSON_FILE_PRIVATE_KEY_ID)
File "/home/rangapv08/.local/lib/python3.5/site-packages/google/auth/crypt/_python_rsa.py", line 171, in from_string
raise ValueError("No key could be detected.")
ValueError: No key could be detected.
fatal: [localhost]: FAILED! => {
"changed": false,
"module_stderr": "Traceback (most recent call last):\n File \"/home/rangapv08/.ansible/tmp/ansible-tmp-1594197593.8106487-11724-166051136371677/AnsiballZ_gcp_compute_instance.py\", line 102, in \n _ansiballz_main()\n File \"/home/rangapv08/.ansible/tmp/ansible-tmp-1594197593.8106487-11724-166051136371677/AnsiballZ_gcp_compute_instance.py\", line 94, in _ansiballz_main\n invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n File \"/home/rangapv08/.ansible/tmp/ansible-tmp-1594197593.8106487-11724-166051136371677/AnsiballZ_gcp_compute_instance.py\", line 40, in invoke_module\n runpy.run_module(mod_name='ansible.modules.cloud.google.gcp_compute_instance', init_globals=None, run_name='__main__', alter_sys=True)\n File \"/usr/lib/python3.5/runpy.py\", line 196, in run_module\n return _run_module_code(code, init_globals, run_name, mod_spec)\n File \"/usr/lib/python3.5/runpy.py\", line 96, in _run_module_code\n mod_name, mod_spec, pkg_name, script_name)\n File \"/usr/lib/python3.5/runpy.py\", line 85, in _run_code\n exec(code, run_globals)\n File \"/tmp/ansible_gcp_compute_instance_payload_057hypv2/ansible_gcp_compute_instance_payload.zip/ansible/modules/cloud/google/gcp_compute_instance.py\", line 1739, in \n File \"/tmp/ansible_gcp_compute_instance_payload_057hypv2/ansible_gcp_compute_instance_payload.zip/ansible/modules/cloud/google/gcp_compute_instance.py\", line 1056, in main\n File \"/tmp/ansible_gcp_compute_instance_payload_057hypv2/ansible_gcp_compute_instance_payload.zip/ansible/modules/cloud/google/gcp_compute_instance.py\", line 1159, in fetch_resource\n File \"/tmp/ansible_gcp_compute_instance_payload_057hypv2/ansible_gcp_compute_instance_payload.zip/ansible/module_utils/gcp_utils.py\", line 85, in get\n File \"/tmp/ansible_gcp_compute_instance_payload_057hypv2/ansible_gcp_compute_instance_payload.zip/ansible/module_utils/gcp_utils.py\", line 150, in full_get\n File \"/tmp/ansible_gcp_compute_instance_payload_057hypv2/ansible_gcp_compute_instance_payload.zip/ansible/module_utils/gcp_utils.py\", line 195, in session\n File \"/tmp/ansible_gcp_compute_instance_payload_057hypv2/ansible_gcp_compute_instance_payload.zip/ansible/module_utils/gcp_utils.py\", line 222, in _credentials\n File \"/home/rangapv08/.local/lib/python3.5/site-packages/google/oauth2/service_account.py\", line 226, in from_service_account_file\n filename, require=[\"client_email\", \"token_uri\"]\n File \"/home/rangapv08/.local/lib/python3.5/site-packages/google/auth/_service_account_info.py\", line 74, in from_filename\n return data, from_dict(data, require=require)\n File \"/home/rangapv08/.local/lib/python3.5/site-packages/google/auth/_service_account_info.py\", line 55, in from_dict\n signer = crypt.RSASigner.from_service_account_info(data)\n File \"/home/rangapv08/.local/lib/python3.5/site-packages/google/auth/crypt/base.py\", line 114, in from_service_account_info\n info[_JSON_FILE_PRIVATE_KEY], info.get(_JSON_FILE_PRIVATE_KEY_ID)\n File \"/home/rangapv08/.local/lib/python3.5/site-packages/google/auth/crypt/_python_rsa.py\", line 171, in from_string\n raise ValueError(\"No key could be detected.\")\nValueError: No key could be detected.\n",
"module_stdout": "",
"msg": "MODULE FAILURE\nSee stdout/stderr for the exact error",
"rc": 1
}
PLAY RECAP *********************************************************************************************************
localhost : ok=1 changed=0 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0
```

_Copied from original issue: ansible/ansible#70519_

Start by reproducing the playbook in gcp.yaml, then read ansible/modules/cloud/google/gcp_compute_instance.py and ansible/module_utils/gcp_utils.py around credential handling. Verify how service_account_contents is passed to the Google authentication library and confirm that a valid JSON credential supplied through this parameter creates the GCP instance without the key-detection error.

Written by the indexing model from the issue text.