saltstack / saltstack/salt

[BUG] `salt-cloud` fails for `gce.avail_sizes`

Open
#57,186 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug Core Salt-Cloud severity-medium
Dominant language
Python
Stars
15.7k
Forks
5.6k
Avg merge
2d 44m
Merged PRs (30d)
80

Description

Description
Querying the available sizes in GCE using salt-cloud --list-sizes gce-test fails using SaltStack 3000.2.

Setup
/etc/salt/cloud.providers.d/google-compute-engine.conf:

gce-test:
    project: test-project
    service_account_email_address: local-dev-test-sa@test-project.iam.gserviceaccount.com
    service_account_private_key: /home/eliasp/.secrets/gce-44fda5a581cff82dba2b3b6b1b95d881c.pem
    driver: gce

Steps to Reproduce the behavior

$ salt-cloud -l debug --list-sizes gce-test
[DEBUG   ] Using cached minion ID from /etc/salt/minion_id: cloud-test
[DEBUG   ] Using cached minion ID from /etc/salt/minion_id: cloud-test
[DEBUG   ] Missing configuration file: /etc/salt/cloud.providers
[DEBUG   ] Missing configuration file: /etc/salt/cloud.providers
[DEBUG   ] Including configuration from '/etc/salt/cloud.providers.d/google-compute-engine.conf'
[DEBUG   ] Including configuration from '/etc/salt/cloud.providers.d/google-compute-engine.conf'
[DEBUG   ] Reading configuration from /etc/salt/cloud.providers.d/google-compute-engine.conf
[DEBUG   ] Reading configuration from /etc/salt/cloud.providers.d/google-compute-engine.conf
[DEBUG   ] Missing configuration file: /etc/salt/cloud.profiles
[DEBUG   ] Missing configuration file: /etc/salt/cloud.profiles
[DEBUG   ] Configuration file path: /etc/salt/master
[DEBUG   ] Configuration file path: /etc/salt/master
[WARNING ] Insecure logging configuration detected! Sensitive data may be logged.
[WARNING ] Insecure logging configuration detected! Sensitive data may be logged.
[INFO    ] salt-cloud starting
[INFO    ] salt-cloud starting
[DEBUG   ] Marking 'base64_encode' as a jinja filter
[...] removed loading of Jinja filters
[DEBUG   ] Marking 'symmetric_difference' as a jinja filter
[DEBUG   ] Marking 'symmetric_difference' as a jinja filter
[DEBUG   ] Could not LazyLoad parallels.avail_sizes: 'parallels' __virtual__ returned False
[DEBUG   ] Could not LazyLoad parallels.avail_sizes: 'parallels' __virtual__ returned False
[DEBUG   ] LazyLoaded parallels.avail_locations
[DEBUG   ] LazyLoaded parallels.avail_locations
[DEBUG   ] LazyLoaded proxmox.avail_sizes
[DEBUG   ] LazyLoaded proxmox.avail_sizes
[DEBUG   ] Starting new HTTPS connection (1): www.googleapis.com
[DEBUG   ] Starting new HTTPS connection (1): www.googleapis.com
[DEBUG   ] https://www.googleapis.com:443 "GET /compute/v1/projects/test-project/zones HTTP/1.1" 200 None
[DEBUG   ] https://www.googleapis.com:443 "GET /compute/v1/projects/test-project/zones HTTP/1.1" 200 None
[DEBUG   ] https://www.googleapis.com:443 "GET /compute/v1/projects/test-project/regions HTTP/1.1" 200 None
[DEBUG   ] https://www.googleapis.com:443 "GET /compute/v1/projects/test-project/regions HTTP/1.1" 200 None
[DEBUG   ] https://www.googleapis.com:443 "GET /compute/v1/projects/test-project/aggregated/machineTypes HTTP/1.1" 200 None
[DEBUG   ] https://www.googleapis.com:443 "GET /compute/v1/projects/test-project/aggregated/machineTypes HTTP/1.1" 200 None
[DEBUG   ] https://www.googleapis.com:443 "GET /compute/v1/projects/test-project/zones/us-central1-d HTTP/1.1" 404 None
[...] removed 76 recurrences
[DEBUG   ] https://www.googleapis.com:443 "GET /compute/v1/projects/test-project/zones/us-central1-d HTTP/1.1" 404 None
[DEBUG   ] https://www.googleapis.com:443 "GET /compute/v1/projects/test-project/zones/us-central2-b HTTP/1.1" 404 None
[...] removed 42 recurrences
[DEBUG   ] https://www.googleapis.com:443 "GET /compute/v1/projects/test-project/zones/us-central2-b HTTP/1.1" 404 None
[DEBUG   ] https://www.googleapis.com:443 "GET /compute/v1/projects/test-project/zones/us-central2-a HTTP/1.1" 404 None
[...] removed 46 recurrences
[DEBUG   ] https://www.googleapis.com:443 "GET /compute/v1/projects/test-project/zones/us-central2-a HTTP/1.1" 404 None
[DEBUG   ] https://www.googleapis.com:443 "GET /compute/v1/projects/test-project/zones/us-central2-c HTTP/1.1" 404 None
[...] removed 42 recurrences
[DEBUG   ] https://www.googleapis.com:443 "GET /compute/v1/projects/test-project/zones/us-central2-c HTTP/1.1" 404 None
[DEBUG   ] https://www.googleapis.com:443 "GET /compute/v1/projects/test-project/zones/us-central2-d HTTP/1.1" 404 None
[...] removed 42 recurrences
[DEBUG   ] https://www.googleapis.com:443 "GET /compute/v1/projects/test-project/zones/us-central2-d HTTP/1.1" 404 None
[ERROR   ] Failed to get the output of 'gce.avail_sizes()': 'NoneType' object has no attribute '__dict__'
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/salt/cloud/__init__.py", line 896, in size_list
    data[alias][driver] = self.clouds[fun]()
  File "/usr/lib/python3/dist-packages/salt/cloud/clouds/gce.py", line 324, in avail_sizes
    size.extra['zone'].update(zone.__dict__)
AttributeError: 'NoneType' object has no attribute '__dict__'
[ERROR   ] Failed to get the output of 'gce.avail_sizes()': 'NoneType' object has no attribute '__dict__'
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/salt/cloud/__init__.py", line 896, in size_list
    data[alias][driver] = self.clouds[fun]()
  File "/usr/lib/python3/dist-packages/salt/cloud/clouds/gce.py", line 324, in avail_sizes
    size.extra['zone'].update(zone.__dict__)
AttributeError: 'NoneType' object has no attribute '__dict__'
[DEBUG   ] LazyLoaded nested.output
[DEBUG   ] LazyLoaded nested.output
gce-test:
    ----------

Expected behavior
I expect salt-cloud --list-sizes gce-test to return a list of available VM sizes + no Traceback/AttributeError.

Versions Report

salt --versions-report (Provided by running salt --versions-report. Please also mention any differences in master/minion versions.)
Salt Version:
            Salt: 3000.2

Dependency Versions:
 Apache Libcloud: 2.2.1
            cffi: Not Installed
        cherrypy: Not Installed
        dateutil: 2.6.1
       docker-py: Not Installed
           gitdb: Not Installed
       gitpython: Not Installed
          Jinja2: 2.10
         libgit2: Not Installed
        M2Crypto: Not Installed
            Mako: 1.0.7
    msgpack-pure: Not Installed
  msgpack-python: 0.6.2
    mysql-python: Not Installed
       pycparser: Not Installed
        pycrypto: 2.6.1
    pycryptodome: Not Installed
          pygit2: Not Installed
          Python: 3.6.9 (default, Apr 18 2020, 01:56:04)
    python-gnupg: 0.4.1
          PyYAML: 3.12
           PyZMQ: 16.0.2
           smmap: Not Installed
         timelib: Not Installed
         Tornado: 4.5.3
             ZMQ: 4.2.5

System Versions:
            dist: Ubuntu 18.04 bionic
          locale: UTF-8
         machine: x86_64
         release: 4.15.0-76-generic
          system: Linux
         version: Ubuntu 18.04 bionic

Additional context
Possibly related: #56862
salt-cloud --list-locations gce-test works just fine, so it's not the provider configuration which is broken.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in salt/cloud/clouds/gce.py at avail_sizes, especially the failing zone update on line 324, and trace how salt/cloud/init.py invokes it through size_list. Reproduce with salt-cloud --list-sizes gce-test using the reported provider configuration, then verify that available VM sizes are returned without the NoneType AttributeError.

Written by the indexing model from the issue text.

Assessment

Tech stack
google-cloud, python
Domain
cloud
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.