saltstack / saltstack/salt

[BUG] RuntimeError: dictionary changed size during iteration in apply_sdb

Open
#58,819 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug Salt-API severity-medium
Dominant language
Python
Stars
15.7k
Forks
5.6k
Avg merge
2d 44m
Merged PRs (30d)
80

Description

Description
I receive the following error when sending reactor events to run salt-cloud commands. I believe it occurs when I fire an event to create a VM, and then before the VM is fnished being created, fire another event to create another VM.

[DEBUG   ] dictionary changed size during iteration
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/salt/utils/process.py", line 491, in _thread_target
    func(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/salt/client/mixins.py", line 306, in low
    self.mminion  # pylint: disable=W0104
  File "/usr/lib/python3.6/site-packages/salt/client/mixins.py", line 265, in mminion
    self.opts, states=False, rend=False
  File "/usr/lib/python3.6/site-packages/salt/minion.py", line 968, in __init__
    opts["conf_file"], ignore_config_errors=ignore_config_errors, role="master"
  File "/usr/lib/python3.6/site-packages/salt/config/__init__.py", line 2190, in minion_config
    apply_sdb(opts)
  File "/usr/lib/python3.6/site-packages/salt/config/__init__.py", line 2346, in apply_sdb
    for key, value in six.iteritems(sdb_opts):
RuntimeError: dictionary changed size during iteration

It then seems to interrupt creation of both VMs. I'm using the salt-cloud vmware provider

Setup

  1. Set up an YAML SDB using GPG which contains the credentials for your salt-cloud provider

  2. Set up a cloud provider in /etc/salt/cloud.providers.d:

YOR:
  driver: vmware
  user: sdb://secret/vsphere:username
  password: sdb://secret/vsphere:password
  url: 'sphere.host.example.com'
  protocol: 'https'
  port: 443

Steps to Reproduce the behavior
(Include debug logs if possible and relevant)

  1. Call webhook to Salt-API to create events to create two VMs, firing the 2nd one before the first one is created.
[DEBUG   ] Gathering reactors for tag salt/netapi/hook/netbox/virtualization/virtual-machines
[DEBUG   ] Compiling reactions for tag salt/netapi/hook/netbox/virtualization/virtual-machines
[DEBUG   ] Re-using gitfs object for process 192406
[DEBUG   ] Re-using gitfs object for process 192406
[DEBUG   ] Re-using gitfs object for process 192406
[DEBUG   ] In saltenv 'base', looking at rel_path 'reactor/vmware_create.sls' to resolve 'salt://reactor/vmware_create.sls'
[DEBUG   ] In saltenv 'base', ** considering ** path '/var/cache/salt/master/files/base/reactor/vmware_create.sls' to resolve 'salt://reactor/vmware_create.sls'
[DEBUG   ] Fetching file from saltenv 'base', ** attempting ** 'salt://reactor/vmware_create.sls'
[DEBUG   ] No dest file found
[DEBUG   ] Re-using gitfs object for process 192406
[DEBUG   ] Re-using gitfs object for process 192406
[DEBUG   ] Re-using gitfs object for process 192406
[DEBUG   ] Re-using gitfs object for process 192406
[INFO    ] Fetching file from saltenv 'base', ** done ** 'reactor/vmware_create.sls'
[DEBUG   ] compile template: /var/cache/salt/master/files/base/reactor/vmware_create.sls
[DEBUG   ] LazyLoaded yaml.get
[DEBUG   ] LazyLoaded jinja.render
[DEBUG   ] LazyLoaded yaml.render
[DEBUG   ] LazyLoaded gpg.render
[DEBUG   ] LazyLoaded config.get
[DEBUG   ] Reading GPG keys from: /etc/salt/gpgkeys
[DEBUG   ] LazyLoaded yaml.get
[DEBUG   ] LazyLoaded jinja.render
[DEBUG   ] LazyLoaded yaml.render
[DEBUG   ] LazyLoaded gpg.render
[DEBUG   ] LazyLoaded config.get
[DEBUG   ] Reading GPG keys from: /etc/salt/gpgkeys
{'new_compute_instance': {'runner.cloud.create': [{'args': [{'provider': 'MY'},
                                                            {'instances': 'uycustmr-mon01'},
                                                            {'cluster': 'MY_Cloud'},
                                                            {'datacenter': 'MY'},
                                                            {'datastore': 'syortin01-T1'},
                                                            {'folder': 'CL '
                                                                       'Client '
                                                                       'Monitoring'},
                                                            {'hardware_version': 13},
                                                            {'image': 'oracleLinux7_64Guest'},
                                                            {'num_cpus': 3},
                                                            {'memory': '6144MB'},
                                                            {'devices': {'cd': {'CD/DVD drive 1': {'controller': 'IDE '
                                                                                                                 '0'}},
                                                                         'disk': {'Hard disk 1': {'controller': 'SCSI '
                                                                                                                'controller '
                                                                                                                '0',
                                                                                                  'size': 90,
                                                                                                  'thin_provision': True}},
                                                                         'ide': {'IDE 0': {}},
                                                                         'network': {'Network adapter 1': {'adapter_type': 'vmxnet3',
                                                                                                           'gateway': ['192.0.2.129'],
                                                                                                           'ip': '192.0.2.174',
                                                                                                           'name': 'pgPROVID-THE120%2fMon%2fMY_DMZ_Isolated',
                                                                                                           'subnet_mask': '255.255.255.128',
                                                                                                           'switch_type': 'distributed'},
                                                                                     'Network adapter 2': {'adapter_type': 'vmxnet3',
                                                                                                           'ip': '192.0.2.116',
                                                                                                           'name': 'pgCUSTMR-THE10%2fMon%2fMY',
                                                                                                           'subnet_mask': '255.255.255.240',
                                                                                                           'switch_type': 'distributed'}},
                                                                         'scsi': {'SCSI controller 0': {'type': 'paravirtual'}}}},
                                                            {'domain': 'example.com'},
                                                            {'dns_servers': ['198.51.100.29',
                                                                             '203.0.113.138']},
                                                            {'power_on': False},
                                                            {'annotation': 'Created '
                                                                           'by '
                                                                           'Salt-Cloud'},
                                                            {'deploy': False},
                                                            {'customization': True},
                                                            {'ssh_username': 'root'},
                                                            {'password': 'root'},
                                                            {'minion': {'master': '192.0.2.68'}}]}]}}
[PROFILE ] Time (in seconds) to render '/var/cache/salt/master/files/base/reactor/vmware_create.sls' using 'py' renderer: 1.0254533290863037
[DEBUG   ] ThreadPool executing func: <bound method SyncClientMixin.low of <salt.runner.RunnerClient object at 0x7f10101359e8>> with args=('cloud.create', {'name': 'new_compute_instance', '__sls__': '/var/cache/salt/master/files/base/reactor/vmware_create.sls', '__id__': 'new_compute_instance', 'order': 1, '__user__': 'Reactor', '__state__': 'runner', 'arg': (), 'kwarg': {'provider': 'MY', 'instances': 'uycustmr-mon01', 'cluster': 'MY_Cloud', 'datacenter': 'MY', 'datastore': 'syortin01-T1', 'folder': 'CL Client Monitoring', 
'hardware_version': 13, 'image': 'oracleLinux7_64Guest', 'num_cpus': 3, 'memory': '6144MB', 'devices': {'ide': {'IDE 0': {}}, 'cd': {'CD/DVD drive 1': {'controller': 'IDE 0'}}, 'scsi': {'SCSI controller 0': {'type': 'paravirtual'}}, 'disk': {'Hard disk 1': {'controller': 'SCSI controller 0', 'size': 90, 'thin_provision': True}}, 'network': {'Network adapter 1': {'adapter_type': 'vmxnet3', 'name': 'pgPROVID-THE120%2fMon%2fMY_DMZ_Isolated', 'switch_type': 'distributed', 'ip': '192.0.2.174', 'subnet_mask': '255.255.255.128', 'gateway': ['192.0.2.129']}, 'Network adapter 2': {'adapter_type': 'vmxnet3', 'name': 'pgCUSTMR-THE10%2fMon%2fMY', 'switch_type': 'distributed', 'ip': '192.0.2.116', 'subnet_mask': '255.255.255.240'}}}, 'domain': 'clinfra.internal', 'dns_servers': ['198.51.100.29', 
'203.0.113.138'], 'power_on': False, 'annotation': 'Created by Salt-Cloud', 'deploy': False, 'customization': True, 'ssh_username': 'root', 'password': 'root', 'minion': {'master': '192.0.2.68'}}}) kwargs={}
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] Gathering reactors for tag salt/netapi/hook/netbox/virtualization/virtual-machines
[DEBUG   ] Compiling reactions for tag salt/netapi/hook/netbox/virtualization/virtual-machines
[DEBUG   ] Re-using gitfs object for process 192406
[DEBUG   ] Including configuration from '/etc/salt/master.d/clmon.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/clmon.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/git_pillar.conf'
[DEBUG   ] Re-using gitfs object for process 192406
[DEBUG   ] Reading configuration from /etc/salt/master.d/git_pillar.conf
[DEBUG   ] Re-using gitfs object for process 192406
[DEBUG   ] Including configuration from '/etc/salt/master.d/gitfs.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/gitfs.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/netbox.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/netbox.conf
[DEBUG   ] In saltenv 'base', looking at rel_path 'reactor/vmware_create.sls' to resolve 'salt://reactor/vmware_create.sls'
[DEBUG   ] In saltenv 'base', ** considering ** path '/var/cache/salt/master/files/base/reactor/vmware_create.sls' to resolve 'salt://reactor/vmware_create.sls'
[DEBUG   ] Including configuration from '/etc/salt/master.d/reactor.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/reactor.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/saltgui.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/saltgui.conf
[DEBUG   ] Fetching file from saltenv 'base', ** attempting ** 'salt://reactor/vmware_create.sls'
[DEBUG   ] No dest file found
[DEBUG   ] Re-using gitfs object for process 192406
[DEBUG   ] Including configuration from '/etc/salt/master.d/secret.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/secret.conf
[DEBUG   ] Guessing ID. The id can be explicitly set in /etc/salt/minion
[DEBUG   ] Re-using gitfs object for process 192406
[DEBUG   ] Re-using gitfs object for process 192406
[DEBUG   ] Re-using gitfs object for process 192406
[INFO    ] Fetching file from saltenv 'base', ** done ** 'reactor/vmware_create.sls'
[DEBUG   ] compile template: /var/cache/salt/master/files/base/reactor/vmware_create.sls
[DEBUG   ] LazyLoaded yaml.get
[DEBUG   ] LazyLoaded jinja.render
[DEBUG   ] LazyLoaded yaml.render
[DEBUG   ] Found minion id from generate_minion_id(): master.example.com
[DEBUG   ] LazyLoaded gpg.render
[DEBUG   ] LazyLoaded config.get
[DEBUG   ] Reading GPG keys from: /etc/salt/gpgkeys
<truncated>
[DEBUG   ] LazyLoaded yaml.get
[DEBUG   ] LazyLoaded jinja.render
[DEBUG   ] LazyLoaded yaml.render
[DEBUG   ] LazyLoaded gpg.render
[DEBUG   ] LazyLoaded config.get
[DEBUG   ] Reading GPG keys from: /etc/salt/gpgkeys
{'new_compute_instance': {'runner.cloud.create': [{'args': [{'provider': 'MY'},
                                                            {'instances': 'uycustmr-mon02'},
                                                            {'cluster': 'MY_Cloud'},
                                                            {'datacenter': 'MY'},
                                                            {'datastore': 'syortin01-T1'},
                                                            {'folder': 'CL '
                                                                       'Client '
                                                                       'Monitoring'},
                                                            {'hardware_version': 13},
                                                            {'image': 'oracleLinux7_64Guest'},
                                                            {'num_cpus': 3},
                                                            {'memory': '6144MB'},
                                                            {'devices': {'cd': {'CD/DVD drive 1': {'controller': 'IDE '
                                                                                                                 '0'}},
                                                                         'disk': {'Hard disk 1': {'controller': 'SCSI '
                                                                                                                'controller '
                                                                                                                '0',
                                                                                                  'size': 90,
                                                                                                  'thin_provision': True}},
                                                                         'ide': {'IDE 0': {}},
                                                                         'network': {'Network adapter 1': {'adapter_type': 'vmxnet3',
                                                                                                           'gateway': ['192.0.2.129'],
                                                                                                           'ip': '192.0.2.176',
                                                                                                           'name': 'pgPROVID-THE120%2fMon%2fMY_DMZ_Isolated',
                                                                                                           'subnet_mask': '255.255.255.128',
                                                                                                           'switch_type': 'distributed'},
                                                                                     'Network adapter 2': {'adapter_type': 'vmxnet3',
                                                                                                           'ip': '192.0.2.116',
                                                                                                           'name': 'pgCUSTMR-THE11%2fMon%2fMY',
                                                                                                           'subnet_mask': '255.255.255.240',
                                                                                                           'switch_type': 'distributed'}},
                                                                         'scsi': {'SCSI controller 0': {'type': 'paravirtual'}}}},
                                                            {'domain': 'example.com'},
                                                            {'dns_servers': ['198.51.100.29',
                                                                             '203.0.113.138']},
                                                            {'power_on': False},
                                                            {'annotation': 'Created '
                                                                           'by '
                                                                           'Salt-Cloud'},
                                                            {'deploy': False},
                                                            {'customization': True},
                                                            {'ssh_username': 'root'},
                                                            {'password': 'root'},
                                                            {'minion': {'master': '192.0.2.68'}}]}]}}
[PROFILE ] Time (in seconds) to render '/var/cache/salt/master/files/base/reactor/vmware_create.sls' using 'py' renderer: 1.093775987625122
[DEBUG   ] ThreadPool executing func: <bound method SyncClientMixin.low of <salt.runner.RunnerClient object at 0x7f10101359e8>> with args=('cloud.create', {'name': 'new_compute_instance', '__sls__': '/var/cache/salt/master/files/base/reactor/vmware_create.sls', '__id__': 'new_compute_instance', 'order': 1, '__user__': 'Reactor', '__state__': 'runner', 'arg': (), 'kwarg': {'provider': 'MY', 'instances': 'uycustmr-mon02', 'cluster': 'MY_Cloud', 'datacenter': 'MY', 'datastore': 'syortin01-T1', 'folder': 'CL Client Monitoring', 'hardware_version': 13, 'image': 'oracleLinux7_64Guest', 'num_cpus': 3, 'memory': '6144MB', 'devices': {'ide': {'IDE 0': {}}, 'cd': {'CD/DVD drive 1': {'controller': 'IDE 0'}}, 'scsi': {'SCSI controller 0': {'type': 'paravirtual'}}, 'disk': {'Hard disk 1': {'controller': 'SCSI controller 0', 'size': 90, 'thin_provision': True}}, 'network': {'Network adapter 1': {'adapter_type': 'vmxnet3', 'name': 'pgPROVID-THE120%2fMon%2fMY_DMZ_Isolated', 'switch_type': 'distributed', 'ip': '192.0.2.176', 'subnet_mask': '255.255.255.128', 'gateway': ['192.0.2.129']}, 'Network adapter 2': {'adapter_type': 'vmxnet3', 'name': 'pgCUSTMR-THE11%2fMon%2fMY', 'switch_type': 'distributed', 'ip': '192.0.2.116', 'subnet_mask': '255.255.255.240'}}}, 'domain': 'clinfra.internal', 'dns_servers': ['198.51.100.29', '203.0.113.138'], 'power_on': False, 'annotation': 'Created by Salt-Cloud', 'deploy': False, 'customization': True, 'ssh_username': 'root', 'password': 'root', 'minion': {'master': '192.0.2.68'}}}) kwargs={}
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] Including configuration from '/etc/salt/master.d/clmon.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/clmon.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/git_pillar.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/git_pillar.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/gitfs.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/gitfs.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/netbox.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/netbox.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/reactor.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/reactor.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/saltgui.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/saltgui.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/secret.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/secret.conf
[DEBUG   ] Guessing ID. The id can be explicitly set in /etc/salt/minion
[DEBUG   ] Found minion id from generate_minion_id(): master.example.com
[DEBUG   ] LazyLoaded yaml.get
[DEBUG   ] LazyLoaded jinja.render
[DEBUG   ] LazyLoaded yaml.render
[DEBUG   ] LazyLoaded gpg.render
[DEBUG   ] LazyLoaded config.get
[DEBUG   ] Reading GPG keys from: /etc/salt/gpgkeys
<truncated>
[DEBUG   ] LazyLoaded yaml.get
[DEBUG   ] LazyLoaded jinja.render
[DEBUG   ] LazyLoaded yaml.render
[DEBUG   ] LazyLoaded gpg.render
[DEBUG   ] LazyLoaded config.get
[DEBUG   ] Reading GPG keys from: /etc/salt/gpgkeys
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] Including configuration from '/etc/salt/master.d/clmon.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/clmon.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/git_pillar.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/git_pillar.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/gitfs.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/gitfs.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/netbox.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/netbox.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/reactor.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/reactor.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/saltgui.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/saltgui.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/secret.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/secret.conf
[DEBUG   ] Guessing ID. The id can be explicitly set in /etc/salt/minion
[DEBUG   ] Found minion id from generate_minion_id(): master.example.com
[DEBUG   ] LazyLoaded yaml.get
[DEBUG   ] LazyLoaded jinja.render
[DEBUG   ] LazyLoaded yaml.render
[DEBUG   ] LazyLoaded gpg.render
[DEBUG   ] LazyLoaded config.get
[DEBUG   ] Reading GPG keys from: /etc/salt/gpgkeys
<truncated>
[DEBUG   ] LazyLoaded yaml.get
[DEBUG   ] LazyLoaded jinja.render
[DEBUG   ] LazyLoaded yaml.render
[DEBUG   ] LazyLoaded gpg.render
[DEBUG   ] LazyLoaded config.get
[DEBUG   ] Reading GPG keys from: /etc/salt/gpgkeys
[DEBUG   ] Sending event: tag = salt/wheel/20201026224315510084/new; data = {'fun': 'wheel.config.values', 'jid': '20201026224315510084', 'tag': 'salt/wheel/20201026224315510084', 'user': 'ggiesen', '_stamp': '2020-10-26T22:43:15.510151'}
[DEBUG   ] MasterEvent PUB socket URI: /var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Sending event: tag = salt/wheel/20201026224315510705/new; data = {'fun': 'wheel.config.values', 'jid': '20201026224315510705', 'user': 'UNKNOWN', 'fun_args': [], '_stamp': '2020-10-26T22:43:15.513624'}
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] Including configuration from '/etc/salt/master.d/clmon.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/clmon.conf
[DEBUG   ] Gathering reactors for tag salt/wheel/20201026224315510084/new
[DEBUG   ] Including configuration from '/etc/salt/master.d/git_pillar.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/git_pillar.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/gitfs.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/gitfs.conf
[DEBUG   ] LazyLoaded yaml.get
[DEBUG   ] Including configuration from '/etc/salt/master.d/netbox.conf'
[DEBUG   ] Gathering reactors for tag salt/wheel/20201026224315510705/new
[DEBUG   ] Reading configuration from /etc/salt/master.d/netbox.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/reactor.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/reactor.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/saltgui.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/saltgui.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/secret.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/secret.conf
[DEBUG   ] LazyLoaded jinja.render
[DEBUG   ] LazyLoaded yaml.render
[DEBUG   ] LazyLoaded gpg.render
[DEBUG   ] LazyLoaded config.get
[DEBUG   ] Reading GPG keys from: /etc/salt/gpgkeys
<truncated>
[DEBUG   ] Reading GPG keys from: /etc/salt/gpgkeys
[DEBUG   ] LazyLoaded yaml.get
[DEBUG   ] LazyLoaded jinja.render
[DEBUG   ] LazyLoaded yaml.render
[DEBUG   ] LazyLoaded gpg.render
[DEBUG   ] LazyLoaded config.get
[DEBUG   ] Reading GPG keys from: /etc/salt/gpgkeys
[DEBUG   ] dictionary changed size during iteration
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/salt/utils/process.py", line 491, in _thread_target
    func(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/salt/client/mixins.py", line 306, in low
    self.mminion  # pylint: disable=W0104
  File "/usr/lib/python3.6/site-packages/salt/client/mixins.py", line 265, in mminion
    self.opts, states=False, rend=False
  File "/usr/lib/python3.6/site-packages/salt/minion.py", line 968, in __init__
    opts["conf_file"], ignore_config_errors=ignore_config_errors, role="master"
  File "/usr/lib/python3.6/site-packages/salt/config/__init__.py", line 2190, in minion_config
    apply_sdb(opts)
  File "/usr/lib/python3.6/site-packages/salt/config/__init__.py", line 2346, in apply_sdb
    for key, value in six.iteritems(sdb_opts):
RuntimeError: dictionary changed size during iteration
[DEBUG   ] LazyLoaded yaml.get
[DEBUG   ] LazyLoaded jinja.render
[DEBUG   ] LazyLoaded yaml.render
[DEBUG   ] LazyLoaded gpg.render
[DEBUG   ] LazyLoaded config.get
[DEBUG   ] Reading GPG keys from: /etc/salt/gpgkeys
<truncated>
[DEBUG   ] LazyLoaded yaml.get
[DEBUG   ] LazyLoaded jinja.render
[DEBUG   ] LazyLoaded yaml.render
[DEBUG   ] LazyLoaded gpg.render
[DEBUG   ] LazyLoaded config.get
[DEBUG   ] Reading GPG keys from: /etc/salt/gpgkeys
[DEBUG   ] dictionary changed size during iteration
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/salt/utils/process.py", line 491, in _thread_target
    func(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/salt/client/mixins.py", line 306, in low
    self.mminion  # pylint: disable=W0104
  File "/usr/lib/python3.6/site-packages/salt/client/mixins.py", line 265, in mminion
    self.opts, states=False, rend=False
  File "/usr/lib/python3.6/site-packages/salt/minion.py", line 968, in __init__
    opts["conf_file"], ignore_config_errors=ignore_config_errors, role="master"
  File "/usr/lib/python3.6/site-packages/salt/config/__init__.py", line 2190, in minion_config
    apply_sdb(opts)
  File "/usr/lib/python3.6/site-packages/salt/config/__init__.py", line 2346, in apply_sdb
    for key, value in six.iteritems(sdb_opts):
RuntimeError: dictionary changed size during iteration
[DEBUG   ] LazyLoaded yaml.get
[DEBUG   ] LazyLoaded jinja.render
[DEBUG   ] LazyLoaded yaml.render
[DEBUG   ] LazyLoaded gpg.render
[DEBUG   ] LazyLoaded config.get

Expected behavior
To VMs to be created

Versions Report

salt --versions-report (Provided by running salt --versions-report. Please also mention any differences in master/minion versions.) 
salt --versions-report
Salt Version:
           Salt: 3001.1

Dependency Versions:
           cffi: 1.14.0
       cherrypy: unknown
       dateutil: 2.6.1
      docker-py: Not Installed
          gitdb: Not Installed
      gitpython: Not Installed
         Jinja2: 2.11.2
        libgit2: 0.26.8
       M2Crypto: 0.35.2
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.6.2
   mysql-python: Not Installed
      pycparser: 2.20
       pycrypto: Not Installed
   pycryptodome: Not Installed
         pygit2: 0.26.4
         Python: 3.6.8 (default, Apr 16 2020, 01:36:27)
   python-gnupg: Not Installed
         PyYAML: 5.3.1
          PyZMQ: 19.0.0
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.5.3
            ZMQ: 4.3.3

System Versions:
           dist: centos 8 Core
         locale: UTF-8
        machine: x86_64
        release: 4.18.0-193.19.1.el8_2.x86_64
         system: Linux
        version: CentOS Linux 8 Core

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in salt/config/init.py at apply_sdb, specifically the iteration shown in the traceback. Reproduce two overlapping Salt-API reactor events that create VMware VMs and verify that configuration processing no longer raises RuntimeError and interrupts both VM creations.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cloud, infrastructure
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.