salt reactor - un-handled exception from the multiprocessing process
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 15.7k
- Forks
- 5.6k
- Avg merge
- 2d 44m
- Merged PRs (30d)
- 80
Description
Description of Issue
I have following very simple reactor.conf and when I trigger it with event from commandline it crashes with
An un-handled exception from the multiprocessing process 'Reactor-4:1'
As far I understand, correct sls is being used (as defined in reactor), then it is being read to OrderedDict, then it tries populate_client_cache and fails.
Setup
reactor.conf on master (/etc/salt/master.d/reactor.conf)
it is the only reactor
reactor:
- 'salt/beacon/ssc-001/diskusage/':
- salt://role/ssc/cleanup.sls
cleanup.sls
create dummy file:
file.managed:
- tgt: 'target'
- name: /tmp/wojtek
- source: salt://role/ssc/files/cleanup.py
Steps to Reproduce Issue
- run
salt-master -l debugon master - trigger the event on minion:
salt-call event.send 'salt/beacon/ssc-001/diskusage/'
[DEBUG ] Rendered data from file: /var/cache/salt/master/files/base/role/ssc/cleanup.sls:
create dummy file:
file.managed:
- tgt: 'ssc-001'
- name: /tmp/wojtek
- source: salt://role/ssc/files/cleanup.py
[DEBUG ] Results of YAML rendering:
OrderedDict([(u'create dummy file', OrderedDict([(u'file.managed', [OrderedDict([(u'tgt', u'ssc-001')]), OrderedDict([(u'name', u'/tmp/wojtek')]), OrderedDict([(u'source', u'salt://role/ssc/files/cleanup.py')])])]))])
[PROFILE ] Time (in seconds) to render '/var/cache/salt/master/files/base/role/ssc/cleanup.sls' using 'yaml' renderer: 0.000705003738403
[DEBUG ] Reactor is populating file client cache
[ERROR ] An un-handled exception from the multiprocessing process 'Reactor-4:1' was caught:
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/salt/utils/process.py", line 765, in _run
return self._original_run()
File "/usr/lib/python2.7/site-packages/salt/utils/reactor.py", line 271, in run
self.call_reactions(chunks)
File "/usr/lib/python2.7/site-packages/salt/utils/reactor.py", line 228, in call_reactions
self.wrap.run(chunk)
File "/usr/lib/python2.7/site-packages/salt/utils/reactor.py", line 330, in run
self.populate_client_cache(low)
File "/usr/lib/python2.7/site-packages/salt/utils/reactor.py", line 324, in populate_client_cache
self.reaction_class[reaction_type](self.opts['conf_file'])
KeyError: u'file'
[CRITICAL] Engine 'reactor' could not be started!
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/salt/engines/__init__.py", line 132, in run
self.engine[self.fun](**kwargs)
File "/usr/lib/python2.7/site-packages/salt/engines/reactor.py", line 36, in start
salt.utils.reactor.Reactor(__opts__).run()
File "/usr/lib/python2.7/site-packages/salt/utils/process.py", line 765, in _run
return self._original_run()
File "/usr/lib/python2.7/site-packages/salt/utils/reactor.py", line 271, in run
self.call_reactions(chunks)
File "/usr/lib/python2.7/site-packages/salt/utils/reactor.py", line 228, in call_reactions
self.wrap.run(chunk)
File "/usr/lib/python2.7/site-packages/salt/utils/reactor.py", line 330, in run
self.populate_client_cache(low)
File "/usr/lib/python2.7/site-packages/salt/utils/reactor.py", line 324, in populate_client_cache
self.reaction_class[reaction_type](self.opts['conf_file'])
KeyError: u'file'
Versions Report
Salt Version:
Salt: 2019.2.0
Dependency Versions:
cffi: 1.6.0
cherrypy: Not Installed
dateutil: Not Installed
docker-py: Not Installed
gitdb: Not Installed
gitpython: Not Installed
ioflo: Not Installed
Jinja2: 2.7.2
libgit2: 0.26.3
libnacl: Not Installed
M2Crypto: Not Installed
Mako: Not Installed
msgpack-pure: Not Installed
msgpack-python: 0.5.6
mysql-python: Not Installed
pycparser: 2.14
pycrypto: 2.6.1
pycryptodome: Not Installed
pygit2: 0.26.4
Python: 2.7.5 (default, Jun 20 2019, 20:27:34)
python-gnupg: Not Installed
PyYAML: 3.11
PyZMQ: 15.3.0
RAET: Not Installed
smmap: Not Installed
timelib: Not Installed
Tornado: 4.2.1
ZMQ: 4.1.4
System Versions:
dist: centos 7.6.1810 Core
locale: UTF-8
machine: x86_64
release: 3.10.0-957.27.2.el7.x86_64
system: Linux
version: CentOS Linux 7.6.1810 Core
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in salt/utils/reactor.py at populate_client_cache and the reaction_class lookup shown in the traceback. Reproduce with the provided reactor.conf, cleanup.sls, and salt-call event.send command while running salt-master -l debug. Done means the file.managed reaction no longer raises KeyError u'file' or stops the reactor.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- devops, infrastructure
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 35/100