Subfilters in top.sls are broken
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 15.7k
- Forks
- 5.6k
- Avg merge
- 2d 44m
- Merged PRs (30d)
- 80
Description
Description
The subfilter keyword in a state tree top.sls is undocumented, has no tests and fails with the following exception:
File "/usr/lib/python3/dist-packages/salt/state.py", line 4073, in _filter_matches
env_key, inc_sls = item.popitem()
KeyError: 'dictionary is empty'
Setup
Define the following top.sls:
base:
'min*':
- state0
- subfilter:
'*ini*':
- state1
- subfilter:
'*ion1':
- state2
Where state0.sls, state1.sls, and state2.sls can look like this:
state0:
test.succeed_with_changes: []
Then add a minion named minion1 and run the highstate:
salt-call state.apply
[ERROR ] An un-handled exception was caught by Salt's global exception handler:
KeyError: 'dictionary is empty'
Traceback (most recent call last):
File "/usr/bin/salt-call", line 11, in <module>
load_entry_point('salt==3005.1', 'console_scripts', 'salt-call')()
File "/usr/lib/python3/dist-packages/salt/scripts.py", line 441, in salt_call
client.run()
File "/usr/lib/python3/dist-packages/salt/cli/call.py", line 50, in run
caller.run()
File "/usr/lib/python3/dist-packages/salt/cli/caller.py", line 95, in run
ret = self.call()
File "/usr/lib/python3/dist-packages/salt/cli/caller.py", line 202, in call
ret["return"] = self.minion.executors[fname](
File "/usr/lib/python3/dist-packages/salt/loader/lazy.py", line 149, in __call__
return self.loader.run(run_func, *args, **kwargs)
File "/usr/lib/python3/dist-packages/salt/loader/lazy.py", line 1228, in run
return self._last_context.run(self._run_as, _func_or_method, *args, **kwargs)
File "/usr/lib/python3/dist-packages/salt/loader/lazy.py", line 1243, in _run_as
return _func_or_method(*args, **kwargs)
File "/usr/lib/python3/dist-packages/salt/executors/direct_call.py", line 10, in execute
return func(*args, **kwargs)
File "/usr/lib/python3/dist-packages/salt/loader/lazy.py", line 149, in __call__
return self.loader.run(run_func, *args, **kwargs)
File "/usr/lib/python3/dist-packages/salt/loader/lazy.py", line 1228, in run
return self._last_context.run(self._run_as, _func_or_method, *args, **kwargs)
File "/usr/lib/python3/dist-packages/salt/loader/lazy.py", line 1243, in _run_as
return _func_or_method(*args, **kwargs)
File "/usr/lib/python3/dist-packages/salt/modules/state.py", line 794, in apply_
return highstate(**kwargs)
File "/usr/lib/python3/dist-packages/salt/modules/state.py", line 1132, in highstate
ret = st_.call_highstate(
File "/usr/lib/python3/dist-packages/salt/state.py", line 4609, in call_highstate
matches = self.top_matches(top)
File "/usr/lib/python3/dist-packages/salt/state.py", line 4080, in top_matches
_filter_matches(match, data, self.opts["nodegroups"])
File "/usr/lib/python3/dist-packages/salt/state.py", line 4066, in _filter_matches
_filter_matches(match, data, _opts)
File "/usr/lib/python3/dist-packages/salt/state.py", line 4073, in _filter_matches
env_key, inc_sls = item.popitem()
KeyError: 'dictionary is empty'
Expected behavior
The expected behavior is that state0, state1, and state2 are applied
Versions Report
salt --versions-report
(Provided by running salt --versions-report. Please also mention any differences in master/minion versions.)Salt Version:
Salt: 3005.1
Dependency Versions:
cffi: Not Installed
cherrypy: Not Installed
dateutil: 2.7.3
docker-py: Not Installed
gitdb: 2.0.6
gitpython: 3.0.7
Jinja2: 2.10.1
libgit2: Not Installed
M2Crypto: Not Installed
Mako: Not Installed
msgpack: 0.6.2
msgpack-pure: Not Installed
mysql-python: Not Installed
pycparser: Not Installed
pycrypto: Not Installed
pycryptodome: 3.6.1
pygit2: Not Installed
Python: 3.8.10 (default, Jun 2 2021, 10:49:15)
python-gnupg: 0.4.5
PyYAML: 5.3.1
PyZMQ: 20.0.0
smmap: 2.0.5
timelib: Not Installed
Tornado: 4.5.3
ZMQ: 4.3.2
System Versions:
dist: ubuntu 20.04 focal
locale: utf-8
machine: x86_64
release: 5.4.0-77-generic
system: Linux
version: Ubuntu 20.04 focal
Additional context
The feature was added in https://github.com/saltstack/salt/pull/10013. The tests were planned but not implemented so far.
My suggestion is either to drop the feature (it was broken in 2016 in https://github.com/saltstack/salt/pull/38296 that also has no tests), or fix and document it.
Cc: @bretep @nevins-b
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in salt/state.py at _filter_matches and top_matches, using the nested top.sls example to reproduce the empty-dictionary exception. Check the history of pull requests 10013 and 38296, then determine whether the feature should be fixed or removed. Done means the decision is implemented, nested subfilters have regression coverage, and the behavior is documented if retained.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- devops, infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100