file.recurse with multiple sources unexpected behavior when dynamic sources are provided.
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 15.7k
- Forks
- 5.6k
- Avg merge
- 2d 44m
- Merged PRs (30d)
- 80
Description
Description of Issue
When multiple sources are provided in file.recurse, and if one of the sources may be excluded by some grains data, file.recurse picks up the last source instead of the first source.
Setup
- Bootstrap salt with git (
bootstrap-salt.sh git develop; note: this problem also happens with salt-2018 the stable version). - Prepare the grain for the minion
- Create files a test state and files as below
minion grains
$ echo xfiles: foobar > /etc/salt/grains
/srv/salt/base/init.sls
"update /xfiles":
file.recurse:
- replace: True
- source:
- salt://xfiles/{{ grains.id }}/
{%- if salt["grains.get"]("xfiles") | length > 0 -%}
- salt://xfiles/{{ salt["grains.get"]("xfiles") }}/
{%- endif %}
- salt://xfiles/_default/
- name: /salt-test/
/srv/salt/xfiles/
# controller-111.internal is my salt-minion's ID
$ mkdir -pv /srv/salt/xfiles/{_default,foobar,controller-111.internal}
$ echo from _default > xfiles/_default/.empty
$ echo from foobar > xfiles/foobar/.empty
$ echo from top level > xfiles/controller-111.internal/.empty
Steps to Reproduce Issue
Step 1: With dynamic list of sources
Now execute highstate on the minion, and see the results
root@controller-111:/srv/salt# salt '*' state.highstate
controller-111.internal:
----------
ID: update /xfiles
Function: file.recurse
Name: /salt-test/
Result: True
Comment: The directory /salt-test/ is in the correct state
Started: 15:59:48.480779
Duration: 49.9999999993 ms
Changes:
Summary for controller-111.internal
------------
Succeeded: 1
Failed: 0
------------
Total states run: 1
Total run time: 50.000 ms
The result is not as expected
$ cat /salt-test/.empty
from _default
Step 2: Without dynamic list of sources:
Now uncomment the dynamic block
"update /xfiles":
file.recurse:
- replace: True
- source:
- salt://xfiles/{{ grains.id }}/
#{%- if salt["grains.get"]("xfiles") | length > 0 -%}
#- salt://xfiles/{{ salt["grains.get"]("xfiles") }}/
#{%- endif %}
- salt://xfiles/_default/
- name: /salt-test/
Apply the new state and see the expected resulted file
root@controller-111:/srv/salt# salt '*' state.highstate
controller-111.internal:
----------
ID: update /xfiles
Function: file.recurse
Name: /salt-test/
Result: True
Comment: Recursively updated /salt-test/
Started: 16:02:21.186379
Duration: 90.0000000001 ms
Changes:
----------
/salt-test/.empty:
----------
diff:
---
+++
@@ -1 +1 @@
-from _default
+from top level
Summary for controller-111.internal
------------
Succeeded: 1 (changed=1)
Failed: 0
------------
Total states run: 1
Total run time: 90.000 ms
The resulted file is execpted:
$; cat /salt-test/.empty
from top level
Versions Report
root@controller-111:~/src/salt-bootstrap# salt --versions-report
Salt Version:
Salt: 2019.8.0-50-g57fd04b
Dependency Versions:
cffi: Not Installed
cherrypy: Not Installed
dateutil: 2.6.1
docker-py: Not Installed
gitdb: 2.0.3
gitpython: 2.1.8
Jinja2: 2.10
libgit2: Not Installed
M2Crypto: 0.27.0
Mako: 1.0.7
msgpack-pure: Not Installed
msgpack-python: 0.5.6
mysql-python: Not Installed
pycparser: Not Installed
pycrypto: 2.6.1
pycryptodome: Not Installed
pygit2: Not Installed
Python: 2.7.15+ (default, Nov 27 2018, 23:36:35)
python-gnupg: 0.4.1
PyYAML: 3.12
PyZMQ: 16.0.2
smmap: 2.0.3
timelib: Not Installed
Tornado: 4.5.3
ZMQ: 4.2.5
System Versions:
dist: Ubuntu 18.04 bionic
locale: UTF-8
machine: x86_64
release: 4.15.0-55-generic
system: Linux
version: Ubuntu 18.04 bionic
``
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the /srv/salt/base/init.sls example and reproduce the behavior through state.highstate using the listed grain and source directories. Trace how file.recurse resolves its multiple source entries when a dynamic path is absent. Done means the first available source is selected consistently, with the provided expected file contents produced.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- devops, infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100