race condition in state.file _check_directory for transient files
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 15.7k
- Forks
- 5.6k
- Avg merge
- 2d 44m
- Merged PRs (30d)
- 80
Description
Description
A race condition exists in _check_directory where a file is removed after the walk_l is generated but the file is tested with the file.stats . While https://github.com/saltstack/salt/pull/36928 handles a similar case this race condition is different. It is harder to reproduce since the timing is smaller.
Setup
Using this state we occasionally observe (less than 1% occurrence) the failure (stack trace below)
m_dir_perms:
file.directory:
- mode: '0755'
- user: user1
- group: user1
- recurse:
- user
- group
- names:
- /opt/m
- [X ] on-prem machine
- VM (Virtualbox, KVM, etc. please specify)
- classic packaging
Steps to Reproduce the behavior
- Setup mongo with data directory of /opt/m
- Setup salt with the above state to ensure user/group recursive permissions on the directory.
Expected behavior
All files in the tree have ownership correctly set. If a transient file exists, it is not reported as a failure since it was removed.
Stack Trace
I have adjusted the file name to remove my specific file.
File \"/usr/lib/python3/dist-packages/salt/states/file.py\", line 756, in _check_directory
stats = __salt__[\"file.stats\"](path, None, follow_symlinks)
File \"/usr/lib/python3/dist-packages/salt/loader/lazy.py\", line 149, in __call__
return self.loader.run(run_func, *args, **kwargs)
File \"/usr/lib/python3/dist-packages/salt/loader/lazy.py\", line 1228, in run
return self._last_context.run(self._run_as, _func_or_method, *args, **kwargs)
File \"/usr/lib/python3/dist-packages/salt/loader/lazy.py\", line 1243, in _run_as
return _func_or_method(*args, **kwargs)
File \"/usr/lib/python3/dist-packages/salt/modules/file.py\", line 4094, in stats
raise CommandExecutionError(\"Path not found: {}\".format(path))
salt.exceptions.CommandExecutionError: Path not found: /opt/m/a/b/c",
Versions Report
Salt Version:
Salt: 3005.1
Dependency Versions:
cffi: Not Installed
cherrypy: Not Installed
dateutil: 2.7.3
docker-py: Not Installed
gitdb: 4.0.5
gitpython: 3.1.14
Jinja2: 2.10
libgit2: Not Installed
M2Crypto: Not Installed
Mako: Not Installed
msgpack: 1.0.0
msgpack-pure: Not Installed
mysql-python: Not Installed
pycparser: Not Installed
pycrypto: Not Installed
pycryptodome: 3.9.7
pygit2: Not Installed
Python: 3.9.2 (default, Feb 28 2021, 17:03:44)
python-gnupg: Not Installed
PyYAML: 5.3.1
PyZMQ: 20.0.0
smmap: 4.0.0
timelib: Not Installed
Tornado: 4.5.3
ZMQ: 4.3.1
System Versions:
dist: debian 11 bullseye
locale: utf-8
machine: x86_64
release: 5.10.0-18-amd64
system: Linux
version: Debian GNU/Linux 11 bullseye
Additional context
As noted it is hard to reproduce. On last ~150k highstates the error was observed 4 times.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in salt/states/file.py at _check_directory and inspect the salt/modules/file.py stats entry point shown in the stack trace. Reproduce with the provided file.directory state and a transient file under /opt/m if possible. Done means removed files do not produce a Path not found failure while ownership is still corrected for files that remain.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- devops
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100