saltstack / saltstack/salt

[BUG] Permission denied: '/etc/salt/minion.d/mine.conf'

Open
#62,428 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug Duplicate needs-triage
Dominant language
Python
Stars
15.7k
Forks
5.6k
Avg merge
2d 44m
Merged PRs (30d)
80

Description

Description
I upgraded an old salt-master from Debian 9 to Debian 11. salt-master is now failing to start which I believe is caused by a change https://bugs.debian.org/891303 (Change salt master user from 'root' to 'salt' by default). /var/log/salt/master contains:

2022-08-04 16:18:09,119 [salt.utils.process:900 ][ERROR   ][2651] An un-handled exception from the multiprocessing process 'MWorker-3' was caught:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/salt/utils/process.py", line 895, in wrapped_run_func
    return run_func()
  File "/usr/lib/python3/dist-packages/salt/master.py", line 1237, in run
    self.clear_funcs = ClearFuncs(self.opts, self.key,)
  File "/usr/lib/python3/dist-packages/salt/master.py", line 2068, in __init__
    self.mminion = salt.minion.MasterMinion(
  File "/usr/lib/python3/dist-packages/salt/minion.py", line 981, in __init__
    self.opts = salt.config.minion_config(
  File "/usr/lib/python3/dist-packages/salt/config/__init__.py", line 2186, in minion_config
    include_config(
  File "/usr/lib/python3/dist-packages/salt/config/__init__.py", line 2065, in include_config
    opts = _read_conf_file(fn_)
  File "/usr/lib/python3/dist-packages/salt/config/__init__.py", line 1902, in _read_conf_file
    with salt.utils.files.fopen(path, "r") as conf_file:
  File "/usr/lib/python3/dist-packages/salt/utils/files.py", line 396, in fopen
    f_handle = open(*args, **kwargs)  # pylint: disable=resource-leakage
PermissionError: [Errno 13] Permission denied: '/etc/salt/minion.d/mine.conf'

I am wondering about this, thought this was resolved via #29831 / #30068.

Steps to Reproduce the behavior
Should be enough to create a permission chain like

# ls -ld /etc/salt
drwxr-xr-x 5 root root 94 Aug  4 15:53 /etc/salt
# ls -ld /etc/salt/minion.d/
drwxr-xr-x 2 root root 45 Mar 21  2017 /etc/salt/minion.d/
# ls -l /etc/salt/minion.d/mine.conf
-rw-r----- 1 root root 17 Mar 21  2017 /etc/salt/minion.d/mine.conf

Expected behavior
According to the linked two issues, minion config shouldn't cause such troubles for salt-master.

Also, salt-master shouldn't consume 100% CPU when running into this...

# ps faux | grep salt-ma
salt         892  0.4  0.3  74280 60084 ?        Ss   16:17   0:02 /usr/bin/python3 /usr/bin/salt-master
salt        1143  0.0  0.0      0     0 ?        Z    16:17   0:00  \_ [salt-master] <defunct>
salt        1871  0.0  0.3 155852 52512 ?        Sl   16:17   0:00  \_ /usr/bin/python3 /usr/bin/salt-master
salt        1872  0.0  0.3  73944 49104 ?        S    16:17   0:00  \_ /usr/bin/python3 /usr/bin/salt-master
salt        1882  0.0  0.3  74400 49348 ?        S    16:17   0:00  \_ /usr/bin/python3 /usr/bin/salt-master
salt        1883  100  0.3 450840 53360 ?        Rl   16:17   8:13  |   \_ /usr/bin/python3 /usr/bin/salt-master
salt        2649  0.0  0.0      0     0 ?        Z    16:18   0:00  |   \_ [salt-master] <defunct>
salt        1895  0.0  0.2 147992 48804 ?        Sl   16:17   0:00  \_ /usr/bin/python3 /usr/bin/salt-master
salt        2765  0.0  0.0      0     0 ?        Z    16:19   0:00  \_ [salt-master] <defunct>
root        3038  0.0  0.0   7892   712 pts/1    S+   16:26   0:00      \_ grep --colour=auto salt-ma

Versions Report

salt --versions-report
# salt-master --versions-report                                                                                                    [1/39]
Salt Version:
          Salt: 3002.6

Dependency Versions:
          cffi: Not Installed
      cherrypy: Not Installed
      dateutil: 2.8.1
     docker-py: Not Installed
         gitdb: Not Installed
     gitpython: Not Installed
        Jinja2: 2.11.3
       libgit2: Not Installed
      M2Crypto: Not Installed
          Mako: Not Installed
       msgpack: 1.0.0
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     pycparser: Not Installed
      pycrypto: Not Installed
  pycryptodome: 3.9.7
        pygit2: Not Installed
        Python: 3.9.2 (default, Feb 28 2021, 17:03:44)
  python-gnupg: Not Installed
        PyYAML: 5.3.1
         PyZMQ: 20.0.0
         smmap: Not Installed
       timelib: Not Installed
       Tornado: 4.5.3
           ZMQ: 4.3.4

System Versions:
          dist: debian 11 bullseye
        locale: utf-8
       machine: x86_64
       release: 5.10.0-16-amd64
        system: Linux
       version: Debian GNU/Linux 11 bullseye

Additional context
Already started with Debian 10, i.e. upgrade salt-master:all 2016.11.2+ds-1+deb9u10 -> 2018.3.4+dfsg1-6+deb10u3

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the traceback paths in salt/config/init.py and salt/master.py, then inspect the process handling in salt/utils/process.py and file access in salt/utils/files.py. Reproduce the unreadable /etc/salt/minion.d/mine.conf case and verify that salt-master handles the permission error without failing to start or consuming 100% CPU.

Written by the indexing model from the issue text.

Assessment

Tech stack
debian, linux, python
Domain
devops, infrastructure, operating-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.