saltstack / saltstack/salt

[BUG] `config.get` appears to ignore pillar values esp. for `saltenv`

Open
#63,594 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug needs-triage
Dominant language
Python
Stars
15.7k
Forks
5.6k
Avg merge
2d 44m
Merged PRs (30d)
80

Description

Description
Setting saltenv: ts-dev in pillar for a minion and then running config.get saltenv returns None, not ts-dev from the pillar.

Setup
Install salt-minion version 3005.1 classic from the RedHat 8 repository on an up-to-date Almalinux 8.7 minimal system (within QEMU in my case). Configure the minion:

$ echo "$MINION_ID" > /etc/salt/minion_id
$ echo "master: $SALT_MASTER_ADDR" > /etc/salt/minion.d/99-master-address.conf
$ cat <<EOF > /etc/salt/minion
master_finger: "$SALT_MASTER_FINGER"
state_output: mixed
EOF
$ systemctl restart salt-minion

The following are the relevant values of my master configuration:

user: root
conf_file: /etc/salt/master
fileserver_backend:
  - git

gitfs_provider: pygit2
gitfs_ref_types:
  - branch
gitfs_base: Production
gitfs_remotes:
  - https://git.xxx.xxx/xxx/saltstates.git:
    - root: states

state_top_saltenv: base
top_file_merging_strategy: same

ext_pillar:
  - custom_mongo: { configuration for our custom MongoDB ext_pillar here }
  - gpg:
  • on-prem machine
  • VM (Virtualbox, KVM, etc. please specify)
  • VM running on a cloud service, please be explicit and add details
  • container (Kubernetes, Docker, containerd, etc. please specify)
  • or a combination, please be explicit
  • jails if it is FreeBSD
  • classic packaging
  • onedir packaging
  • used bootstrap to install

Steps to Reproduce the behavior
Set saltenv to ts-dev within the minion pillar.

$ salt-call pillar.get saltenv
local:
    ts-dev

I personally have an ext_pillar module that uses MongoDB to store pillar data. I have nothing set for pillarenv as all my pillar data should be the same regardless of environment.

Look up the value on the minion using config.get:

$ salt-call config.get saltenv
local:
    None

According to the config.get documentation, config.get looks at the minion configuration for a value first. Confirm that saltenv and environment are not set anywhere on the minion (delete any matching lines):

$ grep -riE '^[^#]*env.*$' /etc/salt/*
$

Confirm that the minion's grains do not have a value (should see no results):

$ salt-call grains.items | grep -i 'env'
$

According to the documentation, the next step should be for config.get to check pillars. Just to be certain, restart the minion again, refresh pillars, confirm the value is present and environment is not set:

$ systemctl restart salt-minion && salt-call saltutil.refresh_pillar
local:
    True
$ salt-call pillar.items | grep -i -A1 'env'
    saltenv:
        ts-dev
$ salt-call config.get saltenv
local:
    None

Expected behavior
If saltenv and environment are unset in the minion configuration and grains, and saltenv is set in pillar, I expect config.get saltenv to return the value of saltenv as set in the pillar. Interestingly, if I set environment: ts-dev in my pillar and use config.get, it returns ts-dev (and according to the documentation, saltenv should pick up its value from environment if it is unset, so I should be able to apply states with just environment set and the states should come from my ts-dev git branch. But, they don't).

Screenshots
image

Versions Report

salt --versions-report on master
Salt Version:
          Salt: 3005.1
 
Dependency Versions:
          cffi: 1.14.5
      cherrypy: Not Installed
      dateutil: 2.6.1
     docker-py: Not Installed
         gitdb: Not Installed
     gitpython: Not Installed
        Jinja2: 2.11.2
       libgit2: 0.99.0
      M2Crypto: 0.35.2
          Mako: Not Installed
       msgpack: 0.6.2
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     pycparser: 2.20
      pycrypto: 2.6.1
  pycryptodome: 3.9.9
        pygit2: 1.1.1
        Python: 3.6.8 (default, Oct 26 2022, 09:13:21)
  python-gnupg: Not Installed
        PyYAML: 3.12
         PyZMQ: 20.0.0
         smmap: Not Installed
       timelib: Not Installed
       Tornado: 4.5.3
           ZMQ: 4.3.4
 
System Versions:
          dist: centos 8 
        locale: UTF-8
       machine: x86_64
       release: 4.18.0-408.el8.x86_64
        system: Linux
       version: CentOS Stream 8
salt-call --versions-report on minion
Salt Version:
          Salt: 3005.1
 
Dependency Versions:
          cffi: 1.11.5
      cherrypy: Not Installed
      dateutil: 2.6.1
     docker-py: Not Installed
         gitdb: Not Installed
     gitpython: Not Installed
        Jinja2: 2.10.1
       libgit2: Not Installed
      M2Crypto: 0.35.2
          Mako: Not Installed
       msgpack: 0.6.2
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     pycparser: 2.14
      pycrypto: 2.6.1
  pycryptodome: Not Installed
        pygit2: Not Installed
        Python: 3.6.8 (default, Sep 13 2022, 07:19:15)
  python-gnupg: Not Installed
        PyYAML: 3.12
         PyZMQ: 20.0.0
         smmap: Not Installed
       timelib: Not Installed
       Tornado: 4.5.3
           ZMQ: 4.3.4
 
System Versions:
          dist: almalinux 8.7 Stone Smilodon
        locale: UTF-8
       machine: x86_64
       release: 4.18.0-425.10.1.el8_7.x86_64
        system: Linux
       version: AlmaLinux 8.7 Stone Smilodon

Additional context
The ultimate goal is to choose which git branch a minion gets state information from by setting the environment in the minion's pillar. If none is set, I want the minion to default to base (Production branch in my case). This way, a highstate will select the environment dynamically, as opposed to writing states that will get the pillar value, change the minion config, cancel the highstate, restart the minion service, and then re-highstate in the correct environment.

top.sls of Production:

{{ saltenv }}:
  'Role_Config:Role:Name:ABC':
    - match: pillar
    - roles.ABC
# ...

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the config.get documentation and reproduce the difference between salt-call pillar.get saltenv and salt-call config.get saltenv using the commands in the issue. Trace how config.get resolves minion configuration, grains, and pillar values, then verify that an unset configuration and grains value allow the pillar’s saltenv or environment value to be returned.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
devops, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.