saltstack / saltstack/salt

[BUG] ext_pillar pillarstack opts matching: pillar.items returns nothing

Open
#57,580 16 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug Pillar severity-high
Dominant language
Python
Stars
15.7k
Forks
5.6k
Avg merge
2d 44m
Merged PRs (30d)
80

Description

Looking at documentation for pillarstack ext_pillar https://docs.saltstack.com/en/latest/ref/pillar/all/salt.pillar.stack.html it should be possible to define stack config files depending of grains, pillar or opts, like in the example

ext_pillar:
  - stack:
      pillar:environment:
        dev: /path/to/dev/stack.cfg
        prod: /path/to/prod/stack.cfg
      grains:custom:grain:
        value:
          - /path/to/stack1.cfg
          - /path/to/stack2.cfg
      opts:custom:opt:
        value: /path/to/stack0.cfg

I tried pillar matching and it works but opts don't.

Here is one basic pillarstack setup that works. Minion has a 'saltenv' setting (it will be used later).
Pillarstack is configured to simply read all yml files in dev or prod directory and only set a corresponding env pillar.

### salt-master config file
ext_pillar:
  - stack:
      - /srv/pillar/dev/base.cfg

### salt-minion config file
saltenv: prod

### pillar directory
/srv/pillar$ tree .
.
├── dev
│   ├── base.cfg
│   └── env.yml
└── prod
    ├── base.cfg
    └── env.yml

### files content
$ cat */base.cfg
*.yml

$ cat */*.yml
---
env: dev
---
env: prod

Results of commands

$ salt 'minion1' config.get saltenv
minion1:
    prod

$ salt 'minion1' pillar.items
minion1:
    ----------
    env:
        dev

config.get returns the __opts__ salt dictionary which contains saltenv.
Now we change pillarstack config to match on this opts saltenv value:

ext_pillar:
  - stack:
      opts:saltenv:
        dev: /srv/pillar/dev/base.cfg
        prod: /srv/pillar/prod/base.cfg
$ salt 'minion1' config.get saltenv
minion1:
    prod

$ salt 'minion1' pillar.items
minion1:
    ----------

As you can see, the saltenv is set and pillar.items returns nothing.
What's weird is that:

$ salt 'minion1' pillar.get env
minion1:
    prod

A specific pillar.get correctly returns the expected value!

Adding some awful log.debug() inside salt/pillar/stack.py I see that it parses the right /srv/pillar/{dev,prod}/base.cfg and *.yml files depending of the saltenv setting

I don't really understand where this bug can be, from inside or outside stack.py (pillarstack) but it seems the module does the job correctly, reading the right files and filling pillar data... but... still pillar.items returns nothing.

Master and minion on version 2019.2.5+ds-1, running on Debian 9.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with salt/pillar/stack.py and reproduce the documented opts:saltenv configuration using the Debian 9 and Salt 2019.2.5 setup described in the issue. Compare the opts and pillar matching paths, then verify that pillar.items returns the expected env value while pillar.get continues to work.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.