[BUG] pillar.stack inconsistent pillar.items/pillar.get and saltenv/pillarenv
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 15.7k
- Forks
- 5.6k
- Avg merge
- 2d 44m
- Merged PRs (30d)
- 80
Description
Description
When using opts.pillarenv and opts.saltenv in pillar.stack, pillar.items and pillar.get give inconsistent answers.
I suspect that's because they're called with inconsistent saltenv and pillarenv values in __opts__:
# salt-call pillar.items
local:
----------
foo:
----------
opts.pillarenv:
master
opts.saltenv:
None
# salt-call pillar.get foo
local:
----------
opts.pillarenv:
None
opts.saltenv:
master
Setup
master.conf:
ext_pillar:
- stack:
opts:saltenv:
master: /tmp/foo.cfg
opts:pillarenv:
master: /tmp/foo.cfg
/tmp/foo.cfg:
foo.yml
/tmp/foo.yml:
foo:
saltenv: {{ saltenv }}
pillarenv: {{ pillarenv }}
opts.pillarenv: {{ __opts__['pillarenv'] }}
opts.saltenv: {{ __opts__['saltenv'] }}
Steps to Reproduce the behavior
An extra debug line (diff below) in stack.py sheds some light:
salt-call pillar.get foo calls ext_pillar.stack with __opts__['pillarenv']=None.
[INFO ] ext_pillar.stack: __opts__['saltenv']=master __opts__['pillarenv']=None pillar:{'foo': {'saltenv': None, 'pillarenv': None, 'opts.pillarenv': 'None', 'opts.saltenv': 'master'}}
salt-call pillar.items does the same, but the result is discarded. Then ext_pillar.stack is called a second time with __opts__['pillarenv']=master
The results from the second call are used as the result:
[INFO ] ext_pillar.stack: __opts__['saltenv']=master __opts__['pillarenv']=None pillar:{'foo': {'saltenv': None, 'pillarenv': None, 'opts.pillarenv': 'None', 'opts.saltenv': 'master'}}
[INFO ] ext_pillar.stack: __opts__['saltenv']=None __opts__['pillarenv']=master pillar:{'foo': {'saltenv': None, 'pillarenv': None, 'opts.pillarenv': 'master', 'opts.saltenv': 'None'}}
diff --git a/salt/pillar/stack.py b/salt/pillar/stack.py
index 57935021e6..f69d762b2a 100644
--- a/salt/pillar/stack.py
+++ b/salt/pillar/stack.py
@@ -413,6 +413,7 @@ def ext_pillar(minion_id, pillar, *args, **kwargs):
log.info('Ignoring pillar stack cfg "%s": file does not exist', cfg)
continue
stack = _process_stack_cfg(cfg, stack, minion_id, pillar)
+ log.info(f"ext_pillar.stack: __opts__['saltenv']={__opts__['saltenv']} __opts__['pillarenv']={__opts__['pillarenv']} pillar:{stack}")
return stack
Expected behavior
I would expect pillar.items and pillar.get to give consistent results. And I do not understand why pillar.items makes two calls.
The supplied test.conf is a workaround for this bug; when removing either the opts.pillarenv or opts.saltenv sections (and restarting the salt-master) pillar.items or pillar.get gives an empty result.
Versions Report
salt --versions-report
Salt Version:
Salt: 3005+197.g1a546537b1
Dependency Versions:
cffi: 1.15.1
cherrypy: Not Installed
dateutil: 2.8.2
docker-py: Not Installed
gitdb: Not Installed
gitpython: Not Installed
Jinja2: 3.1.2
libgit2: Not Installed
M2Crypto: Not Installed
Mako: Not Installed
msgpack: 1.0.4
msgpack-pure: Not Installed
mysql-python: Not Installed
pycparser: 2.21
pycrypto: Not Installed
pycryptodome: 3.15.0
pygit2: Not Installed
Python: 3.10.6 (main, Aug 2 2022, 15:11:03) [GCC 7.5.0]
python-gnupg: 0.4.1
PyYAML: 3.12
PyZMQ: 20.0.0
smmap: Not Installed
timelib: Not Installed
Tornado: 4.5.3
ZMQ: 4.3.3
System Versions:
dist: ubuntu 18.04 Bionic Beaver
locale: utf-8
machine: x86_64
release: 4.15.0-191-generic
system: Linux
version: Ubuntu 18.04 Bionic Beaver```
</details>
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in salt/pillar/stack.py at ext_pillar and reproduce the behavior with salt-call pillar.items and salt-call pillar.get using the supplied master.conf and /tmp/foo.cfg and /tmp/foo.yml. Trace the differing saltenv and pillarenv values and the two pillar.items calls. Done means both commands return consistent values and the unnecessary or inconsistent call behavior is covered by the relevant tests, if available.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- devops, infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100