ext_pillar: s3 cannot be referred to from top.sls
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 15.7k
- Forks
- 5.6k
- Avg merge
- 2d 44m
- Merged PRs (30d)
- 80
Description
Description of Issue
When using s3 as an ext_pillar (on python2, installed from https://repo.saltstack.com, to work around #53244), it ends up in cache (somewhat strangely), but can't be used.
Setup
/etc/salt/master.d/pillar.conf:
pillar_roots:
base:
- /srv/pillar
ext_pillar:
- s3:
bucket: test-pillar
keyid: KEYID
key: KEY
multiple_env: False
environment: base
service_url: nyc3.digitaloceanspaces.com
https_enable: True
/srv/pillar/top.sls:
base:
'*':
- key
s3://test-pillar/key.sls:
key: value
Steps to Reproduce Issue
- Run
salt '*' saltutil.refresh_pillarwith the above configuration - See the following in the logs
[ERROR ] Specified SLS 'key' in environment 'base' is not available on the salt master
[CRITICAL] Pillar render error: Specified SLS 'key' in environment 'base' is not available on the salt master
Due to the strange output of tree in the cache directory, I also tried using test-pillar.key in top.sls, but the results were the same.
tree /var/cache/salt/master/pillar_s3fs:
/var/cache/salt/master/pillar_s3fs
├── base
│ └── test-pillar
│ └── key.sls
└── test-pillar--files.cache
(key.sls contains the expected contents)
If the same bucket is mounted and the path specifies in roots, everything works as expected.
Versions Report
Salt Version:
Salt: 2019.2.0
Dependency Versions:
cffi: Not Installed
cherrypy: unknown
dateutil: 2.6.1
docker-py: Not Installed
gitdb: 2.0.3
gitpython: 2.1.8
ioflo: Not Installed
Jinja2: 2.10
libgit2: Not Installed
libnacl: Not Installed
M2Crypto: Not Installed
Mako: 1.0.7
msgpack-pure: Not Installed
msgpack-python: 0.5.6
mysql-python: Not Installed
pycparser: Not Installed
pycrypto: 2.6.1
pycryptodome: Not Installed
pygit2: Not Installed
Python: 2.7.15rc1 (default, Nov 12 2018, 14:31:15)
python-gnupg: 0.4.1
PyYAML: 3.12
PyZMQ: 16.0.2
RAET: Not Installed
smmap: 2.0.3
timelib: Not Installed
Tornado: 4.5.3
ZMQ: 4.2.5
System Versions:
dist: Ubuntu 18.04 bionic
locale: UTF-8
machine: x86_64
release: 4.15.0-50-generic
system: Linux
version: Ubuntu 18.04 bionic
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at the S3 ext_pillar entry point and trace how the cached base/test-pillar/key.sls path is exposed to pillar resolution. Reproduce with the supplied pillar.conf, top.sls, and cache layout; done means saltutil.refresh_pillar resolves key from S3 without the “SLS 'key' ... is not available” error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- cloud, infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100