[BUG] archive.extracted doesn't use pillar data for s3.key and s3.keyid
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 15.7k
- Forks
- 5.6k
- Avg merge
- 2d 44m
- Merged PRs (30d)
- 80
Description
Description
This is the same behavior as #13850 except when the caching is ran from archive.extracted. If the s3.key and s3.keyid pillars are not set in the minion config files, then an attempt to grab IAM roles is made, follow by an exception.
Could not fetch from s3://<path>/file.tar. Exception: Failed to get file. InvalidAccessKeyId: The AWS Access Key Id you provided does not exist in our records.
If the same values that are assigned to the pillars from the master are directly added as local config on the minion, the caching succeeds.
#28630 contained a fix for some other modules in https://github.com/saltstack/salt/pull/28630/commits/4d38687ed6ccdd999e543c96993cabcdd3207920
Running modules on the minion using the pillar data works, such as s3.head <bucket> <file.tar>.
Setup
unpack_cb_installer:
archive.extracted:
- name: /tmp/cbinstaller
- source: s3://<path>/file.tar
- source_hash: <hash>
- unless:
- fun: pkg.version
args:
- <pkgname>
Steps to Reproduce the behavior
Running salt 'minion' state.sls_id unpack_cb_installer cb
----------
ID: unpack_cb_installer
Function: archive.extracted
Name: /tmp/cbinstaller
Result: False
Comment: Could not fetch from s3://<path>/file.tar. Exception: Failed to get file. InvalidAccessKeyId: The AWS Access Key Id you provided does not exist in our records.
Started: 12:42:18.517717
Duration: 8701.662 ms
Changes:
----------
Expected behavior
Pillar data provided by master should be used for archive.extracted when using s3 as a source.
Versions Report
salt --versions-report
(Provided by running salt --versions-report. Please also mention any differences in master/minion versions.)minion# salt-call --versions-report
Salt Version:
Salt: 3003
Dependency Versions:
cffi: Not Installed
cherrypy: Not Installed
dateutil: Not Installed
docker-py: Not Installed
gitdb: Not Installed
gitpython: Not Installed
Jinja2: 2.11.1
libgit2: Not Installed
M2Crypto: 0.35.2
Mako: Not Installed
msgpack: 0.6.2
msgpack-pure: Not Installed
mysql-python: Not Installed
pycparser: Not Installed
pycrypto: Not Installed
pycryptodome: Not Installed
pygit2: Not Installed
Python: 3.6.8 (default, Nov 16 2020, 16:55:22)
python-gnupg: Not Installed
PyYAML: 3.13
PyZMQ: 17.0.0
smmap: Not Installed
timelib: Not Installed
Tornado: 4.5.3
ZMQ: 4.1.4
System Versions:
dist: centos 7 Core
locale: UTF-8
machine: x86_64
release: 3.10.0-1160.25.1.el7.x86_64
system: Linux
version: CentOS Linux 7 Core
master#: salt-call --versions-report
Salt Version:
Salt: 3003
Dependency Versions:
cffi: 1.14.5
cherrypy: 8.9.1
dateutil: 2.7.3
docker-py: Not Installed
gitdb: 2.0.6
gitpython: 3.0.7
Jinja2: 2.10.1
libgit2: 0.28.4
M2Crypto: 0.31.0
Mako: Not Installed
msgpack: 0.6.2
msgpack-pure: Not Installed
mysql-python: Not Installed
pycparser: 2.20
pycrypto: Not Installed
pycryptodome: 3.6.1
pygit2: 0.28.2
Python: 3.8.10 (default, Jun 2 2021, 10:49:15)
python-gnupg: 0.4.5
PyYAML: 5.3.1
PyZMQ: 18.1.1
smmap: 2.0.5
timelib: Not Installed
Tornado: 4.5.3
ZMQ: 4.3.2
System Versions:
dist: ubuntu 20.04 focal
locale: utf-8
machine: x86_64
release: 5.4.0-67-generic
system: Linux
version: Ubuntu 20.04 focal
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the archive.extracted S3 source path and compare its credential handling with s3.head, using the behavior described in #13850 and the referenced #28630 change as context. Reproduce the state.sls_id example and verify that master-provided s3.key and s3.keyid pillar values let the archive be fetched successfully.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, python
- Domain
- cloud
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100