fileserver: s3fs - contents are ignored
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 15.7k
- Forks
- 5.6k
- Avg merge
- 2d 44m
- Merged PRs (30d)
- 80
Description
Description of Issue
When using fileserver.s3fs, the contents of the bucket are effectively ignored, despite local cache synchronization successfully completing.
NOTE: This is running on python2, installed from https://repo.saltstack.com, to work around #53244.
Setup
/etc/salt/master.d/fileserver.conf:
file_roots:
base:
- /srv/salt
s3:
keyid: KEY
key: KEYID
buckets:
base:
- test-salt
service_url: nyc3.digitaloceanspaces.com
https_enable: True
fileserver_backend:
- roots
- s3fs
/srv/salt/top.sls:
base:
'*':
- test
s3://test-salt/test.sls:
/etc/testfile:
file.managed:
- contents:
- line1
- line2
Steps to Reproduce Issue
- Run
salt '*' state.apply - See the following returned:
minion:
Data failed to compile:
----------
No matching sls found for 'test' in env 'base'
As a sidenote, /var/cache/salt/master/s3cache/buckets_files.cache appears to be mostly empty, despite the logs suggesting cache sync is successful:
buckets_files.cache:
(dp0
.
logs, repeatedly:
[INFO ] Syncing local cache from S3...
[INFO ] Sync local cache from S3 completed.
Versions Report
Salt Version:
Salt: 2019.2.0
Dependency Versions:
cffi: Not Installed
cherrypy: unknown
dateutil: 2.6.1
docker-py: Not Installed
gitdb: 2.0.3
gitpython: 2.1.8
ioflo: Not Installed
Jinja2: 2.10
libgit2: Not Installed
libnacl: Not Installed
M2Crypto: Not Installed
Mako: 1.0.7
msgpack-pure: Not Installed
msgpack-python: 0.5.6
mysql-python: Not Installed
pycparser: Not Installed
pycrypto: 2.6.1
pycryptodome: Not Installed
pygit2: Not Installed
Python: 2.7.15rc1 (default, Nov 12 2018, 14:31:15)
python-gnupg: 0.4.1
PyYAML: 3.12
PyZMQ: 16.0.2
RAET: Not Installed
smmap: 2.0.3
timelib: Not Installed
Tornado: 4.5.3
ZMQ: 4.2.5
System Versions:
dist: Ubuntu 18.04 bionic
locale: UTF-8
machine: x86_64
release: 4.15.0-50-generic
system: Linux
version: Ubuntu 18.04 bionic
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the failure using /etc/salt/master.d/fileserver.conf, /srv/salt/top.sls, and the s3://test-salt/test.sls example, then inspect the fileserver.s3fs cache-sync entry point and /var/cache/salt/master/s3cache/buckets_files.cache. The work is done when the synchronized bucket contents are indexed and state.apply resolves the test SLS without the reported missing-SLS error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, cloud
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100