[Bug]: Salt fileserver update does not work with S3 buckets
- Dominant language
- Python
- Stars
- 15.7k
- Forks
- 5.6k
- Avg merge
- 2d 44m
- Merged PRs (30d)
- 80
Description
What happened?
Error:
Exception occurred in runner fileserver.update: Traceback (most recent call last): File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/client/mixins.py", line 383, in low data["return"] = func(*args, **kwargs) File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/loader/lazy.py", line 163, in __call__ ret = self.loader.run(run_func, *args, **kwargs) File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/loader/lazy.py", line 1275, in run return self._last_context.run(self._run_as, _func_or_method, *args, **kwargs) File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/loader/lazy.py", line 1290, in _run_as ret = _func_or_method(*args, **kwargs) File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/runners/fileserver.py", line 353, in update fileserver.update(back=backend, **kwargs) File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/fileserver/__init__.py", line 474, in update self.servers[fstr](**kwargs) File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/loader/lazy.py", line 163, in __call__ ret = self.loader.run(run_func, *args, **kwargs) File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/loader/lazy.py", line 1275, in run return self._last_context.run(self._run_as, _func_or_method, *args, **kwargs) File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/loader/lazy.py", line 1290, in _run_as ret = _func_or_method(*args, **kwargs) File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/fileserver/s3fs.py", line 125, in update metadata = _init() File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/fileserver/s3fs.py", line 371, in _init _prune_deleted_files(metadata) File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/fileserver/s3fs.py", line 594, in _prune_deleted_files cached_files.add(meta["Key"]) KeyError: 'Key'
https://github.com/saltstack/salt/blob/v3007.7/salt/fileserver/s3fs.py#L593-L594
for meta in env_data: cached_files.add(meta["Key"])
Please change this to:
for meta in env_data: for bucket in meta.keys(): for obj in meta[bucket]: cached_files.add(obj["Key"])
This has been broken for many versions now and we'd like to use a working salt installation out of the box.
Type of salt install
Official deb
Major version
3007.x
What supported OS are you seeing the problem on? Can select multiple. (If bug appears on an unsupported OS, please open a GitHub Discussion instead)
ubuntu-24.04
salt --versions-report output
Salt Version:
Salt: 3007.7
Python Version:
Python: 3.10.18 (main, Aug 27 2025, 20:27:59) [GCC 11.2.0]
Dependency Versions:
cffi: 1.16.0
cherrypy: unknown
cryptography: 42.0.5
dateutil: 2.8.2
docker-py: Not Installed
gitdb: 4.0.12
gitpython: 3.1.44
Jinja2: 3.1.6
libgit2: Not Installed
looseversion: 1.3.0
M2Crypto: Not Installed
Mako: Not Installed
msgpack: 1.0.7
msgpack-pure: Not Installed
mysql-python: Not Installed
packaging: 24.0
pycparser: 2.21
pycrypto: Not Installed
pycryptodome: 3.19.1
pygit2: Not Installed
python-gnupg: 0.5.2
PyYAML: 6.0.1
PyZMQ: 25.1.2
relenv: 0.20.5
smmap: 5.0.2
timelib: 0.3.0
Tornado: 6.4.2
ZMQ: 4.3.4
Salt Package Information:
Package Type: onedir
System Versions:
dist: ubuntu 24.04.1 noble
locale: utf-8
machine: aarch64
release: 6.14.0-1012-aws
system: Linux
version: Ubuntu 24.04.1 noble
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in salt/fileserver/s3fs.py around _prune_deleted_files(), especially the metadata handling at the reported lines. Review the linked pull request before making changes, then verify that fileserver.update handles the S3 metadata structure without raising KeyError.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, python
- Domain
- cloud
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 35/100