Exception while exporting backup when s3 is set as UPLOAD_PROVIDER
- Dominant language
- Python
- Stars
- 6.8k
- Forks
- 2.8k
- Avg merge
- 1d 6h
- Merged PRs (30d)
- 5
Description
**Environment**:
- CTFd Version/Commit: 3.4.x
- Operating System: Windows 10/ Ubuntu 21.04
- Web Browser and Version: Google Chrome v99.0.x
**What happened?**
I've set s3 bucket to be used for uploads and the challenge files are getting uploaded as expected over there. But when I try to export the CTFd backup, it throws the exception mentioned below. If s3 isn't used, the exports are working fine.
**How to reproduce your issue**
Set UPLOAD_PROVIDER as s3 and fill in other required configs for setting up an s3 bucket to be used for file uploads.
**Any associated stack traces or error logs**
```
ERROR [CTFd] Exception on /admin/export [GET]
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 2447, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1952, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python3.7/site-packages/flask_restx/api.py", line 672, in error_router
return original_handler(e)
File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1821, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python3.7/site-packages/flask/_compat.py", line 39, in reraise
raise value
File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1950, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1936, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/opt/CTFd/CTFd/utils/decorators/__init__.py", line 133, in admins_only_wrapper
return f(*args, **kwargs)
File "/opt/CTFd/CTFd/admin/__init__.py", line 111, in export_ctf
backup = export_ctf_util()
File "/opt/CTFd/CTFd/utils/exports/__init__.py", line 66, in export_ctf
uploader.sync()
File "/opt/CTFd/CTFd/utils/uploads/uploaders.py", line 141, in sync
bucket_list = self.s3.list_objects(Bucket=self.bucket).get("Contents", [])
File "/usr/local/lib/python3.7/site-packages/botocore/client.py", line 316, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/usr/local/lib/python3.7/site-packages/botocore/client.py", line 635, in _make_api_call
raise error_class(parsed_response, operation_name)
botocore.errorfactory.NoSuchKey: An error occurred (NoSuchKey) when calling the ListObjects operation: The specified key does not exist.
10.130.0.43 - - [14/Mar/2022:06:16:56 +0000] "GET /admin/export HTTP/1.0" 500 6911 "https:///admin/config" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.51 Safari/537.36"
```
Contributor guide
Research direction
Start at CTFd/utils/exports/__init__.py:66 and follow the uploader.sync() call into CTFd/utils/uploads/uploaders.py:141, then inspect the admin export entry point in CTFd/admin/__init__.py:111. Reproduce an export with UPLOAD_PROVIDER set to s3 and verify that the backup completes without the reported NoSuchKey exception while non-S3 exports continue to work.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, python
- Domain
- backend, cloud
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100