django / django/djangoproject.com
Add optional django-storages (S3) support for media
- Dominant language
- Python
- Stars
- 2k
- Forks
- 1.1k
- Avg merge
- 2d 7h
- Merged PRs (30d)
- 26
Description
Uploaded media and release artifacts currently live on the server's local file system (`MEDIA_ROOT = DATA_DIR / "media"` in `prod.py`), which must be copied when replacing the production server. This issue adds **optional** S3 storage via [django-storages](https://django-storages.readthedocs.io/), selected by environment configuration. Local file system storage remains the default; the feature is enabled from the deployment repo once this lands.
## Changes (this repo)
- **Dependency**: add `django-storages[boto3]` (prod group in `pyproject.toml`, regenerate `requirements/prod.txt`).
- **Settings** (`djangoproject/settings/prod.py`): select `STORAGES["default"]`'s backend based on environment configuration (e.g., use `storages.backends.s3.S3Storage` when the bucket name is set, otherwise keep `FileSystemStorage`). All S3 config (bucket, region) comes from the environment / `secrets.json`; nothing hardcoded. The EC2 instance will have a role with S3 access, so no explicit credentials are required.
- **Release artifacts** (`releases/models.py`): `get_storage()` explicitly returns `FileSystemStorage(allow_overwrite=True)`; make it return the S3 equivalent (with `allow_overwrite=True`) when S3 is enabled, so artifact uploads keep their exact names.
- **Docs release zips** (`docs/management/commands/build_doc_release.py`): the docs HTML zip is written directly to `settings.MEDIA_ROOT / "docs"`; write it through the default storage instead so it lands in S3 when enabled.
- **sorl-thumbnail** (fundraising logos, `djangoproject/thumbnails.py`): verify `get_thumbnail()` works with an S3 source/storage, and configure `THUMBNAIL_STORAGE` so generated thumbnails are written to the bucket.
- **Tests**: suite must pass without AWS credentials (default configuration unchanged); add coverage for the S3 configuration path (backend selection and file operations against a mocked S3 backend, e.g. `moto` or `django-storages`'s in-memory S3).
## Out of scope
- **Enabling in the deployment repo** (private Ansible repo): S3 bucket, the env vars that turn the feature on, one-time copy of existing `media/` contents, nginx changes for `media.djangoproject.com` (including the pre-1.4 `/css|img/` redirect), and dropping the `media/` directory from automated backups.
- **Trac**: config and attachments stay on the file system.
- **Docs rendering**: `DOCS_BUILD_ROOT` and the docs cache remain local; only the downloadable zip is affected (see above).
## AI Disclosure
This issue description was drafted with the assistance of a local model, Qwen 3.8 27B.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start in pyproject.toml, requirements/prod.txt, and djangoproject/settings/prod.py to trace production storage configuration and its environment inputs. Then inspect releases/models.py, docs/management/commands/build_doc_release.py, and djangoproject/thumbnails.py, and run the existing test suite before adding coverage for the S3 path with a mocked backend. Done means local storage remains the default, S3 handles media, artifacts, docs zips, and thumbnails when enabled, and tests pass without AWS credentials.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, django, python
- Domain
- backend, cloud, testing
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 52/100