django / django/daphne

Memory usage: Daphne loading all the file in memory (POST request)

Open
#483 7 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
2.7k
Forks
292
Avg merge
2d 15h
Merged PRs (30d)
3

Description

I am using Debian packaged versions (Debian 12 Bullseye) using:
- daphne 4.0.0
- django 3.2.19
- python 3.11.2

The problem happens when using Django with daphne (debugging with runserver or in production with Nginx in front of it).

For testing purposes I have a form in the application accepting a 4 GB file (with Nginx accepting this file size). This is to make it more visible.

When the request makes it to daphne in (Twisted and Python's cgi.py are not reading it into memory for what I can see, they use a temporary file or passing it into daphne without full read):
https://github.com/django/daphne/blob/4.0.0/daphne/http_protocol.py#L188

Daphne keeps reading the 4 GB and adding it to the queue in 8 KB chunks (the queue was created without any max_size).

If I use uvicorn (or gunicorn with uvicorn workers) the problem does not happen: no memory change POSTing a 4 GB file. If I use runserver without Daphne's application it does not happen either.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.