drogonframework / drogonframework/drogon
Implement Early Filtering for File Uploads to Optimize Server Resources.
- Dominant language
- C++
- Stars
- 14.3k
- Forks
- 1.4k
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 15
Description
**Is your feature request related to a problem? Please describe.**
I am encountering a challenge with my server's file upload functionality. Currently, the server is configured to accept large upload body sizes because the file sizes are not predetermined. However, this setup poses a risk of the server being overwhelmed with random, large data uploads. To mitigate this, I introduced a filter for the upload API. Unfortunately, during testing, I discovered that the server processes the entire HTTP body before applying these filters. This approach is functional for servers dealing with small data transfers or primarily serving data. However, in scenarios with large data uploads, it results in significant bandwidth usage and the creation of numerous temporary files on the disk. Implementing early filtering could greatly alleviate these issues.
**Describe the solution you'd like**
I suggest adding a new set of filters that only examine the path and headers of the HTTP request. These filters would be applied immediately after the server receives the HTTP headers, thus enabling early decision-making about whether to accept or reject the upload request based on predefined criteria. This solution should significantly reduce unnecessary data processing and resource allocation for disallowed or potentially harmful uploads.
**Describe alternatives you've considered**
An alternative I have considered is restructuring the API to support chunked file uploads. This approach would involve rewriting the file upload process to accept smaller pieces of the file sequentially, rather than requiring the entire file at once. While this could also mitigate the issue, it would require more extensive changes to the existing API and client-side implementations.
Contributor guide
Assessment
This issue has not been assessed yet.