tornadoweb / tornadoweb/tornado

parse_multipart_form_data for stream_request_body

Open
#1,842 11 comments 5 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

web
Dominant language
Python
Stars
22.2k
Forks
5.6k
Avg merge
3h 42m
Merged PRs (30d)
16

Description

Hi!

We have a good feature for streaming HTTP body (decorator stream_request_body).
Using stream_request_body we get chunks of HTTP body in data_received method. But this data is raw. And It will be cool to have a support for higher level tool to work with streaming body (in case of multipart_form_data).

If it is not solved yet I can do this, but I'm not sure about architecture for solution.
I want to implement smth like StreamParser, which will get boundary and infinite generator which will return file-like objects. This parser we can instantiate in prepare method and then simply pass chunks in data_received method to it. Parser will write to these file-like objects and efficiently use RAM. But this solution has drawbacks. At least, we don't have universal solution in case if we want to handle data with some logic. Also we can't return any meta info from these file-like objects. But there can be important information such as ObjectId in case of using for example mongodb gridfs.

Another solution is to implement a subclass of RequestHandler with stream_request_body decorator and overridden prepare and data_received methods. We can add some extra abstract methods such as open_file, close_file, new_data and pass to them info about current state and chunks which we can write to file and do whatever we want without thinking about boundaries, headers.

What do you think about it?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing the stream_request_body decorator and the prepare and data_received entry points mentioned in the issue. Compare the proposed StreamParser approach with the RequestHandler subclass callbacks, then clarify which architecture should support multipart boundaries, chunk handling, and metadata. Done requires an agreed design and an implementation plan for streaming multipart data without buffering the full body.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.