Multipart upload content streamed

Hy!

Most of the clients handle multipart-form uploads as a convenient file upload mechanism. If I want to catch a file upload stream, and write it to S3 on the fly for example, in the current implementations I either need to parse the raw bytestream somehow, to actually stream the data, or I need to write down the file to the filesystem, and upload from there. The second method is an obvious attack surface, and also super slow for large files (multiple fast attackers -> big files -> full hdd -> dos).

We generally upload one file at a time, but with some tinkering, it could work with real multipart too (I don't want this right now, but the sake of completeness I write down my ideas). The API could look like something like

def multipartBodySinglePart(partName: String)(s: Streams[S])(schema: Schema[s.BinaryStream], format: CodecFormat, charset: Option[Charset] = None): StreamBodyIO[s.BinaryStream, s.BinaryStream, S]

which is almost the same as streamBody just seeks into the required part, send it downstream, and when the downstream finishes or the part is finished, seeks to the end of the request.
The extended version could be something like a stream of (PartHeader | PartBytes) and the downstream could build a state machine or other logic to drop/seek the unneeded parts and process the PartBytes with the speed of the real downstream application logic. What we absolutely can't do is a Source[(PartHeader, Source[ByteString])] bcs things like mapAsyncUnordered(4) or buffer(10) would ruin the whole streaming. We need something like a Source[Either[PartHeader, PartBytes]] or something similar (like a parent sealed trait).

Prebuild stuff supported by the server interpreters:

• Akka directive; https://doc.akka.io/docs/akka-http/current/routing-dsl/directives/file-upload-directives/fileUpload.html
• Play method; https://dzone.com/articles/stream-a-file-to-aws-s3-using-akka-streams-in-play
• http4s; quick google didn't helped me out, it seems like it materializes a Part stream, but I never used http4s so no idea if that is really a stream or it is using filesystem caching
• finatra; https://github.com/twitter/finatra/issues/538 no support yet?
• aws; no idea

Also, we should somewhat document the accepted input multipart data format. At this point I would allow a non compile-time checked custom Schema.