dotnet / dotnet/aspnetcore

Reading multipart/form-data request stream => file upload issues

Open
#52,321 6 comments 3 reactions 0 assignees View on GitHub
area-networking
Dominant language
C#
Stars
38.4k
Forks
10.9k
Avg merge
2d 6h
Merged PRs (30d)
290

Description

### Is there an existing issue for this?

- [X] I have searched the existing issues

### Describe the bug

This issue has to do with processing the request body stream and uploading files to Azure Blob Storage (or any storage which supports streams).

I found the following issue https://github.com/dotnet/aspnetcore/issues/48958 which originally came from this [Reddit post](https://www.reddit.com/r/dotnet/comments/14dnn5v/comment/jp0iwxk/?utm_source=share&utm_medium=web2x&context=3). This basically describes what I want to do. I want to stream uploads directly to Azure Blob Storage. I have added a [resourcefilter which disables form model binding](https://github.com/saasen/aspnetcore-stream-fileupload/blob/main/Web/DisableFormValueModelBindingAttribute.cs).

The code is almost identical to the code on the [documentation which documents file upload streaming](https://github.com/dotnet/AspNetCore.Docs/blob/main/aspnetcore/mvc/models/file-uploads/samples/3.x/SampleApp/Controllers/StreamingController.cs) which uses `multipart/form-data`.

The issue I am seeing has to do with processing the request stream. I want to verify that the file which is uploaded is actually the file type it says it is. I'm doing this by reading the first few bytes and checking it against known file signatures (magic bytes).

This works as intended, but it is when uploading the file itself to Azure it doesn't work as I want it to. If I first read the bytes to determine if the file is valid, part of the request stream is read. When calling `UploadAsync(section.Body)` the file becomes corrupt, as it skips the first bytes because they were already read. If I try to reset the position of the request stream, that won't work because the `HttpRequestStream` is not seekable. It is possible, however, to [enable re-reading the request stream by calling `EnableBuffering` on the request in a middleware because MVC has started](https://devblogs.microsoft.com/dotnet/re-reading-asp-net-core-request-bodies-with-enablebuffering/). When I do this, the `HttpRequestStream` is indeed seekable, but if I reset the position of the multipart section I'm currently reading after validating the magic bytes, part of the stream is already read (x amount of bytes to determine if the file is valid). It seems that when calling `UploadAsync(stream)` on the blob client, and the `stream.Length` is known, it doesn't try to read the entire stream when uploading to Blob Storage. That results in a file that only contains the magic bytes I previously read, not everything else.

I'm probably doing something wrong. Do I need to copy the request stream to a seperate stream before reading the magic bytes? I guess the original stream will be read regardless of what I do here.

### Expected Behavior

I would expect that it would be possible to read the magic bytes of the multipart section stream, and then upload the entire section to Blob Storage.

### Steps To Reproduce

I added a small isolated re-pro of the issue I am having: https://github.com/saasen/aspnetcore-stream-fileupload.

### Exceptions (if any)

_No response_

### .NET Version

6.0.417

### Anything else?

I'm on Apple MacBook Pro M2 using Rider.

`dotnet --info` output:

```
.NET SDK (reflecting any global.json):
Version: 6.0.417
Commit: 04fde1d2e3

Runtime Environment:
OS Name: Mac OS X
OS Version: 14.1
OS Platform: Darwin
RID: osx-arm64
Base Path: /usr/local/share/dotnet/sdk/6.0.417/

Host:
Version: 8.0.0
Architecture: arm64
Commit: 5535e31a71

.NET SDKs installed:
6.0.411 [/usr/local/share/dotnet/sdk]
6.0.417 [/usr/local/share/dotnet/sdk]
7.0.305 [/usr/local/share/dotnet/sdk]
8.0.100 [/usr/local/share/dotnet/sdk]

.NET runtimes installed:
Microsoft.AspNetCore.App 6.0.19 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 6.0.25 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 7.0.8 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 8.0.0 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.NETCore.App 6.0.19 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
Microsoft.NETCore.App 6.0.25 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
Microsoft.NETCore.App 7.0.8 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
Microsoft.NETCore.App 8.0.0 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]

Other architectures found:
None

Environment variables:
Not set

global.json file:
/Users//code/saasen/aspnetcore-stream-fileupload/global.json

Learn more:
https://aka.ms/dotnet/info

Download .NET:
https://aka.ms/dotnet/download
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.