dotnet / dotnet/aspnetcore

asp.net core api Request.Form URL-encoded Chinese is not restored correctly.

Open
#33,866 11 comments 0 reactions 0 assignees View on GitHub
area-networking bug help wanted
Dominant language
C#
Stars
38.4k
Forks
10.9k
Avg merge
2d 10h
Merged PRs (30d)
281

Description

# Environment description
- .net 5
- asp.net core api
- windows 10、centos 7
```
dotnet --info
.NET SDK (反映任何 global.json):
Version: 5.0.300
Commit: 2e0c8c940e

运行时环境:
OS Name: Windows
OS Version: 10.0.19042
OS Platform: Windows
RID: win10-x64
Base Path: C:\Program Files\dotnet\sdk\5.0.300\

Host (useful for support):
Version: 5.0.6
Commit: 478b2f8c0e

.NET SDKs installed:
5.0.300 [C:\Program Files\dotnet\sdk]

.NET runtimes installed:
Microsoft.AspNetCore.All 2.1.28 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
Microsoft.AspNetCore.App 2.1.28 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 3.1.15 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 5.0.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.NETCore.App 2.1.28 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 3.1.15 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 5.0.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.WindowsDesktop.App 3.1.15 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 5.0.6 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

To install additional .NET runtimes or SDKs:
https://aka.ms/dotnet-download
```

# Problem Description
There is a client which uses GB2312 character set encoding and uses https POST `Content-Type:application/x-www-form-urlencoded; charset=GBK` to call the API provided by me.
The data submitted by the client POST is encoded by `System.Web.HttpUtility.UrlEncode("查重费用", Encoding.GetEncoding("gbk"));` The encoded value is: `%b2%e9%d6%d8%b7%d1%d3%c3`.

The entire request message looks like this:
```
POST https://localhost:44334/api/test HTTP/1.1
User-Agent:Fiddler Everywhere
Content-Type:application/x-www-form-urlencoded; charset=GBK
Host:localhost:44334
Content-Length:27

su=%b2%e9%d6%d8%b7%d1%d3%c3
```

The value received by my API is indeed like this: `%B2%E9%D6ط%D1%D3%C3`.

The received code is like this:
```
// Program.cs
public class Program
{
public static void Main(string[] args)
{
// The point of displaying this code is to tell you that the character set is registered.
Encoding.RegisterProvider(CodePagesEncodingProvider.Instance);

CreateHostBuilder(args).Build().Run();
}
... omit other codes ...
}

// TestController.cs
[Route("api/[controller]")]
[ApiController]
public class TestController : ControllerBase
{
[HttpPost]
public IActionResult Post()
{
// Here the correct value of Request.Form["su"] should be "查重费用", but it is actually "%B2%E9%D6ط%D1%D3%C3"
return Content(Request.Form["su"]);
}
}
```

I see asp.net core source code [FormPipeReader.cs](https://github.com/dotnet/aspnetcore/blob/release/5.0/src/Http/WebUtilities/src/FormPipeReader.cs)
```
private string GetDecodedString(ReadOnlySpan readOnlySpan)
{
if (readOnlySpan.Length == 0)
{
return string.Empty;
}
else if (_encoding == Encoding.UTF8 || _encoding == Encoding.ASCII)
{
... omit other codes ...
}
else
{
// Slow path for Unicode and other encodings.
// Just do raw string replacement.
var decodedString = _encoding.GetString(readOnlySpan);
decodedString = decodedString.Replace('+', ' ');
return Uri.UnescapeDataString(decodedString);
}
}
```

If replace `Uri.UnescapeDataString(decodedString)` with `System.Web.HttpUtility.UrlDecode(decodedString, _encoding)`, the problem I encountered can be solved.

Is it really correct to use `Uri.UnescapeDataString(string)` here?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.