dotnet / dotnet/aspnetcore

JsonSerializer byte[] to stream nested base64 serialization

Open
#49,868 8 comments 0 reactions 0 assignees View on GitHub
area-mvc
Dominant language
C#
Stars
38.4k
Forks
10.9k
Avg merge
2d 6h
Merged PRs (30d)
290

Description

### Description

I have a MVC app in which I am using action filters - I have observed that the byte[] result of a controller method is serialized twice (base64 inside base64) using JsonSerializer overload (introduced in .net 6.0) which serializes byte[] and passes it to a stream. Please see a short NUnit test example below (I am using non-async method in my unit tests while MVC is using async one, but this issue seem to be applicable to all overloads of JsonSerializer which handle serialization from byte[] to stream) - the unit tests are replicable on unit test projects using .net 6.0+ (including 8.0 preview).

- Confirm_no_double_serializing_NewtonsoftJson - pass - example how NewtonsoftJson serializes byte[] data - the output is the same base64 string as input
- Confirm_no_double_serializing_JsonSerializer_non_stream - pass - example how JsonSerializer serializes byte[] data - the output is the same base64 string as input
- Confirm_no_double_serializing_JsonSerializer_stream - fail- example how JsonSerializer serializes byte[] data when asked to copy it directly to a stream rather than returning raw serialization output (this version is used by MVC) - the output is a base64 string encoded twice - it matches input only after double decoding

### Reproduction Steps

```
using Newtonsoft.Json;
using System;
using System.IO;
using System.Text.Json;
using NUnit.Framework;

namespace Tests.Tests
{
public class Tests
{

[Test]
public void Confirm_no_double_serializing_NewtonsoftJson()
{
var input = "dGVzdA==";
byte[] byteObj = Convert.FromBase64String(input);

var output = JsonConvert.SerializeObject(byteObj);

var areEqual = output.Contains(input);
Assert.IsTrue(areEqual);
}

[Test]
public void Confirm_no_double_serializing_JsonSerializer_non_stream()
{
var input = "dGVzdA==";
byte[] byteObj = Convert.FromBase64String(input);

var output = System.Text.Json.JsonSerializer.Serialize(byteObj, typeof(byte[]));

var areEqual = output.Contains(input);
Assert.IsTrue(areEqual);
}

[Test]
public void Confirm_no_double_serializing_JsonSerializer_stream()
{
var input = "dGVzdA==";
byte[] byteObj = Convert.FromBase64String(input);

MemoryStream stream = new MemoryStream();
JsonSerializerOptions SerializerOptions = new JsonSerializerOptions();
System.Text.Json.JsonSerializer.Serialize(stream, byteObj, SerializerOptions);
var output = Convert.ToBase64String(stream.ToArray());

var areEqual = output.Contains(input);
Assert.IsTrue(areEqual);
}
}
}

```
### Expected behavior

Input is a base64 string "dGVzdA==" converted to byte[] which is then serialized
Deserialized output of above serialization should also be the same base64 string ("dGVzdA==")

### Actual behavior

Input is a base64 string "dGVzdA==" converted to byte[] which is then serialized
Deserialized output of above serialization is "ImRHVnpkQT09Ig==" (which is an encoded form of our input ("dGVzdA=="))

### Regression?

The issue does not exist in .NET 5.0 and below as this version does not contain JsonSerializer overloads for serializing and copying to a stream.

### Known Workarounds

_No response_

### Configuration

Which version of .NET is the code running on? 8.0 SDK but is applicable to all .NET 6.0+
What OS and version, and what distro if applicable? Windows 10
What is the architecture (x64, x86, ARM, ARM64)? Windows Laptop - x64
Do you know whether it is specific to that configuration? It is specific to .NET 6.0+ frameworks

### Other information

Nunit tests have been provided for an easy reproduction.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.