JsonSerializer.DeserializeAsyncEnumerable does not dispose the stream

Open
#119,151 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
38/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Stale
Tech stack
csharp

Research direction

Start with JsonSerializer.DeserializeAsyncEnumerableCore and its local CreateAsyncEnumerableFromArray function, then run the supplied reproduction to confirm the stream remains undisposed. Determine and document the intended configuration or default behavior, and consider the work complete when the chosen behavior is implemented and verified against the reproduction.

Written by the indexing model from the issue text.

Description

api-suggestion area-System.Text.Json
Description

The set of overloads for JsonSerializer.DeserializeAsyncEnumerable do not dispose the provided Stream, nor provide a way of configuring this. This should be classed as a bug (or at least very unexpected), as all other uses of IEnumerable and IAsyncEnumerable do dispose the inner collection or stream. The only example I could find that doesn't do this by default is XmlSerializer, although that is configurable, and StreamReader and many other classes that take a Stream offer a bool leaveOpen option as well, with false the default.

Reproduction Steps

The following code reproduces the issue, with a custom stream class that logs whether Dispose was called.

async Task Main()
{
    await foreach (var i in GetStrings())
        Console.WriteLine(i);
}

IAsyncEnumerable<string> GetStrings()
{
    var s = new MyStream(@"[""a"",""b""]"u8.ToArray());
    return JsonSerializer.DeserializeAsyncEnumerable<string>(s);
}

class MyStream : MemoryStream
{
    public MyStream(byte[] array) : base(array)
    {
    }
    
    protected override void Dispose(bool disposing)
    {
        Console.WriteLine("Disposed");
        base.Dispose(disposing);
    }

    public override ValueTask DisposeAsync()
    {
        Console.WriteLine("Dispose async");
        return base.DisposeAsync();
    }
}
Expected behavior

"Disposed" should be logged to console

Actual behavior

It wasn't logged.

Regression?

No response

Known Workarounds

This significantly complicates passing along an IAsyncEnumerable, it means you need to write a separate iterator function to safely dispose the stream, with all the extra compiler magic to make that happen.

async IAsyncEnumerable<string> GetStrings2()
{
    using var s = new MyStream(@"[""a"",""b""]"u8.ToArray());
    await foreach (var str in JsonSerializer.DeserializeAsyncEnumerable<string>(s))
        yield return str;
}
Configuration

.NET 9 x64

Other information

Inside JsonSerializer.DeserializeAsyncEnumerableCore there is a local function CreateAsyncEnumerableFromArray, which finishes with:

                }
                finally
                {
                    bufferState.Dispose();
                }

The finally just needs an extra line added

                    await utf8Json.DisposeAsync();

Whether this feature should be available only via an extra bool parameter, to keep the existing behaviour, is an open question. Arguably at least the default should be to dispose, because JsonSerializer eagerly reads beyond the end of the JSON, so makes little sense to keep it open any more.

Dominant language
C#
Stars
18.3k
Forks
5.6k
PR merge metrics
PR metrics pending

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from dotnet/runtime

All issues in dotnet/runtime

Similar issues

More C# issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.