dotnet / dotnet/reactive

IAsyncEnumerable.SelectMany causes multiple IEnumerable enumeration

Open
#1,552 2 comments 5 reactions 0 assignees View on GitHub
[area] Ix
Dominant language
C#
Stars
7.2k
Forks
798
PR merge metrics
No merged PRs in 30d

Description

Hello! I encountered unexpected behavior.

#### Bug

> Which subcomponent library (Ix, Async.Ix)?

__System.Linq.Async__ package

> Which library version?

__5.0.0__

> What are the platform(s), environment(s) and related component version(s)?

I tested it on Windows and `netcoreapp3.1` as the target framework.

> What is the use case or problem?

Sometimes it is needed to join `IAsyncEnumerable>` into plain `IAsyncEnumerable` like `SelectMany` do. See code snippet for details.

> What is the expected outcome?

Each `IEnumerable` of `IAsyncEnumerable` is enumerated at most once for single `IAsyncEnumerable` enumeration. At most is because we can `.Take(n)` it.

> What is the actual outcome?

In some cases `IEnumerable` is enumerated twice.

> Do you have a code snippet or project that reproduces the problem?

```csharp
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;

namespace Snippet
{
class Program
{
static int yieldMethodEnumerationCount = 0;

static async Task Main()
{
IAsyncEnumerable> mixedSequence = new[]
{
Yield("a")
}.ToAsyncEnumerable();

IAsyncEnumerable fullyAsyncSequence = mixedSequence
.SelectMany(subSequence => subSequence.ToAsyncEnumerable());

var list = await fullyAsyncSequence.ToListAsync();

Console.WriteLine($"list = {string.Join(", ", list)}");
Console.WriteLine($"yieldMethodEnumerationCount = {yieldMethodEnumerationCount}");
}

static IEnumerable Yield(string arg)
{
yieldMethodEnumerationCount++;
yield return arg;
}
}
}
```

> What is the expected output of the snippet?

```
list = a
yieldMethodEnumerationCount = 1
```

> What is the actual output of the snippet?

```
list = a
yieldMethodEnumerationCount = 2
```

> What is hypothetical location of the issue?

I think this is happens because of enumeration in [`AsyncEnumerableAdapter.GetCountAsync(...)`](https://github.com/dotnet/reactive/blob/9f2a8090cea4bf931d4ac3ad071f4df147f4df50/Ix.NET/Source/System.Linq.Async/System/Linq/Operators/ToAsyncEnumerable.cs#L104) to get count of the `IEnumerable`.

> Why is this an issue?

- `IAsyncEnumerable.SelectMany` has different semantics than regular `IEnumerable.SelectMany()`
- There are some stateful `IEnumerable`s that cannot be correctly enumerated twice
- Some `IEnumerable`s are computation expensive, so multiple enumeration slows performance down

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.