dotnet / dotnet/orleans

Overly long traces for `IAsyncEnumerable`

Open
#9,716 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
C#
Stars
10.9k
Forks
2.1k
Avg merge
14h 42m
Merged PRs (30d)
354

Description

I'm currently looking at a partial trace from production that was rejected because it grew beyond our configured limits. That particular example shows a runtime of >40 minutes with >9000 individual spans.

After the initial pair of
```
orleans-client IAsyncEnumerableGrainExtension/StartEnumeration (504.6μs)
orleans-server IAsyncEnumerableGrainExtension/StartEnumeration (54.2μs)
```

all other spans are subsequent pairs of
```
orleans-client IAsyncEnumerableGrainExtension/MoveNext (196.39ms)
orleans-server IAsyncEnumerableGrainExtension/MoveNext (196ms)
```

with up to 15s runtime based on the `IAsyncEnumerable` heartbeat.

In other parts of the application, we frequently create new root activities linked to a common trace to split up long running operations. What are our options here?

Contributor guide

Open the contributing guide

Research direction

Start by reading the IAsyncEnumerableGrainExtension/StartEnumeration and MoveNext entry points and tracing how their client and server spans are created. Compare the current span structure with the reported heartbeat-driven sequence and define how long-running enumerations should be split while preserving trace relationships; done requires an agreed design and validation against the described trace limits.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
distributed-systems, observability
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.