microsoft / microsoft/perfview
Aligned reading strategy in StreamReaderWriter causes event latency for EventPipe
Nobody has claimed this yet.
- Dominant language
- C#
- Stars
- 4.7k
- Forks
- 775
- Avg merge
- 5d 11h
- Merged PRs (30d)
- 9
Description
The above code in StreamReaderWriter can cause latency for EventPipe. If you have infrequent events, e.g., EventCounters, and an event block ends up a length not divisible by 8, then the reader will attempt a blocking read that will not return until more data is sent. In the case of infrequent data, this can create artificial latency.
For example, let's say there's an EventCounter session that is turned on with a frequency of 5 seconds. If any of the counter events in that stream end up in an event block that is not divisible by 8, then the reader will read the entire event block and attempt to read the remaining bytes to get to a mod(8) value. This causes EventPipeEventSource to not realize the event until the next counter shows up. This artificially adds 5 seconds of latency to potentially the entire stream depending on how frequently event blocks end up indivisible by 8.
I found this issue when triaging dotnet/diagnostics#2044. A bandaid fix is to turn on CPU sampling which will send a constant data stream guaranteeing that there will be more data to meet the reader's aligned reading requirements.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the aligned-reading code in src/FastSerialization/StreamReaderWriter.cs at lines 791-814, then trace how EventPipeEventSource consumes the stream. Reproduce the behavior with infrequent EventCounters and an event block whose length is not divisible by 8. Done means the reader no longer adds artificial latency while preserving correct stream processing.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp
- Domain
- performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100