NRE in Kestrel Telemetry EventSources
- Dominant language
- C#
- Stars
- 38.4k
- Forks
- 10.9k
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 281
Description
Mirror of https://github.com/dotnet/runtime/issues/77434
`EventSource.IsEnabled()` may start returning `true` before the `OnEventCommand` completes.
We initialize shared `EventCounter` instances in `OnEventCommand` that we then use from instrumented code paths.
If the `EventSource` is initialized before being enabled, and the instance is accessed from multiple threads, a thread could see `IsEnabled() == true` and then read a `null` `EventCounter` instance.
This race condition only appears once per process.
I believe this is the cause of an exception [YARP hit in CI](https://dev.azure.com/dnceng-public/public/_build/results?buildId=61231&view=ms.vss-test-web.build-test-results-tab&runId=1256708&resultId=100101&paneView=debug):
```
System.NullReferenceException : Object reference not set to an instance of an object.
at System.Net.Http.HttpTelemetry.Http11RequestLeftQueue(Double timeOnQueueMilliseconds)
at System.Net.Http.HttpConnectionPool.GetHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
```
We should fix [these cases](https://grep.app/search?q=%3F%3F%3D%20new%20EventCounter%28&filter[repo.pattern][0]=dotnet).
cc: @Tratcher aspnetcore has 2 such cases as well:
https://github.com/dotnet/aspnetcore/blob/1e8fe7e4ba93935ebc10aee6ef2a4d2b3b0c05d9/src/Middleware/ConcurrencyLimiter/src/ConcurrencyLimiterEventSource.cs#L47
https://github.com/dotnet/aspnetcore/blob/1e8fe7e4ba93935ebc10aee6ef2a4d2b3b0c05d9/src/SignalR/common/Http.Connections/src/Internal/HttpConnectionsEventSource.cs#L46
An internal customer hit something similar:


Contributor guide
Assessment
This issue has not been assessed yet.