dotnet / dotnet/SqlClient

Win32Exception ExecuteDbDataReaderAsync

Open
#4,094 1 comment 0 reactions 1 assignee Claimed by @paulmedynski View on GitHub
Area\Async Performance :chart_with_upwards_trend:
Dominant language
C#
Stars
989
Forks
340
Avg merge
4d 18h
Merged PRs (30d)
69

Description

During a higher load on the API, several Win32Exception are raised that are not due to the database.

```
**Exception message:**
Microsoft.Data.SqlClient.SqlException (0x80131904): Execution Timeout Expired. The timeout period elapsed prior to completion of the operation or the server is not responding.
---> System.ComponentModel.Win32Exception (258): Unknown error 258
at Microsoft.Data.SqlClient.SqlCommand.EndExecuteReaderAsync(IAsyncResult asyncResult)
at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
ClientConnectionId:6076c28d-cfa1-456a-9749-b293d75316d8
Error Number:-2,State:0,Class:11
**Stack trace:**
Microsoft.Data.SqlClient.SqlException:
at System.Threading.Tasks.ContinuationResultTaskFromResultTask`2.InnerInvoke (System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Threading.ExecutionContext.RunInternal (System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Threading.ExecutionContext.RunInternal (System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Threading.Tasks.Task.ExecuteWithThreadLocal (System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1+ConfiguredTaskAwaiter.GetResult (System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at Dapper.SqlMapper+d__34`1.MoveNext (Dapper, Version=2.0.0.0, Culture=neutral, PublicKeyToken=null: /_/Dapper/SqlMapper.Async.cs:489)
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1+ConfiguredTaskAwaiter.GetResult (System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at XX.XX.XX.XX.Infrastructure.XXRepository+d__11.MoveNext (XX.XX.XX.XX.Infrastructure, Version=2.2.0.0, Culture=neutral, PublicKeyToken=null: F:\agent01\_work\17\s\src\Core\src\XX.Infrastructure\XXRepository.cs:73)
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult (System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at XX.XX.XX.XX.Infrastructure.XXRepository+d__13.MoveNext (XX.XX.XX.XX.Infrastructure, Version=2.2.0.0, Culture=neutral, PublicKeyToken=null: F:\agent01\_work\17\s\src\Core\src\XX.Infrastructure\XXRepository.cs:103)
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1+ConfiguredTaskAwaiter.GetResult (System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at XX.XX.XX.XX.Services.Commands.XX.SubmitXXCommandHandler+d__8.MoveNext (XX.XX.XX.XX.Services.Commands, Version=2.2.0.0, Culture=neutral, PublicKeyToken=null: F:\agent01\_work\17\s\src\Core\src\XX.Services.Commands\XX\XXXCommandHandler.cs:142)
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult (System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at XX.XX.XX.XX.WebApp.Controllers.XXController+d__9.MoveNext (XX.XX.XX.XX.Api, Version=2.2.0.0, Culture=neutral, PublicKeyToken=null: F:\agent01\_work\17\s\src\Core\src\XX.Api\Controllers\XXController.cs:183)
Inner exception System.ComponentModel.Win32Exception handled at System.Threading.Tasks.ContinuationResultTaskFromResultTask`2.InnerInvoke:

```

### Further technical details
The stored procedure response time:

Image

Microsoft.Data.SqlClient version: 6.1.4
.NET target: .NET 8.0
SQL Server version: Azure SQL server
Operating system: Azure linux app service

**Additional context**
Actions that we have made trying to fix:

Upgrades:
Dapper: from 2.0.123 to 2.1.35
Microsoft.Data.SqlClient: from 5.1.0 to 6.1.4

Infrastructure scale-out: from 8 instances to 12 instances

Min threads:
When analyzing the memory dump:
![Image](https://github.com/user-attachments/assets/8838da64-35da-4f1d-a4ea-7a26f31e8b1d)

We decided to set the minthread:
`ThreadPool.SetMinThreads(workerThreads: 100, completionPortThreads: 100);`

After setting the minthread to 100 it seems to be fixed. But we think that it's not the right fix to do because the lib is doing sync over async which is causing a thread starvation situation and the fix has to be done there. What do you think?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.