distributedcache using SQL - Execution Timeout Expired.
- Dominant language
- PowerShell
- Stars
- 22k
- Forks
- 4.9k
- Avg merge
- 5d 5h
- Merged PRs (30d)
- 29
Description
We have been using SQL server distributed cache for few days and it does regularly stop working generating the exception reported at the bottom of this issue description.
Which is produced by the simple `SetAsync` call:
```
await SetAsync(q.id, bytes,
new DistributedCacheEntryOptions()
.SetAbsoluteExpiration(TimeSpan.FromMinutes(120))
);
```
I have logged the size of the bytes being saved and are in the range 30/50 Mbytes.
The server is lab machine I am currently using for this purpose only and network and server load are out of question. It's in the same rack, same switch, and has no other load.
I switched to ncache installed on the same machine to check if the problem was the sql or not, and with ncache I have no issue and the SetAsync takes no longer than a couple of seconds. I then switched to a different SQL server machine, and after a little while I started getting back the exceptions.
I would have liked to try extending the connection time out to check if it's taking much much longer than expected or just crashing/looping/something, but I can't figure out where command timeout can be set in `services.AddDistributedSqlServerCache`.
I don't know how to investigate that further.
And since it's working fine with ncache without changing anything in the code, and it's not working with either a lab SQL or a production one, I am lead to think there's something wrong in the SQL distributed cache.
Can please anyone help?
> Execution Timeout Expired. The timeout period elapsed prior to completion of the operation or the server is not responding.
>
> at Microsoft.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
> at Microsoft.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
> at Microsoft.Data.SqlClient.SqlCommand.InternalEndExecuteNonQuery(IAsyncResult asyncResult, Boolean isInternal, String endMethod)
> at Microsoft.Data.SqlClient.SqlCommand.EndExecuteNonQueryInternal(IAsyncResult asyncResult)
> at Microsoft.Data.SqlClient.SqlCommand.EndExecuteNonQueryAsync(IAsyncResult asyncResult)
> at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
> --- End of stack trace from previous location where exception was thrown ---
> at Microsoft.Extensions.Caching.SqlServer.DatabaseOperations.SetCacheItemAsync(String key, Byte[] value, DistributedCacheEntryOptions options, CancellationToken token)
> at Microsoft.Extensions.Caching.SqlServer.SqlServerCache.SetAsync(String key, Byte[] value, DistributedCacheEntryOptions options, CancellationToken token)
> at umusa.DisposableCache.SetAsync(String key, Byte[] value, DistributedCacheEntryOptions options, CancellationToken token) in C:\websites-repo\gommecore\umusa\_Code\DisposableCache.cs:line 51
> at umusa.DisposableCache.Refresh(List`1 queue, gommeautoContext _db) in C:\websites-repo\gommecore\umusa\_Code\DisposableCache.cs:line 333
> at feeds.Controllers.importerController.AttemptToDoJob(FeedsUtility fu, FeedStep feed, Nullable`1 distributorid, Boolean forcestart) in C:\websites-repo\gommecore\feeds\Controllers\importerController.cs:line 540
> at feeds.Controllers.importerController.Trigger(String nonce, Int32 distributorid, String destination, String path, Boolean fromlocalfile, Boolean test, Boolean forcestart) in C:\websites-repo\gommecore\feeds\Controllers\importerController.cs:line 244
> at Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.TaskOfIActionResultExecutor.Execute(IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments)
> at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeActionMethodAsync>g__Awaited|12_0(ControllerActionInvoker invoker, ValueTask`1 actionResultValueTask)
> at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeNextActionFilterAsync>g__Awaited|10_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
> at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context)
> at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
> at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeInnerFilterAsync>g__Awaited|13_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
> at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeNextResourceFilter>g__Awaited|24_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
> at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResourceExecutedContextSealed context)
> at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
> at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeFilterPipelineAsync>g__Awaited|19_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
> at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
> at Microsoft.AspNetCore.Builder.RouterMiddleware.Invoke(HttpContext httpContext)
> at Microsoft.AspNetCore.Localization.RequestLocalizationMiddleware.Invoke(HttpContext context)
> at Microsoft.AspNetCore.Session.SessionMiddleware.Invoke(HttpContext context)
> at Microsoft.AspNetCore.Session.SessionMiddleware.Invoke(HttpContext context)
> at umusa.SerilogMiddleware.Invoke(HttpContext httpContext, IHostingEnvironment _env, IConfiguration _configuration, gommeautoContext dbcontext) in C:\websites-repo\gommecore\umusa\Serilog\CustomEnricher.cs:line 119
>
Contributor guide
Assessment
This issue has not been assessed yet.