HangfireIO / HangfireIO/Hangfire
DI error when deleting scheduled jobs from dashboard after upgrade
- Dominant language
- C#
- Stars
- 10.1k
- Forks
- 1.8k
- Avg merge
- 1h 19m
- Merged PRs (30d)
- 1
Description
After I upgraded one of the minor versions (1.7.18 to 1.7.20), I'm getting an error when trying to delete jobs for retry. It's also happening if I try deleting the scheduled jobs. There is one caveat though - the first time during the application lifetime, the delete is successful. Every other time, it fails.
One thing to mention is that I'm not using standard DI for .NET 5 - I'm using DryIoc which is visible from the logs as well.
The error in the logs is the following:
```
at DryIoc.Throw.It(Int32 error, Object arg0, Object arg1, Object arg2, Object arg3) in /_/src/DryIoc/Container.cs:line 13568
at DryIoc.Container.ThrowIfContainerDisposed() in /_/src/DryIoc/Container.cs:line 684
at DryIoc.Container.ResolveAndCache(Int32 serviceTypeHash, Type serviceType, IfUnresolved ifUnresolved) in /_/src/DryIoc/Container.cs:line 418
at DryIoc.Container.DryIoc.IResolver.Resolve(Type serviceType, IfUnresolved ifUnresolved) in /_/src/DryIoc/Container.cs:line 356
at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetService[T](IServiceProvider provider)
at Hangfire.HangfireServiceCollectionExtensions.GetInternalServices(IServiceProvider provider, IBackgroundJobFactory& factory, IBackgroundJobStateChanger& stateChanger, IBackgroundJobPerformer& performer)
at Hangfire.DefaultClientManagerFactory.GetClient(JobStorage storage)
at Hangfire.Dashboard.AspNetCoreDashboardContext.GetBackgroundJobClient()
at Hangfire.Dashboard.RouteCollectionExtensions.<>c__DisplayClass3_0.b__0(DashboardContext context, String jobId)
at Hangfire.Dashboard.BatchCommandDispatcher.Dispatch(DashboardContext context)
at Hangfire.Dashboard.AspNetCoreDashboardMiddleware.Invoke(HttpContext httpContext)
at Microsoft.AspNetCore.Builder.Extensions.MapMiddleware.Invoke(HttpContext context)
at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)",
"message": "code: Error.ContainerIsDisposed;
message: Container is disposed and should not be used: \"container with ambient DryIoc.AsyncExecutionFlowScopeContext with scope {Name=null}
with Rules with {TrackingDisposableTransients, ResolveIEnumerableAsLazyEnumerable, CaptureContainerDisposeStackTrace, IgnoringReuseForFuncWithArgs} and without {ThrowOnRegisteringDisposableTransient, VariantGenericTypesInResolvedCollection}
with DefaultReuse=Singleton {Lifespan=1000}
with FactorySelector=SelectLastRegisteredFactory
with Made={FactoryMethod=ConstructorWithResolvableArguments, PropertiesAndFields=, ParameterSelector=} has been DISPOSED!
Dispose stack-trace at DryIoc.Container.Dispose()
at DryIoc.Microsoft.DependencyInjection.DryIocServiceScope.Dispose()
at Microsoft.AspNetCore.Http.Features.RequestServicesFeature.DisposeAsync()
at Microsoft.AspNetCore.Http.HttpResponse.<>c.<.cctor>b__36_2(Object disposable)
at Microsoft.AspNetCore.Server.IIS.Core.IISHttpContext.FireOnCompleted()
at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
at Microsoft.AspNetCore.Server.IIS.Core.IISHttpContext.FireOnCompleted()
at Microsoft.AspNetCore.Server.IIS.Core.IISHttpContextOfT`1.ProcessRequestAsync()
at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
at Microsoft.AspNetCore.Server.IIS.Core.IISHttpContextOfT`1.ProcessRequestAsync()
at Microsoft.AspNetCore.Server.IIS.Core.IISHttpContext.HandleRequest()
at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
at Microsoft.AspNetCore.Server.IIS.Core.IISHttpContext.HandleRequest()
at Microsoft.AspNetCore.Server.IIS.Core.IISHttpContext.Execute()
at System.Threading.ThreadPoolWorkQueue.Dispatch()
```
I traced the error to the following commits - https://github.com/HangfireIO/Hangfire/commit/293bc41c3715fd48cd6b377f1369a79b2dcbf75f and https://github.com/HangfireIO/Hangfire/commit/07627d4261852381fc980906f48d32af2c281f1e
What I presume is happening is that `serviceProvider` that is injected in the `DefaultClientManagerFactory` will be OK the first time I try to delete one of the jobs. However, it gets disposed after the request is completed. The next time I try to delete the job, the new request is now trying to use the old, disposed `ServiceProvider` and is throwing this error.
Contributor guide
Research direction
Start by reviewing commits 293bc41c3715fd48cd6b377f1369a79b2dcbf75f and 07627d4261852381fc980906f48d32af2c281f1e, then inspect DefaultClientManagerFactory and the dashboard delete flow described in the stack trace. Reproduce repeated retry or scheduled-job deletions with DryIoc after upgrading. Done means later requests no longer attempt to resolve services from the disposed provider.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100