HangfireIO / HangfireIO/Hangfire

Object disposed exception while running unit tests when accessing ServiceProvider in filter.

Open
#2,074 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
C#
Stars
10.1k
Forks
1.8k
Avg merge
1h 19m
Merged PRs (30d)
1

Description

I am running ASP.NET Core unit tests. Test class implements IDisposable interface and disposes `WebApplicationFactory`.

Unfortunately, I am getting exception in my HF's filter which tries to create a new scope for services.
It looks like HF server is not stopped properly when Host is stopping.

Here's how I register HF and filters:
```
services.AddHangfire((provider, configuration) =>
{
if (_configuration.GetValue("HangfireConfiguration:InMemory"))
{
configuration.UseMemoryStorage();
}
else
{
configuration.UsePostgreSqlStorage(_configuration.GetConnectionString("OrderFulfillment"));
}
configuration.UseFilter(
new HangfireExceptionLoggingFilter(
provider.GetRequiredService>()));
configuration.UseFilter(
new OutboxableJobsFilter(provider));
});
```

and here's the code of the filter:
```
public sealed class OutboxableJobsFilter : IServerFilter
where TDbContext : DbContext
{
private readonly IServiceProvider _provider;

public OutboxableJobsFilter(IServiceProvider provider)
{
_provider = provider.NotNull();
}

public void OnPerformed(PerformedContext context)
{
var isNotOutboxable =
!context.BackgroundJob.Job.Type.IsAssignableTo(typeof(IOutboxableJob))
&& context.BackgroundJob.Job.Type.GetCustomAttribute() is null
&& context.BackgroundJob.Job.Method.GetCustomAttribute() is null;
if (isNotOutboxable)
{
return;
}

if (context.Canceled || context.Exception is not null)
{
return;
}

TryToRemoveJobFromOutboxedJobsList(context);
}

///
/// Tries to perform a cleanup action. If something fails during this operation,
/// exception will be logged and swallowed. At the moment of running this operation job
/// has completed successfully, and it is better to avoid retrials.
///
private void TryToRemoveJobFromOutboxedJobsList(PerformedContext context)
{
try
{
using var scope = _provider.CreateScope();
var dbContext = scope.ServiceProvider.GetRequiredService();
var jobId = context.BackgroundJob.Id;
var outboxedJob = new OutboxedJob(jobId);
dbContext.Set().Remove(outboxedJob);
// This will throw, if there's no outbox job, but it indicates some bug.
// We should not be here then.
dbContext.SaveChanges();
}
catch (Exception ex)
{
var logger = _provider.GetRequiredService>>();
logger.LogError(ex, "Error when removing job from the list of Outboxed Jobs.");
}
}

public void OnPerforming(PerformingContext context)
{
var isNotOutboxable =
!context.BackgroundJob.Job.Type.IsAssignableTo(typeof(IOutboxableJob))
&& context.BackgroundJob.Job.Type.GetCustomAttribute() is null
&& context.BackgroundJob.Job.Method.GetCustomAttribute() is null;
if (isNotOutboxable)
{
return;
}

if (context.Canceled)
{
return;
}

CancelExecutionIfOutboxedJobsListDoesNotContainJob(context);
}

private void CancelExecutionIfOutboxedJobsListDoesNotContainJob(PerformingContext context)
{
var scope = _provider.CreateScope();
var dbContext = scope.ServiceProvider.GetRequiredService();
var jobId = context.BackgroundJob.Id;
var outboxedJobExists = dbContext.Set()
.Any(x => x.JobId == jobId);

if (!outboxedJobExists)
{
// The execution of outbox job is meaningful only in case of successful transaction
// which lead to scheduling the job.
// When transaction fails, there won't be any record, so it indicates, that job must me canceled.
context.Canceled = true;
}
}
}
```
This filter is just a trick for implementing extended variant of Transactional Outbox pattern.

We've discovered the issue when we wrapped the incoming API call into HF job for async processing.
We had to introduce a delay of 30 seconds in each API test, so everything is properly cleaned up.
I played with polling intervals, but they have no effect within this tame range.

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the WebApplicationFactory disposal scenario with the provided OutboxableJobsFilter, focusing on CreateScope calls during host shutdown. Inspect Hangfire server and host-stop lifecycle entry points, then verify that API tests can dispose cleanly without a 30-second delay or ObjectDisposedException.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
backend, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.