HangfireIO / HangfireIO/Hangfire
Job called twice after six hours always.
- Dominant language
- C#
- Stars
- 10.1k
- Forks
- 1.8k
- Avg merge
- 1h 19m
- Merged PRs (30d)
- 1
Description
I have many long running jobs wich should connect to some external websocket servers, and get data from them.
These jobs should work Infinitely, as long as possible, and after error they should be raised again.
And also this jobs should be distributed, and they may be hosted on different servers.
Hangfire allows you to resolve this approach
https://docs.hangfire.io/en/latest/background-processing/running-multiple-server-instances.html
A have 'manager server' wich is responsible for the server where the job will be run on, it also start/stop them.
And i have 'Workers servers'. Every server has unique queue name, so manager server can command to concrete worker server.
Manager server called job for execution:
```
public string Enqueue(string queueName, Expression> expression)
{
var client = new BackgroundJobClient(JobStorage.Current);
var state = new EnqueuedState(queueName);
return client.Create(expression, state);
}
```
And Worker server handled it and started. (I use HostBuilder)
```
internal class WsDistributorService : IHostedService
{
private readonly IServiceProvider _serviceProvider;
private BackgroundJobServer _server;
public WsDistributorService (IServiceProvider serviceProvider)
{
_serviceProvider = serviceProvider;
}
public async Task StartAsync(CancellationToken cancellationToken)
{
GlobalConfiguration.Configuration.UseSqlServerStorage("Connection string to database");
GlobalConfiguration.Configuration.UseActivator(new HangfireActivator(_serviceProvider));
_server = new BackgroundJobServer(new BackgroundJobServerOptions()
{
WorkerCount = 10,
Queues = new[] {"server-unique-queue-name"},
ServerName = "some-server-name"
});
}
public async Task StopAsync(CancellationToken cancellationToken)
{
_server.Dispose();
}
}
```
**The problem is that after 6 hours**, job started twice and then called cancellation token, so job finished.
(I tried many times, and always same behaviour, and always 6 hours)
I'm struggling with this problem, how can i fix it?
Contributor guide
Research direction
Start with WsDistributorService.StartAsync and the BackgroundJobServerOptions configuration shown in the issue, then review how long-running workers, server heartbeats, and cancellation are handled. Reproduce the six-hour behavior with the described manager and worker setup; done means the job does not run concurrently or get cancelled unexpectedly.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp, sql
- Domain
- backend, distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100