HangfireIO / HangfireIO/Hangfire
Hangfire Load by waits (AAS)
- Dominant language
- C#
- Stars
- 10.1k
- Forks
- 1.8k
- Avg merge
- 1h 19m
- Merged PRs (30d)
- 1
Description
Firstly - thanks for an awesome library!
We're seeing an issue with Hangfire suddenly (in the last 24 hours) consuming a large amount of our database CPU.
The queries that seem to be taking over are:
```
delete top (1) JQ
output DELETED.Id, DELETED.JobId, DELETED.Queue
from [HangFire].JobQueue JQ with (readpast, updlock, rowlock, forceseek)
where Queue in (@queues1) and (FetchedAt is null or FetchedAt < DATEADD(second, @timeout, GETUTCDATE()))
```
And the very similar:
```
delete top (1) JQ
output DELETED.Id, DELETED.JobId, DELETED.Queue
from [HangFire].JobQueue JQ with (readpast, updlock, rowlock, forceseek)
where Queue in (@queues1,@queues2) and (FetchedAt is null or FetchedAt < DATEADD(second, @timeout, GETUTCDATE()))
```
I've upgraded from Hangfire Core and SQLServer 1.7.15 -> 1.7.24, but the issue remains.
Hangfire is running on a single server, with 4 queues and 121 "workers". Split 1/50/50/20.
Even running a select on the JobQueue table seems to take > 10 seconds. Is there a sensible check we can make to work out what's causing this table to suddenly respond so slowly?
Contributor guide
Research direction
Start with the reported DELETE TOP (1) queries against [HangFire].JobQueue and the reported slow SELECT, comparing behavior between Hangfire 1.7.15 and 1.7.24 under the four-queue, 121-worker setup. Identify the database or workload condition causing the slowdown and document a reproducible check and resolved behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp, sql
- Domain
- backend, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100