High CPU Usage , High latency

We use Hangfire 1.8.5. We have 3 physical servers, separate for sql server, Hangfire dashboard and windows service for hangfire background server.

Windows service server has 12 Processor and we set 36 background workers.

We have 30 recurring jobs each having different interval. Each recurring job enqueues few more jobs.

It works normally when start (or restart) the service when 36 or more jobs in the queue dashboard shows 36 jobs in “processing” and latency for each job is also very low.

But after couple of hours latency keeps increasing and very less number of jobs are in “processing” while it still shows 36 active workers. And cpu usage goes high.

No source file, test, or entry point is identified. Start by reproducing the reported behavior with Hangfire 1.8.5 using the described three-server setup, 36 workers, and recurring jobs, then inspect how active workers, processing counts, latency, and CPU usage change over several hours. Done means identifying a reproducible cause and documenting or fixing the behavior.

Written by the indexing model from the issue text.