HangfireIO / HangfireIO/Hangfire
Scale out jobs
- Dominant language
- C#
- Stars
- 10.1k
- Forks
- 1.8k
- Avg merge
- 1h 19m
- Merged PRs (30d)
- 1
Description
I have a requirement to run scaled out servers when processing recurring cron jobs. The architecture is a SQL Azure database for scheduling, an Azure Web App for the dashboard and many IAAS VM's as the worker machines.
Some jobs should be a single execution across the set of worker machines. This means that the job is only executed by one machine in the farm per recurring time point. I believe this is the default behavior of Hangfire.
Most jobs however should execute once per worker machine per recurring time point. These jobs already support back-end batching (using Azure queues) such that many machine can execute the same job and process different data sets until the data queue is exhausted and then the cron job fires it again later.
How are these scaled out jobs configured in Hangfire?
From what I can tell by the documentation, one way to do this would be to use a queue name per worker machine. This would require the scheduling of the same job type on each worker machine to target a specific queue name for that machine. If this is the recommended approach, does the dashboard support a view of the same job in multiple queues (one for each machine) and manually triggering the job on a specific machine?
Contributor guide
Research direction
No file, test, or entry point is named. Start by reviewing Hangfire's documentation for recurring jobs, queues, worker coordination, and dashboard behavior, using the described SQL Azure, Azure Web App, and VM architecture as context. Done means the supported configuration and whether per-worker triggering and dashboard views are available are clearly established.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure, csharp
- Domain
- backend, cloud, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100