Azure / Azure/azure-functions-host
Restart unresponsive workers during WorkerStatus heartbeat request
- Dominant language
- C#
- Stars
- 2k
- Forks
- 482
- Avg merge
- 2d 12h
- Merged PRs (30d)
- 38
Description
The [HostPerformanceManager](https://github.com/Azure/azure-functions-host/blob/dev/src/WebJobs.Script/Scale/HostPerformanceManager.cs#L115) starts a timer which periodically makes a `GetWorkerStatusAsync` request. The worker status here is used to determine if workers are overloaded and if we need to start a new worker channel. This is determined through latency, if the worker is taking too long to process grpc requests then it's overloaded/unhealthy.
We want to add to this feature by restarting unresponsive workers, currently we might only be able to do this to multi-threaded workers (dotnet and java)
Contributor guide
Research direction
Start with src/WebJobs.Script/Scale/HostPerformanceManager.cs around the timer and its GetWorkerStatusAsync request. Trace how latency identifies overloaded or unhealthy workers and how new worker channels are started. Done means defining and implementing the restart behavior for unresponsive workers, initially considering multi-threaded .NET and Java workers; the issue does not name tests or a more specific entry point.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp
- Domain
- backend, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100