Unforced TaskHubWorker.StopAsync() is extremely slow (not working correctly)
- Dominant language
- C#
- Stars
- 1.7k
- Forks
- 335
- Avg merge
- 2d 23h
- Merged PRs (30d)
- 6
Description
The implementation of `TaskHubWorker.StopAsync()` does not appear to be behaving correctly, resulting in extremely slow shutdown sequences.
Here is a minimal repro app:
```csharp
public static void Run()
{
const string HubName = "DurableTaskTestHub";
string servicebusConnectionString = ConfigurationManager.ConnectionStrings["DurableFunctions.ServiceBus"].ConnectionString;
string storageConnectionString = ConfigurationManager.ConnectionStrings["DurableFunctions.Storage"].ConnectionString;
var serviceBusOrchestrationService = new ServiceBusOrchestrationService(
servicebusConnectionString,
HubName,
null,
null,
new ServiceBusOrchestrationServiceSettings());
serviceBusOrchestrationService.CreateIfNotExistsAsync().Await();
Console.WriteLine($"Starting the host.");
var worker = new TaskHubWorker(serviceBusOrchestrationService);
worker.AddTaskOrchestrations(typeof(SimpleOrchestration));
worker.StartAsync().Await();
Console.WriteLine("Stopping host");
Stopwatch sw = Stopwatch.StartNew();
worker.StopAsync().Await();
Console.WriteLine($"Stopped after {sw}.");
Console.WriteLine("Press [ENTER] to exit.");
Console.ReadLine();
}
```
It seems like a bug in the shutdown logic in [WorkItemDispatcher.StopAsync(forced: false)](https://github.com/Azure/durabletask/blob/master/src/DurableTask.Core/WorkItemDispatcher.cs#L163-L168). What I observed while debugging is that `activeFetchers` is always set to 1 even though the system is idle, so we loop and sleep forever until the number of retries expire. The same problem exists when trying to shut down the activity dispatcher.
If I specify `TaskHubWorker.StopAsync(true)`, then the shutdown proceeds quickly.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.