DurableTask should be resilient to Azure Blob storage hangs

On our production servers, we have caught an issue in Durable Task framework.

Main issue (how the issue manifests): the call to StopAsync method on TaskHubWorker hangs forever.

I did some deep investigations to the logs and found the original cause. Some facts from logs:
1. The awaiting of "takerTask" in LeaseCollectionBalancer.StopAsync method hangs. This happens because:
2. DownloadLeaseBlob method in BlobLeaseManager.cs hangs.

The latter fact I understood from the logs. We have a logging for all outcoming http requests. During the first 2 days after the service start, every 5 seconds we had the following logs:

- 4 GET requests (to download blob)
- 4 HEAD requests (for fetch attributes)

At some point, the logging stopped (while service was still active for a few days). The last logs contains:

- 4 GET requests
- 3 HEAD requests


All the evidence shows that the hang happens in Azure Blob client. DurableTask SDK should be resilient to it. E.g. it should have a default timeout for all blob operations.

Inizia tracciando TaskHubWorker.StopAsync attraverso LeaseCollectionBalancer.StopAsync e l’attesa di takerTask. Poi esamina DownloadLeaseBlob in BlobLeaseManager.cs e le relative operazioni GET e HEAD. Il lavoro è completato quando un’operazione Azure Blob bloccata non fa più rimanere StopAsync in attesa per sempre, e il comportamento è coperto da un test appropriato.

Scritto dal modello di indicizzazione a partire dal testo della issue.