Azure / Azure/azure-functions-host

Improve Isolated Http dispatch debuggability

Open
#11,457 0 comments 2 reactions 0 assignees View on GitHub
Needs: Triage (Functions)
Dominant language
C#
Stars
2k
Forks
482
Avg merge
2d 10h
Merged PRs (30d)
36

Description

In a recent .NET Isolated CRI https://portal.microsofticm.com/imp/v5/incidents/details/702096432/summary, during the impacted time ranges, logs were showing http requests being dispatched from host, then log silence for 15+ seconds followed by a TimeoutException coming from the **DefaultHttpCoordinator** [here](https://github.com/Azure/azure-functions-dotnet-worker/blob/e99be73f6c1b0d1a59b1788516675a82272667bb/extensions/Worker.Extensions.Http.AspNetCore/src/Coordinator/DefaultHttpCoordinator.cs#L63).

It's unclear what the problem is here since we don't have any more visibility into the GRPC communications for this invocation. The _theory_ was that high load/cpu caused delays, but if possible it'd be good if we can improve logs to make this clear.

In the incident above, I share all the relevant queries.

Contributor guide

Open the contributing guide

Research direction

Start with extensions/Worker.Extensions.Http.AspNetCore/src/Coordinator/DefaultHttpCoordinator.cs at the linked timeout path, then trace the isolated HTTP dispatch and GRPC communication logging around it. Done means the logs make the period between host dispatch and TimeoutException clearer, including enough context to distinguish delayed communication from high load or CPU effects.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure, csharp, grpc
Domain
backend, observability
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.