HttpClient distributed tracing tag enrichment

Start with System.Net.Http/DiagnosticsHandler.cs, where HttpClient creates the activity, and review Activity.AddTag in System.Diagnostics.DiagnosticSource. Use the reproduction scenario to examine how enrichment could be scoped per HttpClient or HttpRequestMessage and expose the relevant activity. Done means a practical enrichment API supports the stated configuration and activity-access requirements.

Description

HttpClient creates an Activity for each HTTP request. The activity is used to create a span in OTel distributed tracing.

OTel spans have attributes and HttpClient automatically adds information about the HTTP request to the span. Additional information can be added by calling Activity.AddTag. However, there isn't an easy way to enrich the span with custom information for an HTTP request. Usually the activity is accessed using Activity.Current, an async local, but HttpClient creates the activity in DiaganosticsHandler which is automatically added as the last handler in the pipeline. It's not possible to add your own handler to work with Activity.Current because it's created afterwards.

An alternative approach is to use a diagnostics listener and listen to System.Net.Http.HttpRequestOut.Start. It is called after the activity has been created. The problem with this approach:

1. It's static across the app. Can't localize enrichment to one HttpClient or request.
2. Accessing the HttpRequestMessage in the raised listener callback requires reflection.
3. It's hard and inaccessible.

A scenario where this information is useful is this Aspire UI enhancement: https://github.com/dotnet/aspire/pull/1061

We want to identify the Browser Link HTTP request an ASP.NET Core app makes to Visual Studio. It would be great if we could enrich the activity to have an attribute to identify the call was for Browser Link.

Because that's not possible we were forced to look at the request URL to infer the request is for Browser Link.

Reproduction Steps

Try to enrich Activity for a HttpClient.

Expected behavior

There is an easy-to-use API for enriching the HttpClient. The API:

1. Can be configured per HttpClient, or ideally, per HttpRequestMessage.
2. Provides the Activity for HttpClient. Providing the activity is better than making people use Activity.Current, because there might be a child activity created, and using Activity.Current forces callers to check the name for each activity in the hierarchy until they find the one for the HttpClient.

Maybe something like:

var httpRequest = new HttpRequestMessage("https://contoso.com");
HttpActivityEnrichmentContext.AddCallback(httpRequest, context =>
{
    context.Activity.AddTag("http.is_browser_link", true);
});

Actual behavior

No easy way to enrich activity for HttpClient.

Regression?

No response

Known Workarounds

No response

Configuration

No response

Other information

No response