dotnet / dotnet/extensions

API Proposal: Support for OpenTelemetry Events

Open
#5,891 13 comments 2 reactions 1 assignee Claimed by @samsp-msft View on GitHub
api-suggestion area-telemetry
Dominant language
C#
Stars
3.2k
Forks
894
Avg merge
1d 12h
Merged PRs (30d)
23

Description

[Issue writing in progress]

# Add support for OpenTelemetry Events

[Events](https://opentelemetry.io/docs/specs/semconv/general/events/) is a newish spec from OpenTelemetry to describe events. The goal is to replace the events on Spans, and be able to use them for client-side events (such as a tracking button clicks), gen_ai telemetry etc.

Events are a specialization of log messages, and have a different schema:
- Events all have a name
- Events don't have a format string / message
- ~Event data should be structured data as part of the `Body` which should be transmitted on the wire using AnyValue.~

AnyValue is a distributed union type defined in the gRPC protos for OTLP. It has a lot of commonality with what can be used in JSON - the main difference is more options for numbers.

## How Events should be exposed to .NET

- There should be new extensions to ILogger to emit events
- Should be able to supply the body as types wrapable in an AnyValue
- bool, int, long, double, string, byte[]
- array of supported types
- dictionary
- AnyValue
- JSON?

## Proposals

### Extension to ILogger for Event API

```
public static void LogEvent(this ILogger logger, string eventName, int eventId=0, LogLevel logLevel = LogLevel.Information, IDictionary tags = null)
```

This can be used for OTel Events or other scenarios that require a bit more control over the data format that is emitted.

It will need to have a marker in the TState (or a custom TState implementation) so that existing logging providers can decide what to do with the data.

The format provider should output all the properties as JSON.

The existing logging providers should be updated to have specialized behavior based on the TState marker:
- JSON console provider should ignore the format provider and just serialize the attributes as it does today
- Other console formatters should use the format provider as normal and get JSON output

The primary scenario for using this API should be new scenarios, so it should not be a break for existing applications.

_Location_: TBD

Arguments can be made for it belonging in:
- `Microsoft.Extensions.Logging` - the API does not have dependencies on OTel, and is just providing an alternative set of fields for logging
- A new `Microsoft.Extensions.*` nuget package, such as `Microsoft.Extensions.OpenTelemetry` or `Microsoft.Extensions.Logging.Events`

For those libraries it could be argued as to whether they belong in the `dotnet/runtime` repo or `dotnet/extensions`. The advantage of the latter is that it can ship more regularly.

---

### Strongly typed Event method generation update - Parameter name mapping

```
log.LogCustomEvent("mary had a little lamb", 4);

...
static partial class MyLogHelpers
{
[LoggerMessage(EventId = 10, EventName ="My.custom.event", Level=LogLevel.Information, Message ="This is a custom message {foo} {bar}")]
public static partial void LogCustomEvent(this ILogger log, string foo, int bar);
}

```

Logging since .NET 7 has supported the `LoggerMessage` attribute to create optimized logging methods to write standard log messages. Because the parameters to this are already optional, it can be used for OTel Events without requiring changes.

_Issue_: If `Body` comes back for Event definition, then it should be added as an option to the `LoggerMessageAttribute` as another optional parameter.

One of the main problems with this API is that the name of attributes in the resulting log message will be taken from the names of the parameters to the method (eg `foo` and `bar` in the above example). This limits the names that can be used, especially for OTel which commonly requires "a.b.c" format names for its semantic conventions. If .NET customers are using this API to implement OTel standard events, then they need to be able to map the parameters to the method to potentially different names in the resulting OTel output.

We can solve this problem in a similar way to how the R9 extension handles object parameters - by enabling attributes on the method parameters:

```
[LoggerMessage(EventId = 10, EventName ="My.custom.event", Level=LogLevel.Information, Message ="This is a custom message {foo} {bar}")]
public static partial void LogCustomEvent(this ILogger log, [MapName("custom.thing.foo")] string foo, int bar);
```

In the above example, the `foo` parameter will be mapped to `custom.thing.foo` in the output, `bar` will be mapped as-is. This enables rich parameter control which is useful for Events and any other usage of the `LoggerMessage` attribute. If the name is remapped this way then it should be possible to use the remapped name as part of the Formatted message if desired, this gives developers more flexibility in how they name the parameters and have full control over the output format.

---

### AnyValue Definition

We could potentially defer the AnyValue support to the OTel library and have a type checker on what is passed in, but I think we should probably embrace AnyValue and enable its use in other places such as tags on Activity, that according to OTLP can have AnyValue members.

AnyValue should be added as a class to a `Microsoft.Extensions.*` package. It is probably better in the Extensions repo rather than runtime, as there is more schedule flexibility. This will then enable applications to use AnyValue typed arguments in logging, activity/spans, and event tags. The existing APIs all accept object parameters, which means that applications can pass in AnyValue.

Having the explicit type will make it much easier for applications to ensure that they have formatted complex data in a way that OTel and other exporters can understand.

We should use the affinity to JSON to override the `AnyValue.ToString()` functionality to return JSON formatted data. In most logging providers, they will use the `ToString()` when they do not understand the type. This will allow applications to use AnyValue with legacy providers and still have a good experience.

#### IAnyValue Interface

To facilitate easier passing of complex objects into APIs such as `Activity.Tags` or as tags on log messages, we should include a conversion interface that custom classes can implement. This is a marker that the type can be converted into an AnyValue and that its values can be translated to JSON.

```
interface IAnyValue
{
AnyValue ToAnyValue();
string ToJson();
}
```

The implementer of the interface is responsible for the work. It will typically involve creating a Dictionary populated with the data from the object.

The advantage of using this approach is that the actual conversion to the AnyValue structure can be delayed to the last minute, so if the telemetry is not emitted, the conversion work can be skipped,

---

### ILogger TState

ILogger uses a struct `FormattedLogValues` to store the log message data. As events do not have a message, it's not always clear what should happen for the format provider function. We should probably default to one that will emit the Event data as a JSON blob - that way it will emit all of the data even with legacy log output providers etc. However, where those providers also emit each value, that becomes duplicative and expensive.
We need a marker that the logging provider can use to know what to do. Today the special name `{OriginalFormat}` is used to mark the format string, but that can be a bit brittle (issues with the `{}` being invalid in data sent to kusto for example). We should have a better way to know what to do.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.