Allow controlling JSON formatting for gen_ai input/output message telemetry
- Dominant language
- C#
- Stars
- 3.2k
- Forks
- 894
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 23
Description
## Summary
`OpenTelemetryChatClient` and related MEAI OpenTelemetry paths currently serialize `gen_ai.input.messages` / `gen_ai.output.messages` as indented JSON, and there does not appear to be a public knob to emit compact JSON instead.
For large prompts, responses, tool calls, tool results, or multimodal content, the extra whitespace increases span attribute size and can make exporter/backend size limits easier to hit.
## Current behavior
The chat path uses `OtelMessageSerializer.DefaultOptions`, which is created by copying `OtelContext.Default.Options`:
```csharp
private static JsonSerializerOptions CreateDefaultOptions()
{
JsonSerializerOptions options = new(OtelContext.Default.Options)
{
Encoder = JavaScriptEncoder.UnsafeRelaxedJsonEscaping
};
options.TypeInfoResolverChain.Add(AIJsonUtilities.DefaultOptions.TypeInfoResolver!);
options.MakeReadOnly();
return options;
}
```
`OtelContext` is source-generated with `WriteIndented = true`:
```csharp
[JsonSourceGenerationOptions(
PropertyNamingPolicy = JsonKnownNamingPolicy.SnakeCaseLower,
WriteIndented = true,
DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull)]
internal sealed partial class OtelContext : JsonSerializerContext;
```
`OtelMessageSerializer.SerializeChatMessages(...)` then serializes the final OTel payload with those options/type metadata:
```csharp
return JsonSerializer.Serialize(output, DefaultOptions.GetTypeInfo(typeof(IList)));
```
`OpenTelemetryChatClient.JsonSerializerOptions` is passed as `customContentSerializerOptions`, but that only influences the fallback serialization path for unknown/custom `AIContent` instances:
```csharp
customContentSerializerOptions?.TryGetTypeInfo(content.GetType(), out JsonTypeInfo? ctsi) is true ? ctsi :
DefaultOptions.TryGetTypeInfo(content.GetType(), out JsonTypeInfo? dtsi) ? dtsi :
null;
```
It does not affect the outer `gen_ai.input.messages` / `gen_ai.output.messages` JSON formatting, nor the built-in OTel message-part shapes.
The realtime path has the same effective behavior: it serializes message payloads through `OtelContext.Default.IEnumerableRealtimeOtelMessage`, which is generated from the same `OtelContext` with `WriteIndented = true`.
## Why this matters
The `gen_ai.input.messages` and `gen_ai.output.messages` attributes can be large, especially when `EnableSensitiveData` is enabled or when `OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=true` is set.
This matters for exporters/backends with size limits. For example, https://github.com/open-telemetry/opentelemetry-dotnet-contrib/issues/873 discusses Geneva exporter string/event limits:
- Geneva exporter currently has a string field limit of 16 KiB / 16,383 UTF-16 characters.
- Maintainers note there is also an approximately 65,360-byte total event limit from underlying ETW / Linux `user_events` constraints.
- The 16 KiB per-string limit is described as artificial/protective, but the overall event-size limit is not easy to lift.
- Later discussion asks for configurable string-size limits because exceeding the total event size can lead to full event loss.
Given those constraints, always emitting pretty-printed JSON for potentially large GenAI message attributes consumes limited telemetry payload budget without adding semantic value for production pipelines. Compact JSON would reduce the risk of truncation or dropped events/spans and reduce storage/export cost.
## Ask
Please consider adding a supported way to control this formatting. Possible approaches:
1. Add an option on `OpenTelemetryChatClient` / `OpenTelemetryRealtimeClient` to choose compact vs indented OTel message JSON.
2. Add a more general `OpenTelemetry*Client` serialization/formatting option for `gen_ai.input.messages` and `gen_ai.output.messages`.
3. If feasible, make existing `JsonSerializerOptions` influence final OTel message serialization, not only fallback serialization for custom/unknown `AIContent`.
The default could remain unchanged for compatibility. The key ask is to make compact JSON possible without post-processing Activity tags.
## Current workaround
The only practical workaround today is an OpenTelemetry processor that rewrites or removes the message attributes before export, for example by parsing and reserializing `gen_ai.input.messages` / `gen_ai.output.messages` compactly in a `BaseProcessor`.
Contributor guide
Assessment
This issue has not been assessed yet.