microsoft / microsoft/agent-framework

Improve native GenAI semantic convention coverage

Open
#6,626 1 comment 0 reactions 2 assignees Claimed by @eavanvalkenburg View on GitHub
Dominant language
Python
Stars
13.6k
Forks
2.3k
Avg merge
2d 45m
Merged PRs (30d)
358

Description

While adding Agent Framework to the OpenTelemetry GenAI semantic-conventions reference scenarios, we switched the scenario to use native Agent Framework telemetry (no manual reference spans) so the coverage reflects what AF emits today.

Observed native coverage from `agent-framework` with `OpenAIChatClient` and sensitive telemetry enabled:

- `invoke_agent_internal`
- `inference`
- `execute_tool`

The native telemetry is already useful, but compared with the GenAI semantic-conventions reference coverage we expected, these pieces are missing or not currently mapped:

1. No GenAI `invoke_workflow` span
- Workflow execution emits native workflow spans such as `workflow.build`, `workflow.run`, `executor.process`, `edge_group.process`, and `message.send`.
- These use `workflow.*`, `executor.*`, and message attributes, but do not produce `gen_ai.operation.name=invoke_workflow` with `gen_ai.workflow.name`, `gen_ai.input.messages`, and `gen_ai.output.messages`.

2. No `gen_ai.client.inference.operation.details` event/log
- The reference repo has event coverage for inference operation details, but native AF telemetry did not emit this event.

3. Some request attributes appear only on the agent span, not the inference span
- `gen_ai.request.max_tokens`, `gen_ai.request.temperature`, and `gen_ai.request.top_p` are present on the `invoke_agent` span.
- The child `chat`/inference spans currently include `gen_ai.request.choice.count` and model/response/usage attributes, but not those request option attributes.

4. Server attributes are partial
- Native inference currently reports `server.address` as `Unknown` against the local mock Responses endpoint and does not include `server.port`.

This came from running the GenAI semconv reference scenario against AF native telemetry in `semantic-conventions-genai`, using the local mock OpenAI Responses endpoint and `OpenAIChatClient`.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.