microsoft / microsoft/agent-framework

Python: Hosted Responses sample loses invoke_agent token telemetry with latest hosting stack

Open
#7,797 1 comment 0 reactions 1 assignee Claimed by @TaoChenOSU View on GitHub
foundry hosting python
Dominant language
Python
Stars
13.6k
Forks
2.3k
Avg merge
2d 45m
Merged PRs (30d)
358

Description

### Describe the bug

The basic Agent Framework Responses hosted-agent sample does not emit the Agent Framework `invoke_agent` span or token usage attributes when deployed with the latest unpinned `agent-framework-foundry-hosting` dependency stack.

Sample:
https://github.com/microsoft-foundry/foundry-samples/tree/main/samples/python/hosted-agents/agent-framework/responses/01-basic

After pinning `agent-framework-foundry-hosting==1.0.0b260730`, the Agent Framework `invoke_agent` span and token attributes are emitted again.

### Reproduction

1. Deploy the sample to Microsoft Foundry using the Responses protocol and a `gpt-5` deployment.
2. Because Foundry invokes the hosted endpoint with `store=true`, enable resilient tasks before host startup:

```python
from azure.ai.agentserver.core.tasks import set_resilient_tasks_enabled

set_resilient_tasks_enabled(True)
```

3. Invoke the deployed agent normally.
4. Inspect the correlated trace in Application Insights.

### Failing dependency resolution

The unpinned sample resolved to:

- `azure-ai-agentserver-core==2.1.0b2`
- `azure-ai-agentserver-responses==2.1.0b1`
- latest available `agent-framework-foundry-hosting`

Observed behavior:

- The request succeeds after resilient tasks are enabled.
- Only the platform `invoke_agent` span is present.
- The Agent Framework `invoke_agent` span is absent.
- No `gen_ai.usage.input_tokens` or `gen_ai.usage.output_tokens` attributes are available.
- Runtime log contains:

```text
WARNING agent_framework_foundry_hosting._responses: Content type 'usage' is not supported yet. This is usually safe to ignore.
```

### Working dependency resolution

Change only the hosting requirement:

```text
agent-framework-foundry-hosting==1.0.0b260730
```

This transitively resolves `azure-ai-agentserver-responses==1.0.0b9` while retaining `azure-ai-agentserver-core==2.1.0b2`.

Observed behavior for the same sample and request:

- The Agent Framework `invoke_agent` span is emitted.
- `gen_ai.usage.input_tokens`, `gen_ai.usage.output_tokens`, and `gen_ai.usage.reasoning.output_tokens` are populated.
- Example observed values: input `29`, output `160`, reasoning `128`.

The `Content type 'usage' is not supported yet` warning is still logged in the working configuration, so the warning alone does not explain the missing span. This appears to be a compatibility/regression issue involving the newer hosting/AgentServer Responses stack.

### Expected behavior

The current unpinned hosted-agent stack should emit the Agent Framework `invoke_agent` span with model token usage attributes for successful Responses invocations, matching the behavior of the older compatible stack.

### Additional context

Without `set_resilient_tasks_enabled(True)`, `store=true` requests fail earlier with:

```text
Resilient task subsystem missing in hosted environment
```

That execution issue is resolved by enabling resilient tasks; this report concerns the remaining missing Agent Framework span and token telemetry after successful execution.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.