Azure / Azure/azure-sdk-for-python

[azure-ai-agentserver-agentframework] Streaming crash on Teams channel — "Streaming completed with 3 updates", no model response

Open
#46,307 3 comments 0 reactions 1 assignee Assigned to @JC-386 View on GitHub
customer-reported Hosted Agents needs-team-attention question Service Attention
Dominant language
Python
Stars
5.6k
Forks
3.4k
Avg merge
1d 21h
Merged PRs (30d)
193

Description

### Bug Report

**Package:** `azure-ai-agentserver-agentframework`
**Version:** `1.0.0b16` (also confirmed unchanged in `1.0.0b17`)
**Dependent packages:** `agent-framework-azure-ai==1.0.0rc2`, `agent-framework-core==1.0.0rc2`
**Python:** 3.12.13
**Platform:** Azure Container Apps (Linux x86_64, glibc 2.41)
**Channel:** Microsoft Teams via Bot Service + `activityprotocol` endpoint

---

### Description

When using `from_agent_framework()` with a hosted Azure AI Foundry agent connected to Microsoft Teams via Bot Service, every conversation turn crashes with `Streaming completed with 3 updates` and the user receives:

> *"We apologize, but something went wrong. Please try again."*

Two distinct bugs were identified through App Insights and container log analysis:

---

### Bug 1 — `call_id` exceeds 64 characters, corrupting the Foundry conversation store

When the agent makes a tool call (e.g. a registered `generate_outputs` function), the agentframework generates a `call_id` for the tool call output item that exceeds 64 characters. The Foundry conversation store API (`/openai/conversations/{id}/items`) enforces a 64-character maximum on this field and rejects the write with HTTP 400.

**Exception logged:**
```
BadRequestError: Failed to save output items to conversation
Error code: 400
{
"error": {
"code": "invalid_payload",
"message": "maxLength: Value should be at most 64 characters",
"param": "/items/1/call_id"
}
}
```

**Consequence:** The corrupted item remains in the conversation store. Every subsequent turn on that conversation loads the corrupted history and fails immediately with 3 streaming updates — no model inference occurs. The conversation is permanently broken until the conversation store entry is manually deleted via:

```bash
az rest --method DELETE \
--url "https://.services.ai.azure.com/api/projects//openai/conversations/?api-version=2025-11-15-preview" \
--resource "https://ai.azure.com/"
```

---

### Bug 2 — User message written to conversation store twice on every turn (Teams channel only)

After deleting the corrupted conversation and starting fresh, every turn still crashes with 3 streaming updates. Container logs show the user message is written to the Foundry conversation store **twice** before the agent run begins:

**Container trace (turn 1 on a brand new conversation):**
```
Start processing CreateResponse request.
Loaded agent session for conversation: conv_1d16a4c9815de76000...
Retrieved 2 messages for conversation conv_1d16a4c9815de76000... from Foundry.
Session messages count: 2
{"role": "user", "parts": [{"type": "text", "content": "Hello"}]}
{"role": "user", "parts": [{"type": "text", "content": "Hello"}]} ← duplicate
{"role": "user", "parts": [{"type": "text", "content": "Hello"}]} ← triplicate
Running agent in streaming mode
Streaming completed with 3 updates ← total elapsed: 1.2 seconds, no model inference
```

The conversation store for a brand new conversation (first message ever sent) already contains 2 entries before the run starts — both are the same user message. The run fails during startup, likely during input validation against the conversation store.

This bug is specific to the Teams channel routing path (`activityprotocol`). The same agent works correctly in the Foundry Playground.

---

### Reproduction Steps

**Environment:**
- Azure AI Foundry Hub with a `kind: hosted` container agent using `from_agent_framework()`
- Bot Service (`Microsoft.BotService/botServices`) connected to Teams channel via `activityprotocol` endpoint
- Agent has at least one registered tool function

**To reproduce Bug 1:**
1. Start a Teams conversation with the bot
2. Have the agent invoke a tool call (any registered tool function)
3. Observe `BadRequestError` 400 in App Insights exceptions table
4. Send any subsequent message — observe `Streaming completed with 3 updates`

**To reproduce Bug 2:**
1. Delete the corrupted conversation store entry (or use a fresh conversation)
2. Send the first message ("Hello") in Teams
3. Observe in container logs: `Session messages count: 2` on first turn, followed by duplicate user message entries
4. Observe `Streaming completed with 3 updates` — total elapsed ~1.2 seconds

**Does NOT reproduce in:** Foundry Playground (same container, same agent, different channel)

---

### Diagnostic evidence

**App Insights exceptions table — Bug 1:**
```
timestamp: 2026-04-13T18:27:24Z
type: BadRequestError
outerMessage: Failed to save output items to conversation: Error code: 400 -
{'error': {'code': 'invalid_payload',
'message': 'maxLength: Value should be at most 64 characters',
'param': '/items/1/call_id'}}
```

**App Insights exceptions table — known streaming crash pattern:**
```
timestamp: 2026-04-13T17:56:39Z
type: ChatClientException
outerMessage: Unhandled exception during streaming updates: Sorry, something went wrong.
```

**Container logs — streaming update counts:**
- `Streaming completed with 3 updates` = crash (no model inference, consistent across all failing turns)
- `Streaming completed with 46–161 updates` = success (observed in working Playground and pre-corruption Teams runs)

---

### Package diff findings

Diffed `azure-ai-agentserver-agentframework` b16 vs b17 wheel files — **the only difference is the version string** in `_version.py`. All source files are byte-for-byte identical.

Diffed `agent-framework-core` rc2 vs rc3 and `agent-framework-azure-ai` rc2 vs rc3 — **no changes to `call_id` handling, conversation store writes, or streaming converter logic** in either package. Changes in rc3 are limited to shell tool support and OAuth consent request handling.

The bugs are inside the closed/compiled streaming converter within the agentframework stack and are not addressable from application code.

---

### Expected behavior

- `call_id` values generated by the agentframework should not exceed 64 characters (the Foundry conversation store API limit)
- The user message should be written to the conversation store exactly once per turn
- A fresh conversation (no prior history) should not start with `Session messages count: 2`

---

### Additional context

- The Teams channel uses `activityprotocol` which only forwards user messages to the container — no assistant turns. This may interact with the conversation store write logic differently than the Playground path.
- The agent uses `DefaultAzureCredential` (ManagedIdentityCredential) for auth — confirmed working (token acquisition succeeds on every turn before the crash).
- Bot Service: `Microsoft.BotService/botServices`, kind `azurebot`, endpoint points to `activityprotocol?api-version=2025-11-15-preview`

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.