Azure / Azure/azure-sdk-for-python
[azure-ai-agentserver-responses] Failed response inputs are replayed in conversation history
- Dominant language
- Python
- Stars
- 5.6k
- Forks
- 3.4k
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 193
Description
- **Package Name**: `azure-ai-agentserver-responses`
- **Package Version**: `2.2.0b2` on current `main`; originally observed through `agent-framework-foundry-hosting` `1.0.0b260730`
- **Operating System**: Cross-platform; reproduced with a hosted Foundry agent and locally
- **Python Version**: 3.13.14 in the original report
## Describe the bug
AgentServer persists a stored response and its `input_items` when it processes the initial `response.created` / `in_progress` event. If the handler later emits `response.failed`, `update_response()` updates the response envelope but does not change the stored input references.
`get_history_item_ids()` subsequently includes those input IDs:
- when resolving responses in a `conversation_id`; and
- when a failed standalone response is later chained through `previous_response_id`.
As a result, invalid input that caused one request to fail is replayed into subsequent otherwise-valid requests, potentially poisoning the conversation indefinitely.
Original report and reproduction: https://github.com/microsoft/agent-framework/issues/7630
Agent Framework workaround under review: https://github.com/microsoft/agent-framework/pull/7637
That workaround has to buffer synchronous handler events so it knows the terminal status before AgentServer performs its initial create, and it wraps/mutates AgentServer's private providers. It cannot address streaming and background failures cleanly without delaying their streams.
## To reproduce
1. Create a conversation.
2. Submit a response containing an unmatched `function_call_output`:
```python
response = client.responses.create(
conversation=conversation.id,
input=[
{"role": "user", "content": "Hello"},
{
"type": "function_call_output",
"call_id": "call_that_does_not_exist",
"output": "invalid output",
},
],
)
```
3. Confirm the response fails because no matching function call exists.
4. Submit a valid request to the same conversation:
```python
response2 = client.responses.create(
conversation=conversation.id,
input="Hello, how are you?",
)
```
5. Observe that the second request fails with the same unmatched-function-call error because the first request's input was included in resolved conversation history.
The equivalent issue occurs if step 2 creates a stored standalone failed response and step 4 uses `previous_response_id=response.id`.
## Actual behavior
`get_history_item_ids()` includes the failed response's own input IDs, so subsequent handlers receive and replay the invalid input.
## Expected behavior
Failed-response inputs should remain stored and retrievable through `/responses/{id}/input_items` for diagnostics, but should not be returned as replayable history for either `conversation_id` or `previous_response_id`. Successful-response history should remain unchanged.
## Ownership rationale
This behavior is defined by `azure-ai-agentserver-responses`:
- its orchestrator decides when response inputs are persisted;
- `ResponseProviderProtocol` defines create, update, and history operations;
- its in-memory and file providers build conversation and previous-response history; and
- its Foundry provider delegates to the hosted `history/item_ids` storage endpoint.
Fixing this only in Agent Framework's `foundry_hosting` adapter leaves other AgentServer hosts and streaming/background request modes with the same behavior.
## Suggested direction
Define and enforce a provider invariant that a failed response's own `input_item_ids` are excluded from replayable history while the stored items remain available for diagnostic retrieval.
- Apply this before history-limit truncation.
- Update the in-memory and file providers.
- Apply the same behavior in the Foundry `history/item_ids` backend, or expose a supported server-side filter.
- Add tests for failed conversation turns and failed standalone responses later chained with `previous_response_id`, across synchronous, streaming, and background modes.
Related:
- https://github.com/microsoft/agent-framework/issues/7630
- https://github.com/microsoft/agent-framework/pull/7637
Contributor guide
Research direction
Start with ResponseProviderProtocol and get_history_item_ids(), then trace how the in-memory, file, and Foundry providers build conversation and previous-response history. Reproduce the failed conversation and standalone-response cases, including streaming and background modes. Done means failed inputs remain available through /responses/{id}/input_items but are excluded from replayable history, while successful history is unchanged.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure, python
- Domain
- api, backend, cloud
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100