dotnet / dotnet/extensions

[Microsoft.Extensions.AI.OpenAI] Intermittent 400 previous_response_not_found when chaining Responses API turns with function invocation

Open
#7,704 4 comments 1 reaction 0 assignees View on GitHub
untriaged
Dominant language
C#
Stars
3.2k
Forks
894
Avg merge
1d 12h
Merged PRs (30d)
23

Description

## Description

When using the OpenAI **Responses** API through `AsIChatClient()` in a multi-turn console app with function/tool invocation, requests intermittently fail with a 400 from the service:

```
Previous response with id 'resp_...' not found.
```
(`"code": "previous_response_not_found"`)

The failure is intermittent — the same conversation can run for several turns, then a turn fails. Once it fails, retrying the same user turn usually succeeds, which suggests a stale/never-persisted response id is being sent as `previous_response_id` rather than a permanently corrupt conversation state.

`Microsoft.Extensions.AI.OpenAI` is what maps `ConversationId` onto `previous_response_id` for the Responses client, so filing here rather than against `openai/openai-dotnet`.

## Versions

| Package | Version |
|---|---|
| `Microsoft.Extensions.AI` | 10.9.0 |
| `Microsoft.Extensions.AI.OpenAI` | 10.9.0 |
| `OpenAI` | 2.12.0 |
| `ModelContextProtocol` | 2.2.0 |

.NET 10, Windows. Endpoint is Azure OpenAI's OpenAI-compatible `/openai/v1/` surface with `BearerTokenPolicy`.

## Repro shape

```csharp
var clientOptions = new OpenAIClientOptions
{
Endpoint = new Uri($"{azureOpenAIEndpoint.TrimEnd('/')}/openai/v1/"),
NetworkTimeout = TimeSpan.FromMinutes(5)
};

var openAIClient = new OpenAIClient(tokenPolicy, clientOptions);
var innerClient = openAIClient.GetResponsesClient().AsIChatClient();

var chatClient = new ChatClientBuilder(innerClient)
.UseFunctionInvocation()
.Build();

var chatOptions = new ChatOptions
{
ModelId = model,
AllowMultipleToolCalls = true,
ToolMode = ChatToolMode.Auto,
Tools = [.. mcpTools] // tools obtained from MCP servers
};

IList chatHistory = [new ChatMessage(ChatRole.System, systemPrompt)];

// per user turn:
chatHistory.Add(new ChatMessage(ChatRole.User, userMessage));

List receivedUpdates = [];
await foreach (var update in chatClient.GetStreamingResponseAsync(chatHistory, chatOptions))
{
receivedUpdates.Add(update);
// ... render text / reasoning / function call + result content ...
}

chatHistory.AddMessages(receivedUpdates);
```

The app keeps a single `IList` history and a single shared `ChatOptions` instance across turns, appends streamed updates back into history via `AddMessages`, and never sets `ChatOptions.ConversationId` itself. Tools are MCP tools, so most turns include at least one function call round trip inside `UseFunctionInvocation`.

## What I ruled out

The original app also had `.UseDistributedCache(...)` and `.UseChatReducer(new SummarizingChatReducer(...))` in the pipeline, so my first assumption was that one of those was replaying or rewriting history in a way that resurrected a stale conversation id. That turned out not to be it:

- Reproduces **with and without** `UseDistributedCache`.
- Reproduces **with and without** `UseChatReducer` / `SummarizingChatReducer`.
- Reproduces with **local (stdio)** MCP servers and with **remote (HTTP)** MCP servers, so it isn't specific to one transport or tool set.

With all of the optional middleware removed the failure still occurs, which is why I believe it lives in the Responses `IChatClient` response-chaining path (or in how `ConversationId` is round-tripped through streamed updates and `AddMessages`) rather than in application middleware.

## Expected

Either:

1. A conversation id that the service no longer recognizes should not be sent as `previous_response_id` — the client should fall back to sending the full input history instead of a dangling chain; or
2. If chaining is expected to be the caller's responsibility here, it would help a lot to have documented guidance on when `ConversationId` is safe to carry across turns when `UseFunctionInvocation` performs intermediate round trips, and how a consumer is meant to detect and recover from `previous_response_not_found`.

## Actual

An intermittent 400 `previous_response_not_found` surfaces to the caller as a `ClientResultException`, terminating the turn.

## Notes

Happy to instrument and capture request payloads (`previous_response_id` vs. submitted input items) for the failing turn if that would help pin it down — let me know what would be most useful.

Contributor guide

Open the contributing guide

Research direction

Start with the Responses client path behind AsIChatClient(), especially GetStreamingResponseAsync, UseFunctionInvocation, ConversationId, and AddMessages. Reproduce the intermittent previous_response_not_found failure while capturing previous_response_id and submitted history; done means identifying the stale-id path and establishing either reliable fallback behavior or documented recovery guidance.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure, csharp
Domain
api, backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.