microsoft / microsoft/agent-framework
.NET: [Flaky test] SessionFilesHostedAgentTests.UploadedFile_IsReadByHostedAgentAsync — HTTP 400 conflict despite in-test retry
@rogerbarreto is already working on this.
Since May 14, 2026.
- Dominant language
- Python
- Stars
- 13.6k
- Forks
- 2.3k
- Avg merge
- 2d 45m
- Merged PRs (30d)
- 358
Description
## Summary
`Foundry.Hosting.IntegrationTests.SessionFilesHostedAgentTests.UploadedFile_IsReadByHostedAgentAsync` is failing intermittently in the `dotnet-foundry-hosted-it` job with an HTTP 400 `conflict` from the hosted-agents service even though the test already implements a 5-attempt retry for this exact failure class.
This is **not** a flaky test in the unstable-by-default sense — it is the test catching a real eventual-consistency/identity-binding behavior in the hosted-agents service that the existing in-test mitigation can no longer absorb. Filing so it can be addressed permanently.
## Failure signature
```
System.ClientModel.ClientResultException : HTTP 400 (ServiceError: conflict)
The resource already exists or was modified concurrently. Please retry. [Request ID: 979e52b45be320cfa0e5adf2ff314cc0]
{
""code"": ""conflict"",
""message"": ""The resource already exists or was modified concurrently. Please retry. ..."",
""type"": ""invalid_request_error"",
...
}
at OpenAI.Responses.ResponsesClient.CreateResponseAsync(...)
at Azure.AI.Extensions.OpenAI.ProjectResponsesClient.CreateResponseAsync(CreateResponseOptions, CancellationToken)
at Foundry.Hosting.IntegrationTests.SessionFilesHostedAgentTests.UploadedFile_IsReadByHostedAgentAsync()
in dotnet/tests/Foundry.Hosting.IntegrationTests/SessionFilesHostedAgentTests.cs:141
```
Test wall time: `1m 02s 367ms` — consistent with all 5 retry attempts being exhausted.
## What the test already does
Step 4 of the test wraps `responses.CreateResponseAsync(readOptions)` in a bounded retry loop that catches exactly this error class:
```csharp
const int MaxAttempts = 5;
for (int attempt = 1; attempt <= MaxAttempts; attempt++)
{
try
{
rawResponse = await responses.CreateResponseAsync(readOptions);
break;
}
catch (ClientResultException ex) when (
ex.Status == 400 &&
ex.Message.Contains(""modified concurrently"", StringComparison.OrdinalIgnoreCase) &&
attempt < MaxAttempts)
{
await Task.Delay(TimeSpan.FromSeconds(2 * attempt));
}
}
```
(`dotnet/tests/Foundry.Hosting.IntegrationTests/SessionFilesHostedAgentTests.cs:135-151`)
The service is staying in conflict state for the full backoff window (2 + 4 + 6 + 8 = 20s plus call latency), so either:
1. The session/conversation revision is taking longer than 20s to settle after the `AgentSessionFiles` upload, or
2. The retry's identity is being re-bound on every attempt and never converges on the same session container.
## Suspected root cause
The test comment already calls this out:
> The platform mutates session/conversation revision when AgentSessionFiles uploads land, so an immediate /responses follow-up races and 400's with ""modified concurrently. Please retry."" — the response message literally tells us to retry. Bounded retry handles it.
Hypothesis: the platform-side write-fence after `UploadSessionFileAsync` is wider than the test's backoff envelope, or the `conversation_id` route is being recomputed by the service and momentarily disagrees with the upload's session id.
## Options to consider
1. **Stronger client-side mitigation**: increase `MaxAttempts`, lengthen backoff, or switch to exponential with jitter (e.g. up to 60s total).
2. **Service-side guarantee**: request that the hosted-agents service either (a) serialize the upload commit before acknowledging, or (b) return a retry-after hint instead of a 400 conflict.
3. **Quarantine** with `[Fact(Skip=""..."")]` short-term while #1 or #2 is implemented.
## Owner
Assigning to myself (PR #5702 area). Will handle so this stops blocking other devs.
## Related
Skip PR for the unrelated DevUI flake: #5846 / issue #5845.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.