microsoft / microsoft/agent-framework
.NET: [Bug]: Uploading images to Gemini causes Bad Request
@stephentoub is already working on this.
Since Jan 23, 2026.
- Dominant language
- Python
- Stars
- 13.6k
- Forks
- 2.3k
- Avg merge
- 2d 45m
- Merged PRs (30d)
- 358
Description
### Description
I used the template defined in this project: [Microsoft.Agents.AI.ProjectTemplates](https://www.nuget.org/packages/Microsoft.Agents.AI.ProjectTemplates/1.0.0-preview.1.25619.3)
I first tried setting the open-ai client to point at the gemini compatible endpoint `https://generativelanguage.googleapis.com/v1beta/openai/`. I can build and then run the project, open devUI in the browser and I see this:
I get an arbitrary "Bad Response" with no further information.
I then installed `GeminiDotnet` to use its `IChatClient` implementation instead. When I run this I get something similar but it tells me that the issue is the MIME type:
If I look through the code, I find that the issue is [here](https://github.com/microsoft/agent-framework/blob/87c9d74bd74f80cb3c34a4e38f11092e51ffcc14/dotnet/src/Microsoft.Agents.AI.Hosting.OpenAI/Responses/Converters/ItemContentConverter.cs#L54).
On this line, the image's MIME type is converted to `"image/*"` arbitrarily. If you set a breakpoint there for a Data URI, you will see that the input content is `data:image/png;base64,iVBORw0KGgo...` with an explicit MIME type.
It seems that this code is coming from something labeled `Microsoft.Agents.AI.Hosting.OpenAI` which makes me think it is an OpenAI specific interface but that makes the `IChatClient` as a generic interface pretty ineffectual.
### Code Sample
```markdown
using System.ClientModel;
using System.ComponentModel;
using Microsoft.Agents.AI;
using Microsoft.Agents.AI.DevUI;
using Microsoft.Agents.AI.Hosting;
using Microsoft.Agents.AI.Workflows;
using Microsoft.Extensions.AI;
using OpenAI;
using OpenAI.Chat;
using OpenTelemetry;
using OpenTelemetry.Logs;
using OpenTelemetry.Resources;
using OpenTelemetry.Trace;
using GeminiDotnet;
using GeminiDotnet.Extensions.AI;
var builder = WebApplication.CreateBuilder(args);
// var options = new GeminiClientOptions
// {
// ApiKey = builder.Configuration["GEMINI_TOKEN"] ?? throw new InvalidOperationException("Missing configuration: GEMINI_TOKEN"),
// ModelId = "gemini-3-flash-preview",
// };
// var gClient = new GeminiChatClient(options);
// IChatClient chatClient = new ChatClientBuilder(gClient)
// .UseFunctionInvocation()
// .Build();
var chatClient = new ChatClient(
//"gemini-3-flash-preview",
"gemini-2.5-flash",
new ApiKeyCredential(builder.Configuration["GEMINI_TOKEN"] ?? throw new InvalidOperationException("Missing configuration: GEMINI_TOKEN")),
new OpenAIClientOptions { Endpoint = new Uri("https://generativelanguage.googleapis.com/v1beta/openai/") })
.AsIChatClient();
builder.Services.AddChatClient(chatClient);
builder.AddAIAgent("writer", "You write short stories (300 words or less) about the specified topic.");
builder.AddAIAgent("editor", (sp, key) => new ChatClientAgent(
chatClient,
name: key,
instructions: "You edit short stories to improve grammar and style, ensuring the stories are less than 300 words. Once finished editing, you select a title and format the story for publishing.",
tools: [AIFunctionFactory.Create(FormatStory)]
));
builder.AddWorkflow("publisher", (sp, key) => AgentWorkflowBuilder.BuildSequential(
workflowName: key,
agents:
[
sp.GetRequiredKeyedService("writer"),
sp.GetRequiredKeyedService("editor")
]
)).AddAsAIAgent("publisher-agent");
// Register services for OpenAI responses and conversations (also required for DevUI)
builder.Services.AddOpenAIResponses();
builder.Services.AddOpenAIConversations();
var app = builder.Build();
//app.UseHttpsRedirection();
// Map endpoints for OpenAI responses and conversations (also required for DevUI)
app.MapOpenAIResponses();
app.MapOpenAIConversations();
if (builder.Environment.IsDevelopment())
{
// Map DevUI endpoint to /devui
app.MapDevUI();
}
app.Run();
[Description("Formats the story for publication, revealing its title.")]
string FormatStory(string title, string story) => $"""
**Title**: {title}
{story}
""";
```
### Error Messages / Stack Traces
```markdown
No stacktrace in the output.
```
### Package Versions
Microsoft.Agents.AI: 1.0.0-preview.251219.1
### .NET Version
.Net 10.0
### Additional Context
_No response_
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.