dotnet / dotnet/extensions

[API Proposal]: Allow MarkItDownMcpReader to send `file:` / `http:` URIs instead of inlining data

Open
#7,619 0 comments 0 reactions 0 assignees View on GitHub
api-suggestion untriaged
Dominant language
C#
Stars
3.2k
Forks
894
Avg merge
1d 12h
Merged PRs (30d)
23

Description

### Background and motivation

`MarkItDownMcpReader` always sends the document as an inline base64 `data:` URI, so large files exceed the MCP message-size limit (`413`/`-32600`) with no alternative — even though the `convert_to_markdown` tool accepts `file:`/`http:` URIs. Callers whose file is reachable by the server (mounted volume, shared path, URL) should be able to have the server read it directly.

### API Proposal

```csharp
namespace Microsoft.Extensions.DataIngestion;

public sealed class MarkItDownMcpReaderOptions
{
public Func? SourceUriResolver { get; set; }
}

public class MarkItDownMcpReader : IngestionDocumentReader
{
public MarkItDownMcpReader(
Uri mcpServerUri,
ModelContextProtocol.Client.McpClientOptions? options = null,
MarkItDownMcpReaderOptions? readerOptions = null);
}
```

### API Usage

```csharp
// Input dir mounted into the MarkItDown container at /data
var reader = new MarkItDownMcpReader(
new Uri("http://localhost:3001/mcp"),
readerOptions: new MarkItDownMcpReaderOptions
{
SourceUriResolver = fi => new Uri($"file:///data/{fi.Name}")
});

var doc = await reader.ReadAsync(new FileInfo("data/input/very_large.docx"), ct); // no inline bytes, no 413
```

### Alternative Designs

overload `ReadAsync(Uri sourceUri, string identifier, …)` so callers pass a server-reachable URI directly; or add a `bool InlineContent` toggle plus a base-path map.

### Risks

additive/opt-in — default (inline `data:`) is unchanged, so no breaking change. Resolver misconfiguration yields a server-side "file not found," which pairs naturally with fixing the `IsError` handling so it surfaces as a real error.

Contributor guide

Open the contributing guide

Research direction

Start at the MarkItDownMcpReader constructor and ReadAsync entry point, then trace how the current inline data: URI is created and how MCP errors are handled. Define the opt-in resolver behavior and verify that default inline handling remains unchanged, including server-side failures for unresolved URIs. The payload names no source files or tests, so locate the reader implementation and its existing test coverage first.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
backend, backend-api-design
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.