[API Proposal]: Allow MarkItDownMcpReader to send `file:` / `http:` URIs instead of inlining data
- Dominant language
- C#
- Stars
- 3.2k
- Forks
- 894
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 23
Description
### Background and motivation
`MarkItDownMcpReader` always sends the document as an inline base64 `data:` URI, so large files exceed the MCP message-size limit (`413`/`-32600`) with no alternative — even though the `convert_to_markdown` tool accepts `file:`/`http:` URIs. Callers whose file is reachable by the server (mounted volume, shared path, URL) should be able to have the server read it directly.
### API Proposal
```csharp
namespace Microsoft.Extensions.DataIngestion;
public sealed class MarkItDownMcpReaderOptions
{
public Func? SourceUriResolver { get; set; }
}
public class MarkItDownMcpReader : IngestionDocumentReader
{
public MarkItDownMcpReader(
Uri mcpServerUri,
ModelContextProtocol.Client.McpClientOptions? options = null,
MarkItDownMcpReaderOptions? readerOptions = null);
}
```
### API Usage
```csharp
// Input dir mounted into the MarkItDown container at /data
var reader = new MarkItDownMcpReader(
new Uri("http://localhost:3001/mcp"),
readerOptions: new MarkItDownMcpReaderOptions
{
SourceUriResolver = fi => new Uri($"file:///data/{fi.Name}")
});
var doc = await reader.ReadAsync(new FileInfo("data/input/very_large.docx"), ct); // no inline bytes, no 413
```
### Alternative Designs
overload `ReadAsync(Uri sourceUri, string identifier, …)` so callers pass a server-reachable URI directly; or add a `bool InlineContent` toggle plus a base-path map.
### Risks
additive/opt-in — default (inline `data:`) is unchanged, so no breaking change. Resolver misconfiguration yields a server-side "file not found," which pairs naturally with fixing the `IsError` handling so it surfaces as a real error.
Contributor guide
Research direction
Start at the MarkItDownMcpReader constructor and ReadAsync entry point, then trace how the current inline data: URI is created and how MCP errors are handled. Define the opt-in resolver behavior and verify that default inline handling remains unchanged, including server-side failures for unresolved URIs. The payload names no source files or tests, so locate the reader implementation and its existing test coverage first.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp
- Domain
- backend, backend-api-design
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100