.NET: [Feature]: Automatically fix malformed JSON responses from LLMs

Description

While working with LLMs for tool calling and structured outputs, the models occasionally return malformed or slightly corrupted JSON responses. Common issues include:

• Markdown Fences: Wrapping the JSON block inside markdown formatting (e.g., ```json ... ```).
• Trailing Commas: Leaving trailing commas at the end of objects or arrays (which standard System.Text.Json strictly rejects with a JsonException).
• Truncated Outputs: Cut-off responses due to token limits, resulting in missing closing brackets (}, ]).
• Stringified Fields: Embedding raw objects inside stringified attributes (e.g., "arguments": "{\"key\": \"value\"}" instead of "arguments": {"key": "value"}).

Implementing this at the gateway of our parser layer will drastically improve system resilience, cut down on redundant error handling, and make our integration with LLMs much more production-ready.

Code Sample

Language/SDK

.NET

Start by locating the .NET parser layer that uses System.Text.Json and review how malformed LLM responses enter it. Define and test the intended handling for markdown fences, trailing commas, truncated brackets, and stringified fields; done means these listed response forms are repaired or handled consistently without the current JsonException failures.