microsoft / microsoft/semantic-kernel

.Net: Bug: JSON serializer options don't match between tool schema and tool calls, causing enum parameters to fail.

Open
#13,589 3 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

.NET bug triage
Dominant language
C#
Stars
28.6k
Forks
4.8k
Avg merge
14h 13m
Merged PRs (30d)
18

Description

Describe the bug

KernelJsonSchemaBuilder specifies Converters = { new JsonStringEnumConverter() }, but the same is not used by the default JSON deserialization used by KernelFunctionFromMethod.TryToDeserializeValue, so any kernel function that has enums anywhere in the parameters will fail by default.

This TODO comment https://github.com/microsoft/semantic-kernel/blob/c086d0944259ab3f28ba16f1f984a3ab2fe4fc7d/dotnet/src/InternalUtilities/src/Schema/KernelJsonSchemaBuilder.cs#L16-L22 indicates that this is a known issue, but I couldn't find any actual issue tracking it.

To Reproduce
Steps to reproduce the behavior:

  1. Define a tool with enum parameters. In my case the enum was nested in an object, and then again in an array.
  2. Perform an invocation that would use the enum value.
  3. Observe System.ArgumentException when SK tries to brute-force the JSON string into the parameter anyway after deserialization fails.
System.ArgumentException
  HResult=0x80070057
  Message=Object of type 'System.String' cannot be converted to type 'TaskReminder[]'.
  Source=System.Private.CoreLib
  StackTrace:
   at System.RuntimeType.CheckValue(Object& value, Binder binder, CultureInfo culture, BindingFlags invokeAttr) in /_/src/runtime/src/libraries/System.Private.CoreLib/src/System/RuntimeType.cs:line 912
   at System.Reflection.MethodBaseInvoker.InvokeWithManyArgs(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture) in /_/src/runtime/src/libraries/System.Private.CoreLib/src/System/Reflection/MethodBaseInvoker.cs:line 278
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture) in /_/src/runtime/src/libraries/System.Private.CoreLib/src/System/Reflection/RuntimeMethodInfo.cs:line 129
   at Microsoft.SemanticKernel.KernelFunctionFromMethod.Invoke(MethodInfo method, Object target, Object[] arguments) in /home/vsts/work/1/s/semantic-kernel/dotnet/src/SemanticKernel.Core/Functions/KernelFunctionFromMethod.cs:line 1016
   at Microsoft.SemanticKernel.KernelFunctionFromMethod.<>c__DisplayClass21_0.<GetMethodDetails>g__Function|0(Kernel kernel, KernelFunction function, KernelArguments arguments, CancellationToken cancellationToken) in /home/vsts/work/1/s/semantic-kernel/dotnet/src/SemanticKernel.Core/Functions/KernelFunctionFromMethod.cs:line 553
   at Microsoft.SemanticKernel.KernelFunctionFromMethod.InvokeCoreAsync(Kernel kernel, KernelArguments arguments, CancellationToken cancellationToken) in /home/vsts/work/1/s/semantic-kernel/dotnet/src/SemanticKernel.Core/Functions/KernelFunctionFromMethod.cs:line 337
   at Microsoft.SemanticKernel.KernelFunction.<>c__DisplayClass32_0.<<InvokeAsync>b__0>d.MoveNext() in /home/vsts/work/1/s/semantic-kernel/dotnet/src/SemanticKernel.Abstractions/Functions/KernelFunction.cs:line 264

  This exception was originally thrown at this call stack:
    System.RuntimeType.CheckValue(ref object, System.Reflection.Binder, System.Globalization.CultureInfo, System.Reflection.BindingFlags) in RuntimeType.cs
    System.Reflection.MethodBaseInvoker.InvokeWithManyArgs(object, System.Reflection.BindingFlags, System.Reflection.Binder, object[], System.Globalization.CultureInfo) in MethodBaseInvoker.cs
    Microsoft.SemanticKernel.KernelFunctionFromMethod.Invoke(System.Reflection.MethodInfo, object, object[]) in KernelFunctionFromMethod.cs
    Microsoft.SemanticKernel.KernelFunctionFromMethod.GetMethodDetails.__Function|0(Microsoft.SemanticKernel.Kernel, Microsoft.SemanticKernel.KernelFunction, Microsoft.SemanticKernel.KernelArguments, System.Threading.CancellationToken) in KernelFunctionFromMethod.cs
    Microsoft.SemanticKernel.KernelFunctionFromMethod.InvokeCoreAsync(Microsoft.SemanticKernel.Kernel, Microsoft.SemanticKernel.KernelArguments, System.Threading.CancellationToken) in KernelFunctionFromMethod.cs
    Microsoft.SemanticKernel.KernelFunction.InvokeAsync.AnonymousMethod__0(Microsoft.SemanticKernel.FunctionInvocationContext) in KernelFunction.cs
System.Text.Json.JsonException
  HResult=0x80131500
  Message=The JSON value could not be converted to TaskReminder+ReminderUnit. Path: $[0].Unit | LineNumber: 0 | BytePositionInLine: 28.
  Source=System.Text.Json
  StackTrace:
   at System.Text.Json.ThrowHelper.ThrowJsonException(String message) in System.Text.Json\ThrowHelper.cs:line 908
   at System.Text.Json.Serialization.Converters.EnumConverter`1.Read(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options) in System.Text.Json.Serialization.Converters\EnumConverter.cs:line 233
   at System.Text.Json.Serialization.Metadata.JsonPropertyInfo`1.ReadJsonAndSetMember(Object obj, ReadStack& state, Utf8JsonReader& reader) in System.Text.Json.Serialization.Metadata\JsonPropertyInfo.cs:line 997
   at System.Text.Json.Serialization.Converters.ObjectDefaultConverter`1.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value) in System.Text.Json.Serialization.Converters\ObjectDefaultConverter.cs:line 197
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue) in System.Text.Json.Serialization\JsonConverter.cs:line 515
   at System.Text.Json.Serialization.JsonCollectionConverter`2.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, TCollection& value) in System.Text.Json.Serialization\JsonCollectionConverter.cs:line 210
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, T& value, JsonSerializerOptions options, ReadStack& state) in System.Text.Json.Serialization\JsonConverter.cs:line 332

  This exception was originally thrown at this call stack:
    System.Text.Json.ThrowHelper.ThrowJsonException(string) in ThrowHelper.cs
    System.Text.Json.Serialization.Converters.EnumConverter<T>.Read(ref System.Text.Json.Utf8JsonReader, System.Type, System.Text.Json.JsonSerializerOptions) in EnumConverter.cs
    System.Text.Json.Serialization.Metadata.JsonPropertyInfo<T>.ReadJsonAndSetMember(object, ref System.Text.Json.ReadStack, ref System.Text.Json.Utf8JsonReader) in JsonPropertyInfo.cs
    System.Text.Json.Serialization.Converters.ObjectDefaultConverter<T>.OnTryRead(ref System.Text.Json.Utf8JsonReader, System.Type, System.Text.Json.JsonSerializerOptions, ref System.Text.Json.ReadStack, out T) in ObjectDefaultConverter.cs
    System.Text.Json.Serialization.JsonConverter<T>.TryRead(ref System.Text.Json.Utf8JsonReader, System.Type, System.Text.Json.JsonSerializerOptions, ref System.Text.Json.ReadStack, out T, out bool) in JsonConverter.cs
    System.Text.Json.Serialization.JsonCollectionConverter<TCollection, TElement>.OnTryRead(ref System.Text.Json.Utf8JsonReader, System.Type, System.Text.Json.JsonSerializerOptions, ref System.Text.Json.ReadStack, out TCollection) in JsonCollectionConverter.cs
    System.Text.Json.Serialization.JsonConverter<T>.ReadCore(ref System.Text.Json.Utf8JsonReader, out T, System.Text.Json.JsonSerializerOptions, ref System.Text.Json.ReadStack) in JsonConverter.cs

Expected behavior
JSON serializer settings match between the schema passed to the LLM and the deserialization of the LLM's response.

Screenshots

Image

Platform

  • Language: C#
  • Source: NuGet
  • AI model: Any
  • IDE: [e.g. Visual Studio, VS Code] Any
  • OS: [e.g. Windows, Mac] Any

Additional context
Add any other context about the problem here.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Read the TODO in dotnet/src/InternalUtilities/src/Schema/KernelJsonSchemaBuilder.cs and the deserialization path in dotnet/src/SemanticKernel.Core/Functions/KernelFunctionFromMethod.cs. Compare the serializer options used for tool schemas with those used for tool-call arguments, then add coverage for enum parameters, including nested objects or arrays. Done means matching settings allow the reported enum invocation to deserialize without the fallback ArgumentException.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
api
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.