tryAGI / tryAGI/LangChain.Providers

当使用工具时,并且使用流式输出的时候,工具接收到的入参为空字符串

Open
#97 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
C#
Stars
23
Forks
13
PR merge metrics
No merged PRs in 30d

Description

Describe the bug

在使用流式输出的时候,如果LLM返回toolcall,会导致工具调用的参数为空

Steps to reproduce the bug
    static async Task Main(string[] args)
    {
        var chatModel = new OpenAiLatestFastChatModel(new OpenAiProvider(openaiApiKey, openAiBaseUrl));

        var calls = new Dictionary<string, Func<string, CancellationToken, Task<string>>>
        {
            { "query_document_tool", QueryDocument },
            { "query_document_tool_pro", QueryDocument }
        };


        var tools = new List<Tool>();

        var tool = new Tool
        {
            Name = "query_document_tool",
            Description = "当从给定的上下文中,无法回答用户的问题的时候,调用该工具来查询文档,通常一个问题如何回答不了就两个工具都调用",
            Parameters = new OpenApiSchema
            {
                Type = "object",
                Properties = new Dictionary<string, OpenApiSchema>
                {
                    ["query_content"] = new OpenApiSchema
                    {
                        Type = "string",
                        Description = "需要查询的内容",
                    }
                },
                Required = ["query_content"],
            },
        };

        tools.Add(tool);

        ChatRequest chatRequest = new ChatRequest()
        {
            Messages = new List<Message>() {
                "你是一个知识库问答助手,你只能通过使用工具获取到的信息进行问题的回答,不能编造内容".AsSystemMessage(),
                "我可以通过pdf服务器将html转为pdf吗?".AsHumanMessage()
            }
        };

        chatModel.AddGlobalTools(tools, calls);
        await foreach (var item in chatModel.GenerateAsync(chatRequest, new ChatSettings() { UseStreaming = true }))
        {
        }
    }

    private static async Task<string> QueryDocument(string arg1, CancellationToken token)
    {
        return "没有找到相关资料";
    }
}

如上代码 ,QueryDocument方法 收到的arg1为一个空字符串,但是如果UseStreaming=false则正常;

我跟了一下代码 ,在OpenAiChatModel.cs的流式输出 部分没有处理工具 相关内容导致的;
Expected behavior
        OpenAiChatModel.cs的206行,这样处理后 ,可以正常了 ,但是我不清楚是否符合你们的代码习惯 

if (usedSettings.UseStreaming == true)
{
var enumerable = provider.Api.Chat.CreateChatCompletionAsStreamAsync(
chatRequest,
cancellationToken).ConfigureAwait(false);

            var stringBuilder = new StringBuilder(capacity: 1024);
            await foreach (CreateChatCompletionStreamResponse streamResponse in enumerable)
            {
                var choice = streamResponse.Choices.ElementAtOrDefault(0);
                var streamDelta = choice?.Delta;
                var delta = new ChatResponseDelta
                {
                    Content = streamDelta?.Content ?? string.Empty,
                };
                if (streamDelta?.ToolCalls != null)
                {
                    if (toolCalls == null)
                    {
                        toolCalls = streamDelta.ToolCalls?.Select(x => new ChatToolCall
                        {
                            Id = x.Id ?? string.Empty,
                            ToolName = x.Function?.Name ?? string.Empty,
                            ToolArguments = x.Function?.Arguments ?? string.Empty,
                        }).ToList();
                    }
                    else
                    {
                        toolCalls[0].ToolArguments += streamDelta?.ToolCalls[0]?.Function?.Arguments;
                    }
                }

                usage ??= GetUsage(streamResponse);
                finishReason ??= choice?.FinishReason switch
                {
                    CreateChatCompletionStreamResponseChoiceFinishReason.Length => ChatResponseFinishReason.Length,
                    CreateChatCompletionStreamResponseChoiceFinishReason.Stop => ChatResponseFinishReason.Stop,
                    CreateChatCompletionStreamResponseChoiceFinishReason.ContentFilter => ChatResponseFinishReason.ContentFilter,
                    CreateChatCompletionStreamResponseChoiceFinishReason.FunctionCall => ChatResponseFinishReason.ToolCalls,
                    CreateChatCompletionStreamResponseChoiceFinishReason.ToolCalls => ChatResponseFinishReason.ToolCalls,
                    _ => null,
                };

                OnDeltaReceived(delta);
                stringBuilder.Append(delta.Content);

                yield return new ChatResponse
                {
                    Messages = messages,
                    UsedSettings = usedSettings,
                    Delta = delta,
                    Usage = Usage.Empty,
                };
            }
            OnDeltaReceived(new ChatResponseDelta
            {
                Content = Environment.NewLine,
            });
            stringBuilder.Append(Environment.NewLine);
            Message newMessage = default;
            if (toolCalls != null)
            {
                newMessage = toolCalls[0].ToolArguments.AsToolCallMessage(toolCalls[0].ToolName + ":" + toolCalls[0].Id);
            }
            else
            {
                newMessage = stringBuilder.ToString().AsAiMessage();
            }
            messages.Add(newMessage);
        }
Screenshots

No response

NuGet package version

No response

Additional context

No response

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in OpenAiChatModel.cs around the streaming-output branch and inspect how streamDelta.ToolCalls and function arguments are handled. Reproduce the supplied example with UseStreaming=true and compare it with false; done means the tool receives the streamed query_content arguments instead of an empty string.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
ai, api
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.