AB-Law / AB-Law/QuietStories

[Feature]: Performance improvement

オープン
#43 コメント 0 件 リアクション 1 件 担当者 2 名 @AB-Law が担当を希望しています GitHub で見る
enhancement
主要言語
Python
スター
0
フォーク
0
PR マージ指標
30日以内にマージされた PR はありません

説明

### Is your feature request related to a problem? Please describe.

I'm currently facing long response times from the LLM when generating story outputs. Each OpenAI API call is taking nearly a minute to complete, which heavily affects the storytelling flow and user experience. My suspicion is that multiple tool calls are being triggered during message generation, and it’s unclear whether all of them are working as intended. Debugging or tracking tool execution is also difficult right now.

### Describe the solution you'd like

I’d like to optimize and monitor LLM calls to significantly reduce latency. Ideally, each LLM response should return within a few seconds, even when tools are involved. Possible solutions could include:

Implementing better tracing/logging to confirm which tool calls are actually being executed.
Introducing batching or concurrency for tool calls.
Reducing unnecessary calls or smoothening out the whole flow.
Adding metrics or timing logs to visualize performance bottlenecks.

### Describe alternatives you've considered

I’ve tried a few approaches to mitigate the delay:

Reducing the number of tool calls per LLM run, but the latency persists, suggesting underlying inefficiencies in how the calls are handled or awaited.
Switching to smaller or faster models temporarily, which slightly improved performance but didn’t resolve the delay when tools were used.

### Additional context

The storyteller bot makes multiple tool calls per story segment (e.g., fetching character data, updating world state, or generating narrative branches). Each of these steps seems to contribute to cumulative delay.
I suspect some calls might be sequential instead of parallel, or waiting unnecessarily for previous responses to complete.

### Would you like to contribute this feature?

- [ ] Yes, I would like to implement this feature

コントリビューションガイド

コントリビューションガイドを開く

調査の方向性

The issue mentions LLM calls and tool execution in a storytelling bot. Start by examining the code that makes OpenAI API calls and handles tool calls, likely in a service or agent module. Look for sequential awaits and add logging to trace tool execution times. Check if there are existing metrics or timing utilities. 'Done' means each LLM response returns within seconds, with clear logs showing bottlenecks.

索引モデルが issue の本文から書いたものです。

評価

領域
backend, performance
issue の種類
機能追加
難易度
4/5
見積もり時間
3〜5日
活発さ
停滞
明瞭さ
おおむね明確
初心者へのやさしさ
35/100

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。