github / github/app

BYOK/Ollama: No user-facing error when local model output is truncated or emits malformed tool calls

オープン
#1,973 コメント 1 件 リアクション 0 件 担当者 0 名 GitHub で見る
Bugs
主要言語
言語のデータがありません
スター
2.1k
フォーク
153
PR マージ指標
30日以内にマージされた PR はありません

説明

### Short summary

Local Ollama & Qwen 3.5 via GitHub Copilot App results in broken conversations

### Affected version or release

v.1.0.22

### Installation context

_No response_

### What happened?

My background: I did my first experiments with a local Ollama, Qwen 3.5 and tried to plug everything into the GitHub Copilot App. I got the model selection working, but the output is broken or interrupted. Here is a long version:

---
### Description

When using the GitHub Copilot app with a **local BYOK model via Ollama, agent turns can end silently when the model produces invalid output**. The app does not distinguish between "model returned invalid/truncated output" and a normal end of turn. As a user, I cannot tell whether the app is stuck, Ollama returned nothing, or the model failed.

The model failures themselves (truncation, malformed tool calls) are most likely caused by my setup (small model, limited VRAM, small context window), not by the app. **This report is about the missing feedback in the app when these failures occur.**

### Environment

- GitHub Copilot app v.1.0.22, Windows 11
- BYOK provider: Ollama (local), host http://localhost:11434
- Model: qwen3.5:9b (default Q4_K_M tag)
- GPU: NVIDIA GeForce RTX 3050, 8 GB VRAM
- Ollama context length: initially default 4096, later 16384 (via Ollama app settings)

Image

### Steps to reproduce

1. Add Ollama as a model provider in the Copilot app (Settings → Model Providers).
2. Select qwen3.5:9b in the model picker.
3. Open a repository and ask a question that requires tool use, e.g. "What does the MeetupController do?" or "Please summarize what this project does."
4. Repeat a few times; with a small model the failure modes below occur reliably within a few turns.
Observed failure modes (all end silently)

### Observed failures(all end silently)
1. Truncated response without any indicator

With the Ollama default context (4096) the response is cut off mid-word (e.g. "...rund um Gesellschaftssp"). The turn ends as if complete. No indication that the output was truncated or that the context window overflowed.

The query was "What is the project about" and the response stopped mid-token. After that, I increased the context windows.

Image

2. Malformed tool call rendered as plain text, turn ends

The model emitted a tool call as plain text instead of a proper function call:

The app displays this as regular text, nothing is executed, and the turn ends. No hint that the model produced an invalid tool call. In a follow-up turn ("Are you stuck?") the model then confabulated a completely unrelated ongoing task of something that was finished months ago.

Image

3. Tool-call fragment rendered as an executable PowerShell block

In another session the model produced only a fragment:

view_range: [1, 200]

The app rendered this inside a "PowerShell" code block, as if it were a command to run.

Image

### Steps to reproduce

_No response_

### Expected behavior

- If a response is truncated (context overflow / max tokens), show an indicator, e.g. "Response was cut off — the model's context window may be too small."
- If the model output cannot be parsed as a valid tool call, surface that, e.g. "The model returned an invalid tool call and the turn was aborted," ideally with the raw output collapsible for debugging.
- Never render unparsable tool-call fragments as executable command blocks.

**Basically, I have no idea if GHCP is still working, if it crashed or why it crashed. I don't know if it is an error on my computer or in the app.**

### Additional context

_No response_

コントリビューションガイド

コントリビューションガイドを開く

調査の方向性

Start by reproducing the silent failures with the Ollama provider and qwen3.5:9b using the reported tool-use prompts. Trace how truncated responses and malformed tool-call fragments are handled and rendered; done means users receive an error for each invalid or truncated output and fragments are not presented as executable commands.

索引モデルが issue の本文から書いたものです。

評価

技術スタック
ollama
領域
ai, desktop
issue の種類
バグ
難易度
4/5
見積もり時間
3〜5日
活発さ
静か
明瞭さ
おおむね明確
初心者へのやさしさ
45/100

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。