github / github/app

BYOK/Ollama: No user-facing error when local model output is truncated or emits malformed tool calls

Đang mở
#1,973 1 bình luận 0 reaction 0 người được giao Xem trên GitHub
Bugs
Ngôn ngữ chính
Không có dữ liệu ngôn ngữ
Star
2.1k
Fork
153
Chỉ số merge pull request
Không có pull request nào được merge trong 30 ngày

Mô tả

### Short summary

Local Ollama & Qwen 3.5 via GitHub Copilot App results in broken conversations

### Affected version or release

v.1.0.22

### Installation context

_No response_

### What happened?

My background: I did my first experiments with a local Ollama, Qwen 3.5 and tried to plug everything into the GitHub Copilot App. I got the model selection working, but the output is broken or interrupted. Here is a long version:

---
### Description

When using the GitHub Copilot app with a **local BYOK model via Ollama, agent turns can end silently when the model produces invalid output**. The app does not distinguish between "model returned invalid/truncated output" and a normal end of turn. As a user, I cannot tell whether the app is stuck, Ollama returned nothing, or the model failed.

The model failures themselves (truncation, malformed tool calls) are most likely caused by my setup (small model, limited VRAM, small context window), not by the app. **This report is about the missing feedback in the app when these failures occur.**

### Environment

- GitHub Copilot app v.1.0.22, Windows 11
- BYOK provider: Ollama (local), host http://localhost:11434
- Model: qwen3.5:9b (default Q4_K_M tag)
- GPU: NVIDIA GeForce RTX 3050, 8 GB VRAM
- Ollama context length: initially default 4096, later 16384 (via Ollama app settings)

Image

### Steps to reproduce

1. Add Ollama as a model provider in the Copilot app (Settings → Model Providers).
2. Select qwen3.5:9b in the model picker.
3. Open a repository and ask a question that requires tool use, e.g. "What does the MeetupController do?" or "Please summarize what this project does."
4. Repeat a few times; with a small model the failure modes below occur reliably within a few turns.
Observed failure modes (all end silently)

### Observed failures(all end silently)
1. Truncated response without any indicator

With the Ollama default context (4096) the response is cut off mid-word (e.g. "...rund um Gesellschaftssp"). The turn ends as if complete. No indication that the output was truncated or that the context window overflowed.

The query was "What is the project about" and the response stopped mid-token. After that, I increased the context windows.

Image

2. Malformed tool call rendered as plain text, turn ends

The model emitted a tool call as plain text instead of a proper function call:

The app displays this as regular text, nothing is executed, and the turn ends. No hint that the model produced an invalid tool call. In a follow-up turn ("Are you stuck?") the model then confabulated a completely unrelated ongoing task of something that was finished months ago.

Image

3. Tool-call fragment rendered as an executable PowerShell block

In another session the model produced only a fragment:

view_range: [1, 200]

The app rendered this inside a "PowerShell" code block, as if it were a command to run.

Image

### Steps to reproduce

_No response_

### Expected behavior

- If a response is truncated (context overflow / max tokens), show an indicator, e.g. "Response was cut off — the model's context window may be too small."
- If the model output cannot be parsed as a valid tool call, surface that, e.g. "The model returned an invalid tool call and the turn was aborted," ideally with the raw output collapsible for debugging.
- Never render unparsable tool-call fragments as executable command blocks.

**Basically, I have no idea if GHCP is still working, if it crashed or why it crashed. I don't know if it is an error on my computer or in the app.**

### Additional context

_No response_

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Hướng nghiên cứu

Bắt đầu bằng cách tái hiện các lỗi im lặng với provider Ollama và qwen3.5:9b bằng các prompt tool-use đã được báo cáo. Theo dõi cách các phản hồi bị cắt ngắn và các mảnh tool-call không đúng định dạng được xử lý và hiển thị; công việc được xem là hoàn tất khi người dùng nhận được lỗi cho mỗi đầu ra không hợp lệ hoặc bị cắt ngắn và các mảnh không được hiển thị dưới dạng lệnh có thể thực thi.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Đánh giá

Công nghệ
ollama
Lĩnh vực
ai, desktop
Loại issue
Lỗi
Độ khó
4/5
Thời gian dự kiến
3-5 ngày
Mức độ hoạt động
Ít trao đổi
Độ rõ ràng
Khá rõ ràng
Mức phù hợp với người mới
45/100

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.