BYOK/Ollama: No user-facing error when local model output is truncated or emits malformed tool calls
- Dominant language
- No language data
- Stars
- 2.1k
- Forks
- 153
- PR merge metrics
- No merged PRs in 30d
Description
### Short summary
Local Ollama & Qwen 3.5 via GitHub Copilot App results in broken conversations
### Affected version or release
v.1.0.22
### Installation context
_No response_
### What happened?
My background: I did my first experiments with a local Ollama, Qwen 3.5 and tried to plug everything into the GitHub Copilot App. I got the model selection working, but the output is broken or interrupted. Here is a long version:
---
### Description
When using the GitHub Copilot app with a **local BYOK model via Ollama, agent turns can end silently when the model produces invalid output**. The app does not distinguish between "model returned invalid/truncated output" and a normal end of turn. As a user, I cannot tell whether the app is stuck, Ollama returned nothing, or the model failed.
The model failures themselves (truncation, malformed tool calls) are most likely caused by my setup (small model, limited VRAM, small context window), not by the app. **This report is about the missing feedback in the app when these failures occur.**
### Environment
- GitHub Copilot app v.1.0.22, Windows 11
- BYOK provider: Ollama (local), host http://localhost:11434
- Model: qwen3.5:9b (default Q4_K_M tag)
- GPU: NVIDIA GeForce RTX 3050, 8 GB VRAM
- Ollama context length: initially default 4096, later 16384 (via Ollama app settings)
### Steps to reproduce
1. Add Ollama as a model provider in the Copilot app (Settings → Model Providers).
2. Select qwen3.5:9b in the model picker.
3. Open a repository and ask a question that requires tool use, e.g. "What does the MeetupController do?" or "Please summarize what this project does."
4. Repeat a few times; with a small model the failure modes below occur reliably within a few turns.
Observed failure modes (all end silently)
### Observed failures(all end silently)
1. Truncated response without any indicator
With the Ollama default context (4096) the response is cut off mid-word (e.g. "...rund um Gesellschaftssp"). The turn ends as if complete. No indication that the output was truncated or that the context window overflowed.
The query was "What is the project about" and the response stopped mid-token. After that, I increased the context windows.
2. Malformed tool call rendered as plain text, turn ends
The model emitted a tool call as plain text instead of a proper function call:
The app displays this as regular text, nothing is executed, and the turn ends. No hint that the model produced an invalid tool call. In a follow-up turn ("Are you stuck?") the model then confabulated a completely unrelated ongoing task of something that was finished months ago.
3. Tool-call fragment rendered as an executable PowerShell block
In another session the model produced only a fragment:
view_range: [1, 200]
The app rendered this inside a "PowerShell" code block, as if it were a command to run.
### Steps to reproduce
_No response_
### Expected behavior
- If a response is truncated (context overflow / max tokens), show an indicator, e.g. "Response was cut off — the model's context window may be too small."
- If the model output cannot be parsed as a valid tool call, surface that, e.g. "The model returned an invalid tool call and the turn was aborted," ideally with the raw output collapsible for debugging.
- Never render unparsable tool-call fragments as executable command blocks.
**Basically, I have no idea if GHCP is still working, if it crashed or why it crashed. I don't know if it is an error on my computer or in the app.**
### Additional context
_No response_
Contributor guide
Research direction
Start by reproducing the silent failures with the Ollama provider and qwen3.5:9b using the reported tool-use prompts. Trace how truncated responses and malformed tool-call fragments are handled and rendered; done means users receive an error for each invalid or truncated output and fragments are not presented as executable commands.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- ollama
- Domain
- ai, desktop
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100