aaif-goose / aaif-goose/goose

Goose AI Desktop not auto-compacting when hitting context limits with LM Studio

Offen
#11,318 27 Kommentare 0 Reaktionen 1 zugewiesene Person Beansprucht von @DOsinga Auf GitHub ansehen
Vorherrschende Sprache
Rust
Sterne
54.2k
Forks
6.2k
Ø Merge
3 T. 2 Std.
Gemergte PRs (30 T.)
262

Beschreibung

**Describe the bug**

Goose AI doesn't auto-compact and continue during the agent loop in goose desktop 1.45.0 or 1.46.0 when running out of context with LM Studio-provided models. This seems to be LM Studio-specific.

In goose 1.45.0 the error is handled as a more general error (it seems because goose is not parsing it right).

In goose 1.46.0 the agent seems to cause an early end of completion with "finish_reason":"length", without having switched anything in LM Studio or restarting it between trying goose 1.45.0 and goose 1.46.0. I don't see anything different in the requests though.

---

**To Reproduce**
Steps to reproduce the behavior:
1. Optional: reduce max tokens for a model in LM Studio
2. Set `Context Overflow` to `Stop at Limit` in LM Studio
3. Load the model in LM Studio
4. Start a new long-running chat in Goose, such as a vague feature addition in a large codebase
5. wait until the model hits the context limit
6. Observe early stopping on 1.46.0 before finishing work with a warning in the goose chat, or an unhandled error message in 1.45.0. In either case, auto-compaction is not triggered.

---

**Expected behavior**
I'd expect the model to receive an error from LM Studio and auto-compact, then continue working on the task.

---

**Please provide the following information**
- **OS & Arch:** Mac aarch64-darwin
- **Interface** UI
- **Version:** v1.45.0, v1.46.0 (different behavior, both undesired)
- **Extensions enabled:** [e.g. Computer Controller, Figma]
- **Provider & Model:** LM Studio v0.4.21+2 & ornith-1.0-35b-mlx@6bit

---
**Screenshots**
### 1.46.0

Image

### 1.45.0

Image

---

**Additional context**

Related to https://github.com/aaif-goose/goose/issues/11099 where it was suggested to let the context error cause auto compaction rather than preemptive compaction based on the context limit and threshold in goose.

In order to test the behavior faster I:
* Set the model config.json limit to 30k
* Set context limit to 30k in LM Studio
* Set all model/global/provider context limits to 30k in goose configs

1.46.0 response payload contains (no error, finish reason is 'length'):
```
data: {"id":"chatcmpl-klmeuzsld3hb8xjqj501po","object":"chat.completion.chunk","created":1786974431,"model":"ornith-1.0-35b-mlx@6bit","system_fingerprint":"ornith-1.0-35b-mlx@6bit","choices":[{"index":0,"delta":{},"logprobs":null,"finish_reason":"length"}]}
```

1.45.0 response payload contains (unhandled error message format):
```
event: error
data: {"error":{"message":"Input does not fit in context length. The input has 30120 tokens, but the context length only supports 30000 tokens."},"message":"Input does not fit in context length. The input has 30120 tokens, but the context length only supports 30000 tokens."}
```

In 1.45.0 we were able to get compaction working by creating a separate proxy to catch the error with a message like `Input does not fit in context length.` and/or code `exceed_context_size_error` and re-write it as `This request exceeds the maximum context length: {err_msg}` and a code of `context_length_exceeded`

Beitragsleitfaden

Beitragsleitfaden öffnen

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.