Goose AI Desktop not auto-compacting when hitting context limits with LM Studio
- 主要言語
- Rust
- スター
- 54.2k
- フォーク
- 6.2k
- 平均マージ
- 3日 2時間
- マージ済み PR(30日)
- 262
説明
**Describe the bug**
Goose AI doesn't auto-compact and continue during the agent loop in goose desktop 1.45.0 or 1.46.0 when running out of context with LM Studio-provided models. This seems to be LM Studio-specific.
In goose 1.45.0 the error is handled as a more general error (it seems because goose is not parsing it right).
In goose 1.46.0 the agent seems to cause an early end of completion with "finish_reason":"length", without having switched anything in LM Studio or restarting it between trying goose 1.45.0 and goose 1.46.0. I don't see anything different in the requests though.
---
**To Reproduce**
Steps to reproduce the behavior:
1. Optional: reduce max tokens for a model in LM Studio
2. Set `Context Overflow` to `Stop at Limit` in LM Studio
3. Load the model in LM Studio
4. Start a new long-running chat in Goose, such as a vague feature addition in a large codebase
5. wait until the model hits the context limit
6. Observe early stopping on 1.46.0 before finishing work with a warning in the goose chat, or an unhandled error message in 1.45.0. In either case, auto-compaction is not triggered.
---
**Expected behavior**
I'd expect the model to receive an error from LM Studio and auto-compact, then continue working on the task.
---
**Please provide the following information**
- **OS & Arch:** Mac aarch64-darwin
- **Interface** UI
- **Version:** v1.45.0, v1.46.0 (different behavior, both undesired)
- **Extensions enabled:** [e.g. Computer Controller, Figma]
- **Provider & Model:** LM Studio v0.4.21+2 & ornith-1.0-35b-mlx@6bit
---
**Screenshots**
### 1.46.0
### 1.45.0
---
**Additional context**
Related to https://github.com/aaif-goose/goose/issues/11099 where it was suggested to let the context error cause auto compaction rather than preemptive compaction based on the context limit and threshold in goose.
In order to test the behavior faster I:
* Set the model config.json limit to 30k
* Set context limit to 30k in LM Studio
* Set all model/global/provider context limits to 30k in goose configs
1.46.0 response payload contains (no error, finish reason is 'length'):
```
data: {"id":"chatcmpl-klmeuzsld3hb8xjqj501po","object":"chat.completion.chunk","created":1786974431,"model":"ornith-1.0-35b-mlx@6bit","system_fingerprint":"ornith-1.0-35b-mlx@6bit","choices":[{"index":0,"delta":{},"logprobs":null,"finish_reason":"length"}]}
```
1.45.0 response payload contains (unhandled error message format):
```
event: error
data: {"error":{"message":"Input does not fit in context length. The input has 30120 tokens, but the context length only supports 30000 tokens."},"message":"Input does not fit in context length. The input has 30120 tokens, but the context length only supports 30000 tokens."}
```
In 1.45.0 we were able to get compaction working by creating a separate proxy to catch the error with a message like `Input does not fit in context length.` and/or code `exceed_context_size_error` and re-write it as `This request exceeds the maximum context length: {err_msg}` and a code of `context_length_exceeded`
コントリビューションガイド
評価
この issue はまだ評価されていません。