antirez / antirez/ds4

ds4-agent: context compaction rebuild fails with "metal prefill failed" in a Vision-Exp session

Aperta
#945 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub
Lingua principale
C
Stelle
22.3k
Fork
2.1k
Merge medio
1g 3h
PR unite (30g)
4

Descrizione

## What happened

In a long-running `ds4-agent` session on Metal using DeepSeek V4 Flash Vision-Exp, automatic context compaction generated its durable summary successfully but failed while rebuilding the compacted context:

```text
COMPACTING rebuilding context: old=85462 summary+tail=13924 tail=10000
ds4-agent: metal prefill failed
COMPACTING soft limit before user turn: summarizing durable task state
```

The previous transcript is restored after the failure, but the agent remains above the compaction threshold, so another user turn can trigger the same failure again.

## What I expected

Compaction should preserve or omit multimodal observations atomically, rebuild a valid compacted context, and continue processing the pending user turn.

## Steps to reproduce

1. Build `ds4-agent` from current `main` and run it on an Apple Silicon Mac with the Metal backend, a DeepSeek V4 Flash Vision-Exp model, and the DeepSeek V4 vision encoder.
2. Run a long coding session that includes one or more `view_image` observations.
3. Continue until the transcript reaches the automatic compaction threshold (about 85k tokens with the default 100k context).
4. Submit another user turn so automatic compaction runs.
5. Observe that summary generation completes, but the compacted-context Metal prefill fails.

## Additional context

- Commit: `819cef0` (local build; the vision encoder is configured to load by default, with inference code otherwise matching this commit).
- Hardware: Apple Silicon Mac with 128 GB unified memory.
- Backend: Metal.
- Context size: 100,000 tokens.
- Model: `DeepSeek-V4-Flash-Vision-Exp-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8.gguf`.
- Vision encoder: `DeepSeek-V4-Flash-Vision-Encoder.gguf`.
- The rebuilt prompt contains only 13,924 tokens, so this is not a context-length overflow.
- The retained tail was exactly the 10,000-token budget. Code inspection suggests that when no later user-turn boundary exists, the raw cutoff can land inside a DeepSeek image token block, while only fully retained image spans/embeddings are carried into the rebuilt transcript. That would leave a partial image block and cause Vision-Exp prefill to reject the prompt.
- If available, the lower-level stderr line immediately before `ds4-agent: metal prefill failed` would help distinguish a malformed image block from a Metal command-buffer failure.
- This appears distinct from #906 (insufficient room to create a summary) and #928 (vision encoding itself fails before inference).

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.