antirez / antirez/ds4

DeepSeek V4 Flash Vision: image inference fails on Metal (M4 Max)

Offen
#928 6 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
Vorherrschende Sprache
C
Sterne
22.3k
Forks
2.1k
Ø Merge
1 T. 3 Std.
Gemergte PRs (30 T.)
4

Beschreibung

## Summary

Text completion works, but image input fails with `400 invalid_request_error: "DeepSeek V4 vision inference failed"`.

## Setup

- Hardware: Apple M4 Max, 128 GiB unified memory
- Backend: Metal
- Commit: `b0982a1` (includes `fc8bf3c` "Add DeepSeek V4 Flash vision support")
- Main model: `DeepSeek-V4-Flash-Vision-Exp-MXFP4Experts-F16HC-F16Compressor-F16Indexer-Q8Attn-Q8Shared-Q8Out.gguf` (145.27 GiB, SSD streaming)
- Encoder: `DeepSeek-V4-Flash-Vision-Encoder.gguf` (0.87 GiB)

## What works

- `--inspect` loads both the main model and the vision encoder without error.
- `ds4-server` starts and serves text completions correctly (the model reasons and answers).

## What fails

- Sending an image (PNG data URI via `image_url`) returns HTTP 400 with `{"error":{"message":"DeepSeek V4 vision inference failed","type":"invalid_request_error"}}`.

## Root cause (source-level)

The failure comes from `ds4_gpu_deepseek4_vision_encode` (`ds4_metal.m:44875`) returning 0. The encoder runs a chain of `ds4_gpu_glm53_matmul_bf16` + `glm53_vision_dispatch_*` calls (the DeepSeek vision encoder reuses the GLM-5.3 Metal kernels), and one of the stages fails silently (`ok = 0` with no stderr detail).

GLM-5.3 vision works on the same machine, so the shared Metal kernels are fine — the DeepSeek-specific dispatch (`deepseek4_vision_dispatch_round`) appears to be where it breaks. There is a debug hook gated by `DS4_DEEPSEEK4_VISION_DEBUG_PREFIX` (`ds4_metal.m:44854`), but no per-stage error logging.

## Repro

```
./ds4-server --vision gguf/DeepSeek-V4-Flash-Vision-Encoder.gguf --ssd-streaming ...
# POST /v1/chat/completions with content [{type:text},{type:image_url,image_url:{url:"data:image/png;base64,..."}}]
```

Thanks for ds4.

Beitragsleitfaden

Beitragsleitfaden öffnen

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.