antirez / antirez/ds4

DeepSeek V4 Flash Vision: image inference fails on Metal (M4 Max)

Abierto
#928 6 comentarios 0 reacciones 0 asignados Ver en GitHub
Lenguaje dominante
C
Estrellas
22.3k
Forks
2.1k
Merge medio
1 d 3 h
PR fusionados (30 d)
4

Descripción

## Summary

Text completion works, but image input fails with `400 invalid_request_error: "DeepSeek V4 vision inference failed"`.

## Setup

- Hardware: Apple M4 Max, 128 GiB unified memory
- Backend: Metal
- Commit: `b0982a1` (includes `fc8bf3c` "Add DeepSeek V4 Flash vision support")
- Main model: `DeepSeek-V4-Flash-Vision-Exp-MXFP4Experts-F16HC-F16Compressor-F16Indexer-Q8Attn-Q8Shared-Q8Out.gguf` (145.27 GiB, SSD streaming)
- Encoder: `DeepSeek-V4-Flash-Vision-Encoder.gguf` (0.87 GiB)

## What works

- `--inspect` loads both the main model and the vision encoder without error.
- `ds4-server` starts and serves text completions correctly (the model reasons and answers).

## What fails

- Sending an image (PNG data URI via `image_url`) returns HTTP 400 with `{"error":{"message":"DeepSeek V4 vision inference failed","type":"invalid_request_error"}}`.

## Root cause (source-level)

The failure comes from `ds4_gpu_deepseek4_vision_encode` (`ds4_metal.m:44875`) returning 0. The encoder runs a chain of `ds4_gpu_glm53_matmul_bf16` + `glm53_vision_dispatch_*` calls (the DeepSeek vision encoder reuses the GLM-5.3 Metal kernels), and one of the stages fails silently (`ok = 0` with no stderr detail).

GLM-5.3 vision works on the same machine, so the shared Metal kernels are fine — the DeepSeek-specific dispatch (`deepseek4_vision_dispatch_round`) appears to be where it breaks. There is a debug hook gated by `DS4_DEEPSEEK4_VISION_DEBUG_PREFIX` (`ds4_metal.m:44854`), but no per-stage error logging.

## Repro

```
./ds4-server --vision gguf/DeepSeek-V4-Flash-Vision-Encoder.gguf --ssd-streaming ...
# POST /v1/chat/completions with content [{type:text},{type:image_url,image_url:{url:"data:image/png;base64,..."}}]
```

Thanks for ds4.

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.