antirez / antirez/ds4

DeepSeek V4 Flash Vision: image inference fails on Metal (M4 Max)

Open
#928 6 comments 0 reactions 0 assignees View on GitHub
Dominant language
C
Stars
22.3k
Forks
2.1k
Avg merge
1d 3h
Merged PRs (30d)
4

Description

## Summary

Text completion works, but image input fails with `400 invalid_request_error: "DeepSeek V4 vision inference failed"`.

## Setup

- Hardware: Apple M4 Max, 128 GiB unified memory
- Backend: Metal
- Commit: `b0982a1` (includes `fc8bf3c` "Add DeepSeek V4 Flash vision support")
- Main model: `DeepSeek-V4-Flash-Vision-Exp-MXFP4Experts-F16HC-F16Compressor-F16Indexer-Q8Attn-Q8Shared-Q8Out.gguf` (145.27 GiB, SSD streaming)
- Encoder: `DeepSeek-V4-Flash-Vision-Encoder.gguf` (0.87 GiB)

## What works

- `--inspect` loads both the main model and the vision encoder without error.
- `ds4-server` starts and serves text completions correctly (the model reasons and answers).

## What fails

- Sending an image (PNG data URI via `image_url`) returns HTTP 400 with `{"error":{"message":"DeepSeek V4 vision inference failed","type":"invalid_request_error"}}`.

## Root cause (source-level)

The failure comes from `ds4_gpu_deepseek4_vision_encode` (`ds4_metal.m:44875`) returning 0. The encoder runs a chain of `ds4_gpu_glm53_matmul_bf16` + `glm53_vision_dispatch_*` calls (the DeepSeek vision encoder reuses the GLM-5.3 Metal kernels), and one of the stages fails silently (`ok = 0` with no stderr detail).

GLM-5.3 vision works on the same machine, so the shared Metal kernels are fine — the DeepSeek-specific dispatch (`deepseek4_vision_dispatch_round`) appears to be where it breaks. There is a debug hook gated by `DS4_DEEPSEEK4_VISION_DEBUG_PREFIX` (`ds4_metal.m:44854`), but no per-stage error logging.

## Repro

```
./ds4-server --vision gguf/DeepSeek-V4-Flash-Vision-Encoder.gguf --ssd-streaming ...
# POST /v1/chat/completions with content [{type:text},{type:image_url,image_url:{url:"data:image/png;base64,..."}}]
```

Thanks for ds4.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.