antirez / antirez/ds4

DeepSeek V4 Flash Vision: image inference fails on Metal (M4 Max)

Đang mở
#928 6 bình luận 0 reaction 0 người được giao Xem trên GitHub
Ngôn ngữ chính
C
Star
22.3k
Fork
2.1k
Merge trung bình
1 ngày 3 giờ
Pull request đã merge (30 ngày)
4

Mô tả

## Summary

Text completion works, but image input fails with `400 invalid_request_error: "DeepSeek V4 vision inference failed"`.

## Setup

- Hardware: Apple M4 Max, 128 GiB unified memory
- Backend: Metal
- Commit: `b0982a1` (includes `fc8bf3c` "Add DeepSeek V4 Flash vision support")
- Main model: `DeepSeek-V4-Flash-Vision-Exp-MXFP4Experts-F16HC-F16Compressor-F16Indexer-Q8Attn-Q8Shared-Q8Out.gguf` (145.27 GiB, SSD streaming)
- Encoder: `DeepSeek-V4-Flash-Vision-Encoder.gguf` (0.87 GiB)

## What works

- `--inspect` loads both the main model and the vision encoder without error.
- `ds4-server` starts and serves text completions correctly (the model reasons and answers).

## What fails

- Sending an image (PNG data URI via `image_url`) returns HTTP 400 with `{"error":{"message":"DeepSeek V4 vision inference failed","type":"invalid_request_error"}}`.

## Root cause (source-level)

The failure comes from `ds4_gpu_deepseek4_vision_encode` (`ds4_metal.m:44875`) returning 0. The encoder runs a chain of `ds4_gpu_glm53_matmul_bf16` + `glm53_vision_dispatch_*` calls (the DeepSeek vision encoder reuses the GLM-5.3 Metal kernels), and one of the stages fails silently (`ok = 0` with no stderr detail).

GLM-5.3 vision works on the same machine, so the shared Metal kernels are fine — the DeepSeek-specific dispatch (`deepseek4_vision_dispatch_round`) appears to be where it breaks. There is a debug hook gated by `DS4_DEEPSEEK4_VISION_DEBUG_PREFIX` (`ds4_metal.m:44854`), but no per-stage error logging.

## Repro

```
./ds4-server --vision gguf/DeepSeek-V4-Flash-Vision-Encoder.gguf --ssd-streaming ...
# POST /v1/chat/completions with content [{type:text},{type:image_url,image_url:{url:"data:image/png;base64,..."}}]
```

Thanks for ds4.

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.