Help wanted: test DeepSeek V4 Flash Vision-Exp AProjQ4 on Metal, CUDA, and ROCm

## Summary

We are looking for volunteers to test the new **DeepSeek V4 Flash Vision-Exp AProjQ4** GGUF on:

- Apple Silicon / Metal
- NVIDIA / CUDA
- AMD Strix Halo / ROCm
- Fully resident and SSD-streaming configurations

The runtime implementation is tracked in [PR #621](https://github.com/antirez/ds4/pull/621).

This request complements [#838](https://github.com/antirez/ds4/issues/838), which covers the non-Vision AProjQ4 model, and follows the original Vision support request in [#851](https://github.com/antirez/ds4/issues/851).

You only need to test one hardware/backend combination. Reports of crashes, incorrect image understanding, quality drift, unexpectedly low performance, or successful runs are all useful.

## Model artifact

The experimental model is available from [Hugging Face PR #22](https://huggingface.co/antirez/deepseek-v4-gguf/discussions/22):

[`DeepSeek-V4-Flash-Vision-Exp-IQ2XXS-w2Q2K-AProjQ4-SExpQ8-OutQ8-imatrix.gguf`](https://huggingface.co/antirez/deepseek-v4-gguf/blob/refs%2Fpr%2F22/DeepSeek-V4-Flash-Vision-Exp-IQ2XXS-w2Q2K-AProjQ4-SExpQ8-OutQ8-imatrix.gguf)

```text
Size: 84,420,584,608 bytes (78.62 GiB)
SHA-256: a1d0fde967c910e37e4bea00c4db1b2e8a88978770828a95772bc66837386cb8
```

The model was produced by requantizing 215 dense-attention projections—five projections across 43 layers—from `Q8_0` to `Q4_K`, using the 220k routed-and-dense DS4 imatrix.

All tensors outside the selected attention projections retain the source model’s quantization layout.

The matching Vision-Exp encoder sidecar is also required:

```text
DeepSeek-V4-Flash-Vision-Encoder.gguf
```

## Download

Install the current Hugging Face CLI if needed, then run:

```sh
mkdir -p gguf

hf download antirez/deepseek-v4-gguf \
DeepSeek-V4-Flash-Vision-Exp-IQ2XXS-w2Q2K-AProjQ4-SExpQ8-OutQ8-imatrix.gguf \
--revision refs/pr/22 \
--local-dir gguf

hf download antirez/deepseek-v4-gguf \
DeepSeek-V4-Flash-Vision-Encoder.gguf \
--revision main \
--local-dir gguf
```

Verify the language-model GGUF:

```sh
shasum -a 256 \
gguf/DeepSeek-V4-Flash-Vision-Exp-IQ2XXS-w2Q2K-AProjQ4-SExpQ8-OutQ8-imatrix.gguf
```

On Linux, use `sha256sum` instead.

## Build the current PR head

Please fetch the latest PR head immediately before testing and report its exact commit:

```sh
git clone https://github.com/antirez/ds4.git ds4-vision-q4
cd ds4-vision-q4

git fetch origin pull/621/head:pr-621
git checkout pr-621
git rev-parse HEAD
```

Build for your backend:

```sh
# Apple Silicon / Metal
make clean
make

# DGX Spark / GB10
make clean
make cuda-spark

# Other NVIDIA CUDA systems
make clean
make cuda-generic

# AMD Strix Halo / ROCm
make clean
make strix-halo -j"$(nproc)"
```

## Inspect the model first

```sh
MODEL=gguf/DeepSeek-V4-Flash-Vision-Exp-IQ2XXS-w2Q2K-AProjQ4-SExpQ8-OutQ8-imatrix.gguf
VISION=gguf/DeepSeek-V4-Flash-Vision-Encoder.gguf

./ds4 --cpu --inspect -m "$MODEL"
```

Expected high-level metadata:

```text
GGUF v3
1328 tensors
43 layers
q4_k: 215 tensors
q8_0: 130 tensors
file size: 78.62 GiB
```

## Run the model

### Metal with SSD streaming

```sh
./ds4 --metal --ssd-streaming \
--ctx 4096 --temp 0 \
-m "$MODEL" \
--vision "$VISION"
```

### CUDA with SSD streaming

```sh
./ds4 --cuda --ssd-streaming \
--ctx 4096 --temp 0 \
-m "$MODEL" \
--vision "$VISION"
```

### ROCm with SSD streaming

```sh
./ds4 --rocm --ssd-streaming \
--ctx 4096 --temp 0 \
-m "$MODEL" \
--vision "$VISION"
```

For fully resident testing, remove `--ssd-streaming`. The model, vision encoder, KV cache, and runtime buffers must all fit in available memory.

Inside the interactive CLI, submit a PNG or JPEG with:

```text
/read /path/to/test-image.png
```

Please use public or non-sensitive images.

## Minimum useful test

Please try at least:

1. One ordinary photograph with several recognizable objects.
2. One screenshot or image containing readable text.
3. One diagram, chart, or spatial-reasoning image.
4. A follow-up question referring to details from the same image.
5. A text-only prompt in the same session, to detect general language-quality regressions.

Check for:

- crashes or backend errors;
- NaNs or non-finite logits;
- repeated BOS tokens or looping output;
- incorrect image-token handling;
- severe hallucinations or loss of image grounding;
- unexpected quality differences from the AProjQ8 model;
- unusually slow vision encoding, prefill, or generation;
- excessive memory usage or SSD-cache churn.

## Optional AProjQ8 comparison

If storage and memory permit, compare against the matching Vision-Exp AProjQ8 language model:

```text
DeepSeek-V4-Flash-Vision-Exp-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8.gguf
```

Please use the same:

- DS4 commit;
- backend and execution mode;
- context size;
- vision encoder;
- image;
- prompt;
- temperature.

A matched AProjQ8/AProjQ4 comparison is especially valuable for detecting quantization-related quality drift.

## Reporting template

Please post results using this template:

```text
Hardware:
RAM / VRAM:
OS:
Backend:
Build command:
DS4 commit:
Resident or SSD streaming:
Context size:

Q4 SHA-256:
Vision encoder filename:

Exact command:
Image format and dimensions:
Image or public source link:

Result:
Response excerpt:
Vision grounding issues:
Crash/error output:

Vision/prefill timing:
Generation speed:
Peak or reported memory:

AProjQ8 comparison, if available:
Additional environment variables:
Other notes:
```

Please include complete error messages where possible, but redact private paths, tokens, credentials, and sensitive image content.

## Current validation status

The GGUF has already passed:

- structural GGUF inspection;
- tensor-count and tensor-type validation;
- strict imatrix compatibility checks;
- a text-only Metal SSD-streaming smoke test covering all 43 layers.

Full multimodal validation across different GPUs and operating systems is still needed. No broad Vision quality-equivalence claim is being made yet.