[Proposal] DeepSeek-OCR example — SAM + CLIP dual vision encoder → DeepSeek-V2 MoE

Would you accept a DeepSeek-OCR example? I have a complete, working MLX port and I'd like to contribute it, but I want to check fit before opening a PR.

The model (baidu/Unlimited-OCR): a document-OCR VLM — SAM ViT-B + CLIP ViT-L encoders, linear projector, and a 12-layer DeepSeek-V2 MoE LLM (64 routed experts top-6 + 2 shared, use_mla=False). It does free OCR, layout parsing with grounding boxes, high-res tiling, and multi-page PDF → markdown.

Code: https://github.com/vignesh-kumar-v/mlx-ocr-unlimited

State of the port:

• Pure MLX at inference (PyTorch only for one-time weight conversion, like the llava example).
• Verified against the reference CUDA implementation: 0.9998 output similarity in fp32 (0.99 in fp16) on a full 15-page arXiv PDF.
• ~175 tok/s decode, ~6 GB in fp16 on an M5 Pro (24 GB).
• Unit tests (MoE batched-vs-loop equivalence, gating) plus a weight-gated end-to-end test.

Some pieces may be interesting beyond this one model: a batched gather_mm MoE, a KV cache that pins the image/prompt tokens while rotating a 128-token decode window (the model's decode-time attention pattern), and position-embedding resizing that is bit-identical to torch.nn.functional.interpolate (bicubic+antialias) via cached separable resize matrices.

Fit question: I'm aware mlx-vlm has DeepSeek-OCR support. My thinking is that a self-contained, readable example belongs here in the same spirit as llava (there's currently no OCR/document-parsing example), but if you'd rather point this elsewhere I'd like to know before I open the PR. If there's interest, I'll restructure to the flat llava-style layout (generate.py / model files / test.py / README.md / requirements.txt), black-formatted per CONTRIBUTING.md, and upload converted weights to mlx-community (per #155).

Start by comparing the proposed flat layout with the existing llava example and read CONTRIBUTING.md for formatting requirements. Review generate.py, the model files, test.py, README.md, and requirements.txt in the linked port, then confirm whether a self-contained OCR example fits here; done means maintainer agreement followed by a compliant PR and the stated tests.