antirez/ds4

View on GitHub

DeepSeek 4 Flash and PRO local inference engine for Metal, CUDA and ROCm

Stars: 22.4k
Forks: 2.1k
Open beginner issues: 0
Indexed issues: 256
Avg merge: 2d 13h
Merged PRs (30d): 5

Dominant language: C
License: MIT
Last GitHub push: Sep 13, 2026
Latest indexed: Sep 13, 2026
Contributing guide: Contributing guide
Code of conduct: No code of conduct
Beginner labels: help wanted

Open All

256 issues indexed so far

test_deepseek41_metal: new router check fails on M1 Ultra (pre-M5); sqrt(softplus) compared at 3e-6 but 1.0f+exp is ~0.7% imprecise Open

antirez/ds4#1039 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 14, 2026
Request: support glm 5.3 flash Closed

antirez/ds4#1040 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 14, 2026
AMD GPU (ROCm/HIP) and especially Strix Halo discussion group Open

antirez/ds4#16 · 266 comments · 24 reactions · 0 assignees · Indexed Sep 13, 2026
SSD tree cache KV cache Open

antirez/ds4#17 · 3 comments · 1 reaction · 0 assignees · Indexed Sep 13, 2026
Feature Request: Add HTTP CORS Support Open

antirez/ds4#20 · 1 comment · 1 reaction · 0 assignees · Indexed Sep 13, 2026
Question: any interest in OpenAI Responses API support for Codex compatibility? Open

antirez/ds4#22 · 2 comments · 4 reactions · 0 assignees · Indexed Sep 13, 2026
Context Window Exhaustion Causes Unterminated Tool Calls Open

antirez/ds4#48 · 1 comment · 1 reaction · 0 assignees · Indexed Sep 13, 2026
Speculative Decoding via "Draft KV Stitching" Open

antirez/ds4#63 · 1 comment · 1 reaction · 0 assignees · Indexed Sep 13, 2026
can you add deepseek q8 ? Open

antirez/ds4#64 · 3 comments · 0 reactions · 0 assignees · Indexed Sep 13, 2026
FP4 inference for 4 bit Open

correctness

antirez/ds4#74 · 8 comments · 2 reactions · 0 assignees · Indexed Sep 13, 2026
Feature request: Natural Language Level Speculative Decoding with heterogeneous Draft Models (e.g. Qwen 2B) Open

help wanted speed would-skip-this

antirez/ds4#80 · 2 comments · 1 reaction · 0 assignees · Indexed Sep 13, 2026
Please provide a ModelScope mirror for Chinese users who cannot easily download large models from Hugging Face Open

antirez/ds4#85 · 2 comments · 1 reaction · 0 assignees · Indexed Sep 13, 2026
ds4 parser vulnerability: user prompt can insert special tokens that are interpreted by the model Open

antirez/ds4#95 · 1 comment · 4 reactions · 0 assignees · Indexed Sep 13, 2026
[feature] Offload KV cache from GPU (VRAM) to RAM Open

antirez/ds4#108 · 2 comments · 1 reaction · 0 assignees · Indexed Sep 13, 2026
Question: why does readme recommend using openai-completions api for coding agent usage. Open

documentation

antirez/ds4#113 · 3 comments · 0 reactions · 0 assignees · Indexed Sep 13, 2026
API access security verification Open

antirez/ds4#120 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 13, 2026
Multi-GPU for nvidia Open

cuda distributed-inference ideas-to-explore

antirez/ds4#123 · 29 comments · 13 reactions · 0 assignees · Indexed Sep 13, 2026
Logits Processor Support Open

would-skip-this

antirez/ds4#142 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 13, 2026
Empirical RSS curve for incremental ctx loading (10K to 250K) on M4 Max 128GB - is --kv-disk-dir masking the indexer growth? Open

antirez/ds4#164 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 13, 2026
fedora f44: rocm: build fails to find libstdc++ Open

antirez/ds4#179 · 6 comments · 1 reaction · 0 assignees · Indexed Sep 13, 2026
Request: Split large models into chunks Open

antirez/ds4#188 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 13, 2026
Server: expose directional steering as a model-callable DSML tool Open

antirez/ds4#190 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 13, 2026
MoE disk offload Open

antirez/ds4#202 · 3 comments · 0 reactions · 0 assignees · Indexed Sep 13, 2026
我在 GitHub 上的首次留言，献给让 V4-Flash 在 Mac 上奔跑的大神！ Open

antirez/ds4#208 · 0 comments · 13 reactions · 0 assignees · Indexed Sep 13, 2026
Roadmap question: multi-session / concurrent multi-user inference support Open

ideas-to-explore

antirez/ds4#209 · 8 comments · 0 reactions · 0 assignees · Indexed Sep 13, 2026
Add support for structured outputs Open

antirez/ds4#210 · 12 comments · 3 reactions · 0 assignees · Indexed Sep 13, 2026
Return deepseek-chat in GET models Open

antirez/ds4#216 · 3 comments · 0 reactions · 0 assignees · Indexed Sep 13, 2026
Shared Library Support Open

antirez/ds4#217 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 13, 2026
support for vulkan backend Open

antirez/ds4#237 · 3 comments · 14 reactions · 0 assignees · Indexed Sep 13, 2026
ds4-agent: separate runtime asset root from workspace cwd Open

antirez/ds4#240 · 1 comment · 5 reactions · 0 assignees · Indexed Sep 13, 2026
ds4 / DGX Spark / MTP and Performance Improvements Open

cuda speed

antirez/ds4#244 · 9 comments · 0 reactions · 0 assignees · Indexed Sep 13, 2026
download_model.sh pro-imatrix --> error 400 Open

antirez/ds4#249 · 5 comments · 2 reactions · 0 assignees · Indexed Sep 13, 2026
Support for 2 GX10s cluster with 128GB each - support tensor parallelism Open

antirez/ds4#262 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 13, 2026
cuda prefill failed on NVIDIA A100-SXM4-80GB during both inference and bench Open

antirez/ds4#284 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 13, 2026
on naming Open

antirez/ds4#288 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 13, 2026
Attach file from open-webui does not work Open

antirez/ds4#291 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 13, 2026
Distributed CUDA worker host-registers the whole GGUF, not just its layer slice → Q4 OOM on 128GB DGX Spark Open

antirez/ds4#293 · 8 comments · 0 reactions · 0 assignees · Indexed Sep 13, 2026
How to choose between models for the 512GB max m3ultra Open

antirez/ds4#303 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 13, 2026
Support distributed prefill with local decode Open

antirez/ds4#304 · 4 comments · 0 reactions · 0 assignees · Indexed Sep 13, 2026
Onboarding skill for DS4: a docs-grounded agent skill tutor Open

antirez/ds4#309 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 13, 2026
Poor mans parallelism: ~1.6× from 2 processes Open

antirez/ds4#314 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 13, 2026
Unclosed </think> before tool call stalls agent session (finish=stop, no tool call delivered) Open

antirez/ds4#318 · 8 comments · 9 reactions · 0 assignees · Indexed Sep 13, 2026
SSE body read failed: DeepSeek stream timed out after 660000ms Open

antirez/ds4#329 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 13, 2026
Server does not abort prefill when client disconnects Open

antirez/ds4#333 · 1 comment · 5 reactions · 0 assignees · Indexed Sep 13, 2026
Health checkpoint Open

antirez/ds4#334 · 0 comments · 1 reaction · 0 assignees · Indexed Sep 13, 2026
Distributed SSD streaming Open

antirez/ds4#340 · 3 comments · 3 reactions · 0 assignees · Indexed Sep 13, 2026
Feature request: more information in usage Open

antirez/ds4#342 · 0 comments · 1 reaction · 0 assignees · Indexed Sep 13, 2026
ROCm is just a "CUDA" alternative, isn't it? Open

antirez/ds4#357 · 0 comments · 1 reaction · 0 assignees · Indexed Sep 13, 2026
ROCm: high-context OOM on large-VRAM-carveout APUs (Strix Halo) — general tensor allocator is device-only Open

antirez/ds4#359 · 2 comments · 1 reaction · 0 assignees · Indexed Sep 13, 2026
ROCm/Windows: native AMD HIP SDK build for gfx1151 + >2 GiB staged-read off_t truncation Open

antirez/ds4#360 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 13, 2026

antirez/ds4

Get new issues in your inbox