ml-explore/mlx-lm

Run LLMs with MLX

Stars: 7.1k
Forks: 1.1k
Open beginner issues: 0
Indexed issues: 149
Avg merge: 3d 6h
Merged PRs (30d): 54

Dominant language: Python
License: MIT
Last GitHub push: Sep 20, 2026
Latest indexed: Sep 19, 2026
Contributing guide: Contributing guide
Code of conduct: Code of conduct
Beginner labels: No beginner labels indexed

Beginner friendly Everything

Open All

148 open issues indexed

Missing general Knowledge Distillation paradigm Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

ml-explore/mlx-lm#1900 · Opened Sep 17, 2026
Can we add support for deepseek v4.1? Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

ml-explore/mlx-lm#1894 · 1 reaction · Opened Sep 16, 2026
mlx_lm.server: option to restrict which models a request may load (allow-list) Open

Difficulty 3/5 1-2 days Newbie friendliness 72/100

ml-explore/mlx-lm#1892 · Opened Sep 15, 2026
RuntimeError: There is no Stream(gpu, 3) in current thread at KVCache __del__ teardown Open

Difficulty 3/5 1-2 days Newbie friendliness 68/100

ml-explore/mlx-lm#1888 · Opened Sep 14, 2026
MiniCPM5-1B-MLX tool calling not supported by mlx_lm.server Open

Difficulty 4/5 3-5 days Newbie friendliness 55/100

ml-explore/mlx-lm#1885 · 2 reactions · Opened Sep 13, 2026
`phimoe.py` subscripts `rope_scaling` unconditionally, so `Phi-mini-MoE-instruct` dies with a bare `TypeError` Open

Difficulty 2/5 1-3 hours Newbie friendliness 76/100

ml-explore/mlx-lm#1884 · Opened Sep 13, 2026
`Lfm2MoeSparseMoeBlock` routing diverges from HF: softmax instead of sigmoid, `expert_bias` in the weights, and `routed_scaling_factor` dropped Open

Difficulty 3/5 1-2 days Newbie friendliness 74/100

ml-explore/mlx-lm#1883 · Opened Sep 13, 2026
Support for k2_horizon Request. Open

Difficulty 4/5 3-5 days Newbie friendliness 45/100

ml-explore/mlx-lm#1876 · Opened Sep 10, 2026
Missing support for Orion model architecture Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

ml-explore/mlx-lm#1874 · 1 comment · Opened Sep 10, 2026
ArraysCache metadata graph grows during sustained Qwen3.8-27B batch decoding Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

ml-explore/mlx-lm#1871 · Opened Sep 10, 2026
Paged KV storage and external cache integration Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

ml-explore/mlx-lm#1849 · 1 comment · Opened Sep 5, 2026
`ArraysCache.advance()` still leaks one Metal buffer per SSM layer per token through `BatchGenerator` after #1632 — long completions crash at ~499000/(#SSM−1) tokens on main Open

Difficulty 4/5 3-5 days Newbie friendliness 68/100

ml-explore/mlx-lm#1845 · Opened Sep 4, 2026
RFC: Streaming conversion - quantize checkpoints larger than local disk in `mlx_lm.convert` Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

ml-explore/mlx-lm#1842 · Opened Sep 4, 2026
mlx_lm.server deadlocks in mlx::core::eval_impl (condition_variable::wait) when the prompt cache is full under concurrent load — no Python exception Open

Difficulty 5/5 Over a week Newbie friendliness 28/100

ml-explore/mlx-lm#1834 · 1 reaction · Opened Sep 3, 2026
Long-running mlx_lm.server: memory grows unboundedly past prompt-cache cap on hybrid-attention models (Qwen3.8-27B 4-bit) Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

ml-explore/mlx-lm#1807 · Opened Aug 30, 2026
mlx_lm.server: per-request chat_template_kwargs bypasses the prompt cache (0.31.3) Open

Difficulty 3/5 1-2 days Newbie friendliness 72/100

ml-explore/mlx-lm#1803 · Opened Aug 29, 2026
mlx_lm.server: streamed generations do not store prompt-cache entries (0.31.3) Open

Difficulty 3/5 1-2 days Newbie friendliness 55/100

ml-explore/mlx-lm#1802 · Opened Aug 29, 2026
Missing support for meituan-longcat/LongCat-Flash-Lite-Sparse Open

Difficulty 3/5 1-2 days Newbie friendliness 48/100

ml-explore/mlx-lm#1794 · Opened Aug 27, 2026
feat: MXFP8 / fp8 mixed-precision training path (LoRA + full finetune) Open

Difficulty 5/5 Over a week Newbie friendliness 30/100

ml-explore/mlx-lm#1793 · Opened Aug 27, 2026
gated_delta: vectorized Metal kernel reads dt_bias per-head; callers (kimi_linear) pass per-element [H, head_dim] — GPU/CPU gate mismatch Open

Difficulty 2/5 1-3 hours Newbie friendliness 82/100

ml-explore/mlx-lm#1792 · Opened Aug 27, 2026
Serving a sliding-window model alongside another model corrupts the other model's BatchKVCache: silent degenerate outputs on cache hits, then _idx=-1 crash (deterministic repro, 3/3; negative control 200 rounds clean) Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

ml-explore/mlx-lm#1776 · Opened Aug 23, 2026
Pipeline parallelism support for qwen3_moe Open

enhancement

Difficulty 3/5 1-2 days Newbie friendliness 72/100

ml-explore/mlx-lm#1763 · Opened Aug 19, 2026
Distributed JACCL server crashes when two requests arrive concurrently Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

ml-explore/mlx-lm#1737 · 3 comments · Opened Aug 15, 2026
Code in the first example for Python SDK do not pass type check on load()'s returned values Open

Difficulty 2/5 1-3 hours Newbie friendliness 68/100

ml-explore/mlx-lm#1732 · Opened Aug 14, 2026
Fused Metal KL-divergence kernel overflows (−3.39e38) in backward pass during DWQ for models >~1B Open

Difficulty 4/5 3-5 days Newbie friendliness 55/100

ml-explore/mlx-lm#1700 · 2 comments · Opened Aug 10, 2026
gemma3n: KV-shared layers only reuse cached KV when a cache is present, so no-cache forward (LoRA training, evaluation) diverges from generation Open

Difficulty 3/5 1-2 days Newbie friendliness 68/100

ml-explore/mlx-lm#1674 · Opened Aug 7, 2026
mlx_lm.generate: repetition_penalty has no default (unlike other sampling params), model can loop into repeated text with no warning Open

Difficulty 3/5 1-2 days Newbie friendliness 58/100

ml-explore/mlx-lm#1669 · 2 comments · Opened Aug 7, 2026
completeMemory() prepare count underflow kernel panic recurs under continuous batching despite #883's KV-cache-bound mitigations Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

ml-explore/mlx-lm#1666 · 2 comments · 2 reactions · Opened Aug 5, 2026
LoRA trainer has no gradient clipping or non-finite guard: one bad batch loses the whole run Open

Difficulty 4/5 3-5 days Newbie friendliness 55/100

ml-explore/mlx-lm#1658 · 1 comment · Opened Aug 5, 2026
chat prompt emits raw readline markers (\x01/\x02) because readline is never imported Open

Difficulty 3/5 1-2 days Newbie friendliness 72/100

ml-explore/mlx-lm#1653 · Opened Aug 3, 2026
load() pays an avoidable Hub round-trip per call even when the model is fully cached Open

Difficulty 2/5 1-3 hours Newbie friendliness 78/100

ml-explore/mlx-lm#1649 · Opened Aug 3, 2026
Mistral Small 3.2 MLX does not emit tool calls with mlx_lm.server 0.31.3 Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

ml-explore/mlx-lm#1639 · 1 comment · Opened Jul 29, 2026
/v1/models commits 200 before scan_cache_dir(), hiding CacheNotFound Open

Difficulty 2/5 1-3 hours Newbie friendliness 72/100

ml-explore/mlx-lm#1636 · 1 comment · Opened Jul 29, 2026
save_model: 5 GB shards force one-shot eval of quantized weights, tripping the IOGPU watchdog (leaves a 0-byte shard) Open

Difficulty 3/5 1-2 days Newbie friendliness 76/100

ml-explore/mlx-lm#1633 · Opened Jul 29, 2026
make_prompt_cache(max_kv_size=N) sink tokens: policy is enforced in one place and dropped in another Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

ml-explore/mlx-lm#1631 · 3 comments · Opened Jul 28, 2026
Add toggle/disable for flawed model context "auto-fit" Open

Difficulty 4/5 3-5 days Newbie friendliness 45/100

ml-explore/mlx-lm#1610 · 1 comment · Opened Jul 23, 2026
Importing mlx_lm lowers RLIMIT_NOFILE and irreversibly caps the hard limit Open

Difficulty 2/5 1-3 hours Newbie friendliness 78/100

ml-explore/mlx-lm#1589 · Opened Jul 20, 2026
QuantizedKVCache shows higher peak memory than fp16 KVCache (worse at longer contexts) Open

Difficulty 4/5 3-5 days Newbie friendliness 42/100

ml-explore/mlx-lm#1587 · 11 comments · Opened Jul 18, 2026
KV-cache quantization unreachable in BatchGenerator (RotatingKVCache.to_quantized() unimplemented) Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

ml-explore/mlx-lm#1583 · Opened Jul 18, 2026
Bug: mlx-community/DeepSeek-Coder-V2-Lite-Instruct-4bit-AWQ does not support tool calling Open

Difficulty 3/5 1-2 days Newbie friendliness 48/100

ml-explore/mlx-lm#1574 · Opened Jul 16, 2026
RotatingKVCache.to_quantized() raises NotImplementedError — blocks --kv-bits on Gemma 4's sliding-window attention layers Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

ml-explore/mlx-lm#1573 · 10 comments · Opened Jul 16, 2026
One-shot `mx.eval(model.parameters())` of >300 GB models triggers IOGPU command-buffer watchdog timeout at load (crash; fixed by chunked eval) Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

ml-explore/mlx-lm#1572 · 2 comments · Opened Jul 15, 2026
Support Nemotron Puzzle 75B / nemotron_h_puzzle heterogeneous MoE models Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

ml-explore/mlx-lm#1506 · 2 comments · Opened Jul 9, 2026
`mlx_lm.server` pins a CPU core at ~100% while idle (generation worker busy-polls the request queue with `get_nowait()`) Open

Difficulty 2/5 1-3 hours Newbie friendliness 78/100

ml-explore/mlx-lm#1500 · 6 comments · Opened Jul 8, 2026
mlx_lm.server: generation hangs at 0% CPU right after prompt processing on ~22-26k-token streaming requests (gemma-4-26b-a4b, 0.31.3) Open

Difficulty 4/5 3-5 days Newbie friendliness 45/100

ml-explore/mlx-lm#1493 · 11 comments · Opened Jul 7, 2026
Model type gemma4_unified not supported Open

Difficulty 3/5 1-2 days Newbie friendliness 48/100

ml-explore/mlx-lm#1481 · 1 comment · Opened Jul 6, 2026
MLX/Metal OOM during long-context prefill for Qwen3.6/Qwen3.5 hybrid MoE despite only 10 KV-cache layers Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

ml-explore/mlx-lm#1480 · 2 comments · Opened Jul 6, 2026
Project status: what should contributors expect going forward? Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

ml-explore/mlx-lm#1475 · 1 comment · 9 reactions · Opened Jul 5, 2026
speculative_generate_step diverges from plain generation at temp=0 with num_draft_tokens=4 (dense model, no hybrid layers) Open

Difficulty 4/5 3-5 days Newbie friendliness 52/100

ml-explore/mlx-lm#1470 · 7 comments · Opened Jul 4, 2026
mlx_lm.server --draft-model fails to load any Qwen3.6 MTP checkpoint (model_type qwen3_5_mtp not registered) Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

ml-explore/mlx-lm#1462 · 1 comment · Opened Jul 4, 2026

A short digest of beginner-friendly GitHub issues.