lmstudio-ai / lmstudio-ai/mlx-engine
New MTP models not supported
- Dominant language
- Python
- Stars
- 1.2k
- Forks
- 133
- Avg merge
- 21h 6m
- Merged PRs (30d)
- 1
Description
Post from Google about new MTP models: https://blog.google/innovation-and-ai/technology/developers-tools/multi-token-prediction-gemma-4/
HF link to models: https://huggingface.co/collections/mlx-community/gemma-4-assistant-mtp
Simple script to download and prompt the model:
```
from mlx_lm import load, generate
model, tokenizer = load("mlx-community/gemma-4-31B-it-assistant-bf16")
messages = [{"role": "user", "content": "Explain quantum entanglement simply."}]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)
response = generate(model, tokenizer, prompt=prompt, max_tokens=512, verbose=True)
```
Resulting error: ValueError: Model type gemma4_assistant not supported.
Contributor guide
Assessment
This issue has not been assessed yet.