lmstudio-ai / lmstudio-ai/mlx-engine

Draft-only (MTP) repos indexed as loadable models, then crash at generation: 'Qwen3_5MTPDraftModel' object has no attribute 'get_input_embeddings'

Open
#371 1 comment 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
1.2k
Forks
133
Avg merge
21h 6m
Merged PRs (30d)
1

Description

## Environment

- LM Studio's bundled MLX engine (observed on `app-mlx-generate` @34, macOS)
- Model: `mlx-community/Qwen3.8-27B-MTP-4bit` (draft-only / MTP repo)

## Observed behavior

The draft-only MTP repo is indexed as if it were a normal runnable model:

1. It appears in the model catalog.
2. It **loads successfully**.
3. **Every** generation request then crashes deterministically with:

```
AttributeError: 'Qwen3_5MTPDraftModel' object has no attribute 'get_input_embeddings'
```

## Expected behavior

Either support MTP drafting, or refuse to index/load draft-only repos with a
clear error **at load time** instead of a generation-time traceback.

## Impact

Downstream users select these repos believing they are runnable base models;
the failure surfaces only mid-session, after load "succeeded."

Contributor guide

Open the contributing guide

Research direction

Start by tracing how model repositories are indexed and loaded, then reproduce the failure with mlx-community/Qwen3.8-27B-MTP-4bit. Determine whether draft-only MTP repositories should be supported or rejected during loading, and verify that the chosen behavior prevents a generation-time get_input_embeddings traceback.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
65/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.