lmstudio-ai / lmstudio-ai/mlx-engine

Removing external code when models are upstreamed into Transformers

Open
#211 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.2k
Forks
133
Avg merge
21h 6m
Merged PRs (30d)
1

Description

As of writing, we keep Ernie 4.5/4.5 MoE code in `external` and tell the engine to use this code in `utils/register_models.py`. This pattern was introduced in #185. However, as of Transformers 4.54.0, these models now exist upstream in Transformers, resulting in an error when we try to overwrite the registration. #209 solves this by setting `exist_ok=True` for upstreamed models, but the need for this workaround should really serve as an indication that external code *should be removed*.

Contributor guide

Open the contributing guide

Research direction

Start with the Ernie 4.5/4.5 MoE code under `external` and its registration in `utils/register_models.py`, then trace the pattern introduced by #185. Done means the upstreamed model code and registration are removed, so the workaround from #209 is no longer needed.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Refactor
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.