ml-explore / ml-explore/mlx-examples

mlx_lm support for Bloom models

Open
#1,187 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Python
Stars
9k
Forks
1.2k
PR merge metrics
No merged PRs in 30d

Description

Thanks for building the mlx_lm library - it is absolutely great to work with this.

I was trying to work with this huggingface model: bigscience/bloom-560m (Link: https://huggingface.co/bigscience/bloom-560m) and while trying to convert using the mlx_lm.convert() command, I encountered this error:

Traceback (most recent call last):
  File "<path>/venv/lib/python3.10/site-packages/mlx_lm/utils.py", line 123, in _get_classes
    arch = importlib.import_module(f"mlx_lm.models.{model_type}")
  File "<path>/.pyenv/versions/3.10.13/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1004, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'mlx_lm.models.bloom'

Is there any plan to add support for bloom models?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the model-class lookup in mlx_lm/utils.py and the mlx_lm.convert command, then inspect the existing model modules to determine how Bloom support should fit. Use bigscience/bloom-560m as the target case; done means conversion no longer raises ModuleNotFoundError for mlx_lm.models.bloom.

Written by the indexing model from the issue text.

Assessment

Tech stack
huggingface, python
Domain
machine-learning
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.