ml-explore / ml-explore/mlx-examples

Unable to convert CogVLM due to the model not existing.

Open
#695 1 comment 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
9k
Forks
1.2k
PR merge metrics
No merged PRs in 30d

Description

I am experiencing issues converting the cogvlm model from Hugging Face to MLX format. The process fails due to an unsupported model type. Enhanced logging and documentation on adding new models would be beneficial.

Environment
  • Operating System:
  • Python Version:
  • MLX Version:
Steps to Reproduce
  1. Attempt to convert the cogvlm model with the following command:
    python -m mlx_lm.convert \
        --hf-path THUDM/cogvlm-chat-hf \
        -q \
        --upload-repo raoulritter/cogvlm-mlx
    
  2. Observe the error indicating the model type is unsupported.
Expected Behavior

The conversion process should identify the model type and either convert it successfully or provide detailed reasons why the conversion cannot be performed.

Actual Behavior

The process terminates with an unsupported model type error, without detailed debugging information or guidance on how to resolve or work around the issue.

Possible Solutions
  • Verbose Logging: Implement more detailed logging during the conversion process to help users diagnose issues related to model compatibility.
  • Model Support Documentation: Provide detailed guidelines or examples on how to add support for new models in mlx.models, similar to how the llava model support was added (referenced in issue #461).

Thank you for your help. I look forward to broader model compatibility in MLX.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the mlx_lm.convert entry point and the mlx.models model-support guidance, then run the documented CogVLM conversion command to reproduce the unsupported-model error. Done means the issue has either a supported conversion path or clear diagnostic output and guidance explaining the limitation.

Written by the indexing model from the issue text.

Assessment

Tech stack
huggingface, python
Domain
machine-learning, tooling
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.