Issue with Fusing Models - Output is Bad

Hi,

When I train my model using the code below (Note this was all done in jupyter Notebook).

!python -m mlx_lm.lora \
    --model mlx-community/Meta-Llama-3-8B-Instruct-4bit \
    --train \
    --batch-size 1 \
    --lora-layers 1 \
    --iters 1000 \
    --data Data \
    --seed 0

And load the model using the load function where I pass the adapters path.

# Load the fine-tuned model with LoRA weights
model_lora, tokenizer_lora = load(
    path_or_hf_repo="mlx-community/Meta-Llama-3-8B-Instruct-4bit", 
    adapter_path="adapters"
)

I get nicely generated outputs able to solve annoyingly verbose math problems. Like so (I called my model LlaMATH).

However when fusing using the code below and importing the fused model from either hugging face or locally the outputs are bad.

!python -m mlx_lm.fuse \
    --model mlx-community/Meta-Llama-3-8B-Instruct-4bit \
    --adapter-path adapters \
    --upload-repo MyRepo/LlaMATH-3-8B-Instruct-4bit \
    --hf-path mlx-community/Meta-Llama-3-8B-Instruct-4bit

Using the fused model

# Can also load the fused model locally
fused_model, fused_tokenizer = load("./lora_fused_model/")

# Running model with test question
response = generate(
    fused_model,
    fused_tokenizer,
    prompt=prompt,
    max_tokens=100,
    temp=0.0, 
    verbose=False
)

This will give me some nonsense. Any help would be great. I am using python 3.11, and mlx_lm.version == 0.12.1. Thank you and I appreciate any advice or help! :D

Start with the mlx_lm.fuse entry point and compare its output with the adapter-loaded model from load(..., adapter_path="adapters"). Reproduce the report using Python 3.11, mlx_lm 0.12.1, the shown model, adapter path, and prompt. Done means the locally loaded or uploaded fused model produces the same useful output as the LoRA-loaded model.