libc++abi: terminating due to uncaught exception of type std::runtime_error

Training Gemma encountered the following exception, but training Mistral did not.

python -m mlx_lm.lora \
    --model google/gemma-2b-it \
    --train \
    --data /Users/danny/mlx_demo/data \
    --iters 600 --adapter-path /Users/danny/mlx_demo/models/gemma

Loading pretrained model
Fetching 9 files: 100%|████████████████████████████████████████████████████████| 9/9 [00:00<00:00, 112347.43it/s]
Trainable parameters: 0.033% (0.819M/2506.172M)
Loading datasets
Training
Starting training..., iters: 600
libc++abi: terminating due to uncaught exception of type std::runtime_error: [METAL] Command buffer execution failed: Internal Error (0000000e:Internal Error)
[1]    72864 abort      python -m mlx_lm.lora --model google/gemma-2b-it --train --data  --iters 600
/opt/homebrew/Cellar/python@3.12/3.12.2_1/Frameworks/Python.framework/Versions/3.12/lib/python3.12/multiprocessing/resource_tracker.py:254: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
  warnings.warn('resource_tracker: There appear to be %d '

certifi==2024.2.2
charset-normalizer==3.3.2
filelock==3.13.3
fsspec==2024.3.1
huggingface-hub==0.22.2
idna==3.6
jinja2==3.1.3
markupsafe==2.1.5
mlx==0.9.1
mlx-lm==0.7.0
mpmath==1.3.0
networkx==3.3
numpy==1.26.4
packaging==24.0
protobuf==5.26.1
pyyaml==6.0.1
regex==2023.12.25
requests==2.31.0
safetensors==0.4.2
sympy==1.12
tokenizers==0.15.2
torch==2.2.2
tqdm==4.66.2
transformers==4.39.3
typing-extensions==4.11.0
urllib3==2.2.1

Start by reproducing the python -m mlx_lm.lora Gemma training command on the reported environment, using the listed package versions and the /Users/danny/mlx_demo/data dataset. Compare it with the successful Mistral training path and inspect the point where training reaches the Metal command-buffer failure; done means Gemma training completes without the uncaught runtime error or its cause is documented with a confirmed workaround.