huggingface / huggingface/optimum-executorch
Cannot export Voxtral to Metal backend
- Dominant language
- Python
- Stars
- 141
- Forks
- 48
- Avg merge
- 26m
- Merged PRs (30d)
- 2
Description
[Failed to export Voxtral to Metal #15621](https://github.com/pytorch/executorch/issues/15621)
## Incompatibility Issue
```
pip list | grep torch
executorch 1.0.0
optimum-executorch 0.1.0
pytorch_tokenizers 1.0.1
torch 2.9.0
torchao 0.14.0
```
With these versions of libraries, exporting `Voxtral` model with "metal" recipe doesn't work.
I found the main reason was that the [recipes/metal.py](https://github.com/huggingface/optimum-executorch/blob/cebeb3d43959cc977893ac7fe7975169e30f66b6/optimum/exporters/executorch/recipes/metal.py#L24) requirement for Metal backend was Executorch v1.1.0, which is not compatible with v1.0 for now.
Error messages:
```
optimum-cli export executorch \
--model "mistralai/Voxtral-Mini-3B-2507" \
--task "multimodal-text-to-text" \
--recipe "metal" \
--dtype bfloat16 \
--max_seq_len 1024 \
--output_dir="voxtral"
Traceback (most recent call last):
File "/Users/younghan/miniconda3/envs/executorch290/bin/optimum-cli", line 7, in
sys.exit(main())
^^^^^^
File "/Users/younghan/miniconda3/envs/executorch290/lib/python3.12/site-packages/optimum/commands/optimum_cli.py", line 208, in main
service.run()
File "/Users/younghan/miniconda3/envs/executorch290/lib/python3.12/site-packages/optimum/commands/export/executorch.py", line 230, in run
main_export(
File "/Users/younghan/miniconda3/envs/executorch290/lib/python3.12/site-packages/optimum/exporters/executorch/__main__.py", line 145, in main_export
return export_to_executorch(
^^^^^^^^^^^^^^^^^^^^^
File "/Users/younghan/miniconda3/envs/executorch290/lib/python3.12/site-packages/optimum/exporters/executorch/convert.py", line 82, in export_to_executorch
executorch_progs = recipe_func(model, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: 'NoneType' object is not callable
```
So, I upgraded the `executorch` to 1.1.0.dev20251105, but got different errors.
```
pip list | grep torch
executorch 1.1.0.dev20251105
optimum-executorch 0.2.0.dev0
pytorch_tokenizers 1.0.1
torch 2.10.0.dev20251104
torchao 0.15.0.dev20251104+cpu
torchaudio 2.10.0.dev20251104
torchvision 0.25.0.dev20251104
```
Error messages:
```
optimum-cli export executorch \
--model "mistralai/Voxtral-Mini-3B-2507" \
--task "multimodal-text-to-text" \
--recipe "metal" \
--dtype bfloat16 \
--max_seq_len 1024 \
--output_dir="voxtral"
...
torch._inductor.exc.InductorError: TypeError: collect_unsupported_fallback_kernels..generate_c_shim_extern_kernel_call_and_collect_unsupported_kernels() got an unexpected keyword argument 'stack_traces'
```
Contributor guide
Assessment
This issue has not been assessed yet.