huggingface / huggingface/optimum-executorch

Cannot export Voxtral to Metal backend

Open
#181 5 comments 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
141
Forks
48
Avg merge
26m
Merged PRs (30d)
2

Description

[Failed to export Voxtral to Metal #15621](https://github.com/pytorch/executorch/issues/15621)

## Incompatibility Issue

```
pip list | grep torch
executorch 1.0.0
optimum-executorch 0.1.0
pytorch_tokenizers 1.0.1
torch 2.9.0
torchao 0.14.0
```

With these versions of libraries, exporting `Voxtral` model with "metal" recipe doesn't work.

I found the main reason was that the [recipes/metal.py](https://github.com/huggingface/optimum-executorch/blob/cebeb3d43959cc977893ac7fe7975169e30f66b6/optimum/exporters/executorch/recipes/metal.py#L24) requirement for Metal backend was Executorch v1.1.0, which is not compatible with v1.0 for now.

Error messages:
```
optimum-cli export executorch \
--model "mistralai/Voxtral-Mini-3B-2507" \
--task "multimodal-text-to-text" \
--recipe "metal" \
--dtype bfloat16 \
--max_seq_len 1024 \
--output_dir="voxtral"

Traceback (most recent call last):
File "/Users/younghan/miniconda3/envs/executorch290/bin/optimum-cli", line 7, in
sys.exit(main())
^^^^^^
File "/Users/younghan/miniconda3/envs/executorch290/lib/python3.12/site-packages/optimum/commands/optimum_cli.py", line 208, in main
service.run()
File "/Users/younghan/miniconda3/envs/executorch290/lib/python3.12/site-packages/optimum/commands/export/executorch.py", line 230, in run
main_export(
File "/Users/younghan/miniconda3/envs/executorch290/lib/python3.12/site-packages/optimum/exporters/executorch/__main__.py", line 145, in main_export
return export_to_executorch(
^^^^^^^^^^^^^^^^^^^^^
File "/Users/younghan/miniconda3/envs/executorch290/lib/python3.12/site-packages/optimum/exporters/executorch/convert.py", line 82, in export_to_executorch
executorch_progs = recipe_func(model, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: 'NoneType' object is not callable
```

So, I upgraded the `executorch` to 1.1.0.dev20251105, but got different errors.

```
pip list | grep torch
executorch 1.1.0.dev20251105
optimum-executorch 0.2.0.dev0
pytorch_tokenizers 1.0.1
torch 2.10.0.dev20251104
torchao 0.15.0.dev20251104+cpu
torchaudio 2.10.0.dev20251104
torchvision 0.25.0.dev20251104
```

Error messages:
```
optimum-cli export executorch \
--model "mistralai/Voxtral-Mini-3B-2507" \
--task "multimodal-text-to-text" \
--recipe "metal" \
--dtype bfloat16 \
--max_seq_len 1024 \
--output_dir="voxtral"

...

torch._inductor.exc.InductorError: TypeError: collect_unsupported_fallback_kernels..generate_c_shim_extern_kernel_call_and_collect_unsupported_kernels() got an unexpected keyword argument 'stack_traces'
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.