huggingface / huggingface/optimum-intel

Gemma 3n support

Open
#1,419 0 comments 2 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
620
Forks
270
Avg merge
4d 47m
Merged PRs (30d)
25

Description

Trying to open a Gemma 3n model results in an error:
```python
model = OVModelForCausalLM.from_pretrained("google/gemma-3n-e4b-it", device_map="auto")
No OpenVINO files were found for google/gemma-3n-e4b-it, setting `export=True` to convert the model to the OpenVINO IR. Don't forget to save the resulting model with `.save_pretrained()`
Traceback (most recent call last):
File "", line 1, in
File "/home/paris/.pyenv/versions/npu/lib/python3.12/site-packages/optimum/intel/openvino/modeling_base.py", line 505, in from_pretrained
return super().from_pretrained(
^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/paris/.pyenv/versions/npu/lib/python3.12/site-packages/optimum/modeling_base.py", line 419, in from_pretrained
return from_pretrained_method(
^^^^^^^^^^^^^^^^^^^^^^^
File "/home/paris/.pyenv/versions/npu/lib/python3.12/site-packages/optimum/intel/openvino/modeling_decoder.py", line 347, in _export
main_export(
File "/home/paris/.pyenv/versions/npu/lib/python3.12/site-packages/optimum/exporters/openvino/__main__.py", line 267, in main_export
raise ValueError(
ValueError: Trying to export a gemma3n model, that is a custom or unsupported architecture, but no custom export configuration was passed as `custom_export_configs`. Please refer to https://huggingface.co/docs/optimum/main/en/exporters/onnx/usage_guides/export_a_model#custom-export-of-transformers-models for an example on how to export custom models. Please open an issue at https://github.com/huggingface/optimum-intel/issues if you would like the model type gemma3n to be supported natively in the OpenVINO export.
```

The same error occurs when trying to export a Gemma 3n model with `optimum-cli`.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the export path reached by OVModelForCausalLM.from_pretrained and the equivalent optimum-cli flow, focusing on the reported custom_export_configs handling for the gemma3n architecture. Verify that a Gemma 3n model can be exported and then loaded without the unsupported-architecture error; the issue's Python traceback provides the initial entry points.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.