huggingface / huggingface/optimum-executorch

Missing out variants: {'torchao::dequantize_affine'}

Open
#182 4 comments 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
141
Forks
48
Avg merge
26m
Merged PRs (30d)
2

Description

My optimum convertion looks like...
```
model_id = "meta-llama/Llama-3.2-1B-Instruct"
model = ExecuTorchModelForCausalLM.from_pretrained(
model_id,
export=True,
recipe="xnnpack",
attn_implementation="custom_sdpa",
use_custom_kv_cache=True,
**{"qlinear": "8w", "qlinear_encoder": "4w", "qembedding": "4w"},
)
generated_text = model.text_generation(
tokenizer=tokenizer,
prompt=prompt,
max_seq_len=512,
)
```
This error comes for Llama3.2-1B-Instruct, Llama3.2-3B-Instruct when trying with **--qlinear is set to '4w' or '8w'**. But it works fine for '8da4w' irrespective of --qlinear_encoder, --qembedding parameters.

**Error:**
```
Exception has occurred: RuntimeError (note: full exception trace is shown but execution is paused at: _run_module_as_main)
Missing out variants: {'torchao::dequantize_affine'}
File "/home/user/Desktop/my_code/all_envs/optimum_env/lib/python3.10/site-packages/executorch/exir/passes/__init__.py", line 435, in call
raise RuntimeError(f"Missing out variants: {missing_out_vars}")
File "/home/user/Desktop/my_code/all_envs/optimum_env/lib/python3.10/site-packages/torch/fx/passes/infra/pass_base.py", line 46, in __call__
res = self.call(graph_module)
File "/home/user/Desktop/my_code/all_envs/optimum_env/lib/python3.10/site-packages/executorch/exir/program/_program.py", line 1716, in to_executorch
new_gm_res = p(new_gm)
File "/home/user/Desktop/my_code/all_envs/optimum_env/lib/python3.10/site-packages/executorch/exir/program/_program.py", line 114, in wrapper
return func(*args, **kwargs)
File "/home/user/Desktop/my_code/all_envs/optimum_env/lib/python3.10/site-packages/optimum/exporters/executorch/recipes/xnnpack.py", line 93, in _lower_to_executorch
et_prog = et_prog.to_executorch(
File "/home/user/Desktop/my_code/all_envs/optimum_env/lib/python3.10/site-packages/optimum/exporters/executorch/recipes/xnnpack.py", line 121, in export_to_executorch_with_xnnpack
return _lower_to_executorch(exported_progs, model.metadata)
File "/home/user/Desktop/my_code/all_envs/optimum_env/lib/python3.10/site-packages/optimum/exporters/executorch/convert.py", line 80, in export_to_executorch
executorch_progs = recipe_func(model, **kwargs)
File "/home/user/Desktop/my_code/all_envs/optimum_env/lib/python3.10/site-packages/optimum/exporters/executorch/__main__.py", line 145, in main_export
return export_to_executorch(
File "/home/user/Desktop/my_code/all_envs/optimum_env/lib/python3.10/site-packages/optimum/executorch/modeling.py", line 287, in _export
executorch_progs = main_export(
File "/home/user/Desktop/my_code/all_envs/optimum_env/lib/python3.10/site-packages/optimum/executorch/modeling.py", line 422, in from_pretrained
models_dict, temp_dir = cls._export(
File "/home/user/Desktop/my_code/all_envs/optimum_env/optimum_pyscript.py", line 128, in main
model = ExecuTorchModelForCausalLM.from_pretrained(
File "/home/user/Desktop/my_code/all_envs/optimum_env/optimum_pyscript.py", line 170, in
main()
File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main (Current frame)
return _run_code(code, main_globals, None,
RuntimeError: Missing out variants: {'torchao::dequantize_affine'}
```

Can the respective team support this and help us resolve the error.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.