huggingface / huggingface/optimum-intel

Not able to load Intel/bge-base-en-v1.5-rag-int8-static model

Open
#731 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
620
Forks
270
Avg merge
3d 10h
Merged PRs (30d)
22

Description

```
from optimum.intel import IPEXModel
model_path = "/data/notebook/Dilip/models/bge-base-en-v1.5-rag-int8-static"
model = IPEXModel.from_pretrained(model_path)
```

Error:

```
Passing the argument `library_name` to `get_supported_tasks_for_model_type` is required, but got library_name=None. Defaulting to `transformers`. An error will be raised in a future version of Optimum if `library_name` is not provided.
/data/notebook/Dilip/virtualenv/intel-env/lib/python3.11/site-packages/torch/amp/autocast_mode.py:267: UserWarning: In CPU autocast, but the target dtype is not supported. Disabling autocast.
CPU Autocast only supports dtype of torch.bfloat16, torch.float16 currently.
warnings.warn(error_message)
Traceback (most recent call last):
File "/data/notebook/Dilip/virtualenv/intel-env/lib/python3.11/site-packages/optimum/intel/ipex/modeling_base.py", line 317, in _call_model
out = self.model(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/notebook/Dilip/virtualenv/intel-env/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/notebook/Dilip/virtualenv/intel-env/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: The following operation failed in the TorchScript interpreter.
Traceback of TorchScript (most recent call last):

graph(%add_a, %add_b, %alpha, %shape:int[], %w, %b, %eps:float, %cudnn_enable:bool):
%r = ipex::add_layernorm(%add_a, %add_b, %alpha, %shape, %w, %b, %eps, %cudnn_enable)
~~~~ <--- HERE
return (%r)
RuntimeError: could not create a primitive descriptor for a layer normalization forward propagation primitive

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "", line 1, in
File "/data/notebook/Dilip/virtualenv/intel-env/lib/python3.11/site-packages/optimum/modeling_base.py", line 402, in from_pretrained
return from_pretrained_method(
^^^^^^^^^^^^^^^^^^^^^^^
File "/data/notebook/Dilip/virtualenv/intel-env/lib/python3.11/site-packages/optimum/intel/ipex/modeling_base.py", line 258, in _from_pretrained
return cls(model, config=config, model_save_dir=model_save_dir, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/notebook/Dilip/virtualenv/intel-env/lib/python3.11/site-packages/optimum/intel/ipex/modeling_base.py", line 152, in __init__
self._init_warmup()
File "/data/notebook/Dilip/virtualenv/intel-env/lib/python3.11/site-packages/optimum/intel/ipex/modeling_base.py", line 332, in _init_warmup
self(**dummy_inputs)
File "/data/notebook/Dilip/virtualenv/intel-env/lib/python3.11/site-packages/optimum/modeling_base.py", line 92, in __call__
return self.forward(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/notebook/Dilip/virtualenv/intel-env/lib/python3.11/site-packages/optimum/intel/ipex/modeling_base.py", line 279, in forward
outputs = self._call_model(**inputs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/notebook/Dilip/virtualenv/intel-env/lib/python3.11/site-packages/optimum/intel/ipex/modeling_base.py", line 319, in _call_model
out = self.model(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/notebook/Dilip/virtualenv/intel-env/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/notebook/Dilip/virtualenv/intel-env/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
**RuntimeError: The following operation failed in the TorchScript interpreter.
Traceback of TorchScript (most recent call last):
RuntimeError: could not create a primitive descriptor for a softmax forward propagation primitive**
```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.