huggingface / huggingface/optimum-intel
Not able to load Intel/bge-base-en-v1.5-rag-int8-static model
- Dominant language
- Jupyter Notebook
- Stars
- 620
- Forks
- 270
- Avg merge
- 3d 10h
- Merged PRs (30d)
- 22
Description
```
from optimum.intel import IPEXModel
model_path = "/data/notebook/Dilip/models/bge-base-en-v1.5-rag-int8-static"
model = IPEXModel.from_pretrained(model_path)
```
Error:
```
Passing the argument `library_name` to `get_supported_tasks_for_model_type` is required, but got library_name=None. Defaulting to `transformers`. An error will be raised in a future version of Optimum if `library_name` is not provided.
/data/notebook/Dilip/virtualenv/intel-env/lib/python3.11/site-packages/torch/amp/autocast_mode.py:267: UserWarning: In CPU autocast, but the target dtype is not supported. Disabling autocast.
CPU Autocast only supports dtype of torch.bfloat16, torch.float16 currently.
warnings.warn(error_message)
Traceback (most recent call last):
File "/data/notebook/Dilip/virtualenv/intel-env/lib/python3.11/site-packages/optimum/intel/ipex/modeling_base.py", line 317, in _call_model
out = self.model(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/notebook/Dilip/virtualenv/intel-env/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/notebook/Dilip/virtualenv/intel-env/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: The following operation failed in the TorchScript interpreter.
Traceback of TorchScript (most recent call last):
graph(%add_a, %add_b, %alpha, %shape:int[], %w, %b, %eps:float, %cudnn_enable:bool):
%r = ipex::add_layernorm(%add_a, %add_b, %alpha, %shape, %w, %b, %eps, %cudnn_enable)
~~~~ <--- HERE
return (%r)
RuntimeError: could not create a primitive descriptor for a layer normalization forward propagation primitive
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "", line 1, in
File "/data/notebook/Dilip/virtualenv/intel-env/lib/python3.11/site-packages/optimum/modeling_base.py", line 402, in from_pretrained
return from_pretrained_method(
^^^^^^^^^^^^^^^^^^^^^^^
File "/data/notebook/Dilip/virtualenv/intel-env/lib/python3.11/site-packages/optimum/intel/ipex/modeling_base.py", line 258, in _from_pretrained
return cls(model, config=config, model_save_dir=model_save_dir, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/notebook/Dilip/virtualenv/intel-env/lib/python3.11/site-packages/optimum/intel/ipex/modeling_base.py", line 152, in __init__
self._init_warmup()
File "/data/notebook/Dilip/virtualenv/intel-env/lib/python3.11/site-packages/optimum/intel/ipex/modeling_base.py", line 332, in _init_warmup
self(**dummy_inputs)
File "/data/notebook/Dilip/virtualenv/intel-env/lib/python3.11/site-packages/optimum/modeling_base.py", line 92, in __call__
return self.forward(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/notebook/Dilip/virtualenv/intel-env/lib/python3.11/site-packages/optimum/intel/ipex/modeling_base.py", line 279, in forward
outputs = self._call_model(**inputs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/notebook/Dilip/virtualenv/intel-env/lib/python3.11/site-packages/optimum/intel/ipex/modeling_base.py", line 319, in _call_model
out = self.model(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/notebook/Dilip/virtualenv/intel-env/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/notebook/Dilip/virtualenv/intel-env/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
**RuntimeError: The following operation failed in the TorchScript interpreter.
Traceback of TorchScript (most recent call last):
RuntimeError: could not create a primitive descriptor for a softmax forward propagation primitive**
```
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.