huggingface / huggingface/optimum-intel

Fail to execute quantized model on Ray

Open
#834 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
620
Forks
270
Avg merge
3d 10h
Merged PRs (30d)
22

Description

when execute the following code:
```python
quantized_model = IPEXModel.from_pretrained(self.model_path).eval()
model_ref = ray.put(quantized_model )
```

Error message:
```
RuntimeError: Tried to serialize object __torch__.intel_extension_for_pytorch.quantization._quantize_utils.___torch_mangle_972.QuantizationDispatchModule which does not have a __getstate__ method defined!
```
Is there any workaround to this problem?

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.