huggingface / huggingface/setfit
ONNX export for multilabel - discrepancy between pytorch and onnx outputs
- Dominant language
- Jupyter Notebook
- Stars
- 2.8k
- Forks
- 267
- Avg merge
- 36m
- Merged PRs (30d)
- 5
Description
Hi,
I was trying out exporting a multilabel setfit model with onnx and the exported model was giving different outputs as compared to the pytorch one. Am I doing something wrong in my inference code? Is multilabel onnx support not there at the moment?
Here is a [Colab](https://colab.research.google.com/drive/1dfR5lh1d0kJ0Sl5BuhtIclcqmJI8FEz9?usp=sharing) to recreate the issue.
EDIT: Adding code here too -
```
import setfit
from setfit import SetFitHead, SetFitHead, SetFitModel
from sentence_transformers import SentenceTransformer
import onnxruntime
from transformers import AutoTokenizer
from setfit.exporters.onnx import export_onnx
model_id = "sentence-transformers/paraphrase-mpnet-base-v2"
model_body = SentenceTransformer(model_id)
model_head = SetFitHead(in_features = model_body.get_sentence_embedding_dimension(), out_features = 4, multitarget = True)
model = SetFitModel(model_body = model_body, model_head = model_head, multi_target_strategy="multi-output")
sample_text = "this is a sample"
pytorch_pred = model([sample_text])
output_path = "setfit_onnx"
export_onnx(model.model_body,
model.model_head,
opset=12,
output_path=output_path)
tokenizer = AutoTokenizer.from_pretrained(model_id)
inputs = tokenizer(
sample_text,
padding=True,
truncation=True,
return_attention_mask=True,
return_token_type_ids=True,
return_tensors="np",
)
session = onnxruntime.InferenceSession(output_path)
onnx_preds = session.run(None, dict(inputs))[0]
print(onnx_preds)
print(pytorch_pred)
```
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.