huggingface / huggingface/setfit
Inconsistent onnx-optimum output
- Dominant language
- Jupyter Notebook
- Stars
- 2.8k
- Forks
- 267
- Avg merge
- 36m
- Merged PRs (30d)
- 5
Description
Hi, I am currently working on an optimization pipeline for a SetFit model.
As I'm testing different approaches, I've tried the process shown in the notebook: https://github.com/huggingface/setfit/blob/58a3600d2764e9c815594bec57812c9c2408931a/notebooks/setfit-onnx-optimum.ipynb
In this section:
```python
!optimum-cli export onnx \
--model moshew/bge-small-en-v1.5_setfit-sst2-english \
--task feature-extraction \
--optimize O4 \
--device cuda \
bge_auto_opt_O4
```
It shows:
```
- Validating ONNX Model output "last_hidden_state":
-[✓] (2, 16, 384) matches (2, 16, 384)
-[x] values not close enough, max diff: 2.1155929565429688 (atol: 0.0001)
```
But, If i try to replicate the same exact notebook, model output shown are "token_embeddings" and "sentence_embeddings".
Does anyone has an explanation for this? Thanks
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.