huggingface / huggingface/setfit

Inconsistent onnx-optimum output

Open
#467 2 comments 1 reaction 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
2.8k
Forks
267
Avg merge
36m
Merged PRs (30d)
5

Description

Hi, I am currently working on an optimization pipeline for a SetFit model.

As I'm testing different approaches, I've tried the process shown in the notebook: https://github.com/huggingface/setfit/blob/58a3600d2764e9c815594bec57812c9c2408931a/notebooks/setfit-onnx-optimum.ipynb

In this section:

```python

!optimum-cli export onnx \
--model moshew/bge-small-en-v1.5_setfit-sst2-english \
--task feature-extraction \
--optimize O4 \
--device cuda \
bge_auto_opt_O4

```

It shows:

```
- Validating ONNX Model output "last_hidden_state":
-[✓] (2, 16, 384) matches (2, 16, 384)
-[x] values not close enough, max diff: 2.1155929565429688 (atol: 0.0001)
```

But, If i try to replicate the same exact notebook, model output shown are "token_embeddings" and "sentence_embeddings".
Does anyone has an explanation for this? Thanks

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.