cdpierse / cdpierse/transformers-interpret
Binary Classification: How is predicted label computed?
- Dominant language
- Jupyter Notebook
- Stars
- 1.4k
- Forks
- 99
- PR merge metrics
- No merged PRs in 30d
Description
Hi there,
I am observing the following (strange) behavior when using `pipeline` from the `transformers` library and `transformer-interpret`:
```
text = "Now Accord networks is a company in video, and he led the sales team, and the marketing group at Accord, and he took it from start up, sound familiar, it's from start up to $60 million company in two years."
classifier = pipeline('text-classification', model=model, tokenizer=tokenizer, device=0)
classifier(text)
```
```
[{'label': 'LABEL_1', 'score': 0.9711543321609497}]
```
while `transformer-interpret` gives me slightly different scores:
```
explainer = SequenceClassificationExplainer(model, tokenizer)
attributions = explainer(text)
html = explainer.visualize()
```

In both cases I apply the exact same `model` and `tokenizer`...
I am grateful for any hint and/or advice! 🤗
Contributor guide
Research direction
Start by reproducing the shown `pipeline` and `SequenceClassificationExplainer` calls with the same `model`, `tokenizer`, and text. Compare how each computes and reports the predicted label and score, then document or correct the discrepancy and verify the result against both outputs.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100