cdpierse / cdpierse/transformers-interpret

[SEP] index is 1 smaller than it should be

Open
#77 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
1.4k
Forks
99
PR merge metrics
No merged PRs in 30d

Description

When using ZeroshotClassificationExplainer, the last token's value is not contained in outputs.

```
word_attributions = zeroshot_explainer(
text = "吾輩は猫である",
labels = ["文学"],
hypothesis_template = "この文は、{}に関するものである。",
)

print(word_attributions)

# the last token "ある" 's value is not contained in outputs
>> {'文学': [('[CLS]', 0.0), ('吾', -0.07784473169266609), ('##輩', -0.2678669763950776), ('は', -0.5911349137518062), ('猫', -0.6037091075576813), ('で', -0.45637956560324705)]}
```

I guess the reason is that [SEP] index is 1 smaller than it should be.
In ```ZeroshotClassificationExplainer._make_input_reference_pair```, L187 should be ```len(text_ids) + 1```, not ```len(text_ids)``` .

- https://github.com/cdpierse/transformers-interpret/blob/master/transformers_interpret/explainers/zero_shot_classification.py#L187

Contributor guide

Open the contributing guide

Research direction

Start in transformers_interpret/explainers/zero_shot_classification.py, specifically ZeroshotClassificationExplainer._make_input_reference_pair around line 187. Reproduce the issue with the Japanese text, label, and hypothesis template from the report, then verify that the output includes the final token "ある" and its attribution.

Written by the indexing model from the issue text.

Assessment

Tech stack
huggingface, python
Domain
machine-learning
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.