cdpierse / cdpierse/transformers-interpret
ZeroShotClassificationExplainer does not correctly explain ZeroShotClassificationPipeline results (single label)
- Dominant language
- Jupyter Notebook
- Stars
- 1.4k
- Forks
- 99
- PR merge metrics
- No merged PRs in 30d
Description
In the case of a single label, the logic to calculate the classification probability with the `ZeroShotClassificationExplainer` (see [here](https://github.com/cdpierse/transformers-interpret/blob/v0.6.0/transformers_interpret/explainers/sequence_classification.py#L193-L194)) is different than the logic in the Huggingface `ZeroShotClassificationPipeline` (see [here](https://github.com/huggingface/transformers/blob/v4.15.0/src/transformers/pipelines/zero_shot_classification.py#L249-L251)):
* The Huggingface `ZeroShotClassificationPipeline` calculates the softmax over entailment **and contradiction** scores and returns the resulting value for entailment, but
* the `ZeroShotClassificationExplainer` returns just the sigmoid of the entailment score.
At least, if this is intended, it should be documented somewhere. My usecase is multi-label classification and I used the single label approach to simulate that, but it took me some time to figure out that this does not work to explain `ZeroShotClassificationPipeline` predictions.
Contributor guide
Research direction
Start in transformers_interpret/explainers/sequence_classification.py at the cited lines, then compare its single-label probability calculation with the linked Hugging Face ZeroShotClassificationPipeline implementation. Confirm whether the explainer should match the pipeline or document the difference; done means the chosen behavior is verified and clearly covered by the project’s relevant checks.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100