cdpierse / cdpierse/transformers-interpret

ZeroShotClassificationExplainer does not correctly explain ZeroShotClassificationPipeline results (single label)

Open
#84 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
1.4k
Forks
99
PR merge metrics
No merged PRs in 30d

Description

In the case of a single label, the logic to calculate the classification probability with the `ZeroShotClassificationExplainer` (see [here](https://github.com/cdpierse/transformers-interpret/blob/v0.6.0/transformers_interpret/explainers/sequence_classification.py#L193-L194)) is different than the logic in the Huggingface `ZeroShotClassificationPipeline` (see [here](https://github.com/huggingface/transformers/blob/v4.15.0/src/transformers/pipelines/zero_shot_classification.py#L249-L251)):
* The Huggingface `ZeroShotClassificationPipeline` calculates the softmax over entailment **and contradiction** scores and returns the resulting value for entailment, but
* the `ZeroShotClassificationExplainer` returns just the sigmoid of the entailment score.

At least, if this is intended, it should be documented somewhere. My usecase is multi-label classification and I used the single label approach to simulate that, but it took me some time to figure out that this does not work to explain `ZeroShotClassificationPipeline` predictions.

Contributor guide

Open the contributing guide

Research direction

Start in transformers_interpret/explainers/sequence_classification.py at the cited lines, then compare its single-label probability calculation with the linked Hugging Face ZeroShotClassificationPipeline implementation. Confirm whether the explainer should match the pipeline or document the difference; done means the chosen behavior is verified and clearly covered by the project’s relevant checks.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.