cdpierse / cdpierse/transformers-interpret

Support for Reformer

Open
#139 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
1.4k
Forks
99
PR merge metrics
No merged PRs in 30d

Description

Even though ReformerForSequenceClassification follows the "{MODEL_NAME}ForSequenceClassification" pattern as indicated in the documentation, I end up with a RuntimeError as follows. Since it's so simple to use the Explainers, I'm assuming Reformer models aren't supported yet.

Here's the last frame in the trace in the Explainer code:
File ~/.local/lib/python3.10/site-packages/transformers_interpret/explainer.py:197, in BaseExplainer._get_preds(self, input_ids, token_type_ids, position_ids, attention_mask)
194 return preds
196 elif self.accepts_position_ids:
--> 197 preds = self.model(
198 input_ids=input_ids,
199 position_ids=position_ids,
200 attention_mask=attention_mask,
201 )
203 return preds
204 elif self.accepts_token_type_ids:

Here's the last frame where error occured:
File ~/.local/lib/python3.10/site-packages/transformers/models/reformer/modeling_reformer.py:2174, in ReformerModel._pad_to_mult_of_chunk_length(self, input_ids, inputs_embeds, attention_mask, position_ids, input_shape, padding_length, padded_seq_length, device)
2172 # Pad position ids if given
2173 if position_ids is not None:
-> 2174 padded_position_ids = torch.arange(input_shape[-1], padded_seq_length, dtype=torch.long, device=device)
2175 padded_position_ids = position_ids.unsqueeze(0).expand(input_shape[0], padding_length)
2176 position_ids = torch.cat([position_ids, padded_position_ids], dim=-1)

RuntimeError: upper bound and larger bound inconsistent with step sign

I received the exact same error and stack trace with both SequenceClassificationExplainer and MultiLabelClassificationExplainer which I tried because I am trying to do multi-label text classification.

Please let me know if you need the full trace or any other information. Thank you.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.