cdpierse / cdpierse/transformers-interpret
Issue using BertTokenizer (AttributeError)
- Dominant language
- Jupyter Notebook
- Stars
- 1.4k
- Forks
- 99
- PR merge metrics
- No merged PRs in 30d
Description
Hey there,
I'm trying to use your transofmers-interpret package, while using a bert-base-uncased model with the corresponding tokenizer. I'm loading the tokenizer using:
`tokenizer = BertTokenizer.from_pretrained( BERT_USED_MODEL_PATH, do_lower_case=True )`
When I use the interpreter, using:
`from transformers_interpret import SequenceClassificationExplainer
cls_explainer = SequenceClassificationExplainer(model=model,
tokenizer=tokenizer)`
I get an error:
`Traceback (most recent call last):
File "/home/khazmadoz/ML_GG/main.py", line 232, in
cls_explainer = SequenceClassificationExplainer(model=model,
File "/home/khazmadoz/anaconda3/lib/python3.9/site-packages/transformers_interpret/explainers/text/sequence_classification.py", line 53, in __init__
super().__init__(model, tokenizer)
File "/home/khazmadoz/anaconda3/lib/python3.9/site-packages/transformers_interpret/explainer.py", line 22, in __init__
self.ref_token_id = self.tokenizer.pad_token_id
AttributeError: 'BertTokenizer' object has no attribute 'pad_token_id'`
Do you know how to fix this? The tokenizer has an attribute 'pad_token' but no 'pad_token_id'.
Thank you,
David
Contributor guide
Assessment
This issue has not been assessed yet.