cdpierse / cdpierse/transformers-interpret

Issue using BertTokenizer (AttributeError)

Open
#119 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
1.4k
Forks
99
PR merge metrics
No merged PRs in 30d

Description

Hey there,

I'm trying to use your transofmers-interpret package, while using a bert-base-uncased model with the corresponding tokenizer. I'm loading the tokenizer using:
`tokenizer = BertTokenizer.from_pretrained( BERT_USED_MODEL_PATH, do_lower_case=True )`

When I use the interpreter, using:
`from transformers_interpret import SequenceClassificationExplainer
cls_explainer = SequenceClassificationExplainer(model=model,
tokenizer=tokenizer)`

I get an error:
`Traceback (most recent call last):
File "/home/khazmadoz/ML_GG/main.py", line 232, in
cls_explainer = SequenceClassificationExplainer(model=model,
File "/home/khazmadoz/anaconda3/lib/python3.9/site-packages/transformers_interpret/explainers/text/sequence_classification.py", line 53, in __init__
super().__init__(model, tokenizer)
File "/home/khazmadoz/anaconda3/lib/python3.9/site-packages/transformers_interpret/explainer.py", line 22, in __init__
self.ref_token_id = self.tokenizer.pad_token_id
AttributeError: 'BertTokenizer' object has no attribute 'pad_token_id'`

Do you know how to fix this? The tokenizer has an attribute 'pad_token' but no 'pad_token_id'.

Thank you,
David

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.