Azure / Azure/Azure-Sentinel-Notebooks
Mitremap-notebook: Error - Gpt2tokenizer object has no attribute _added_tokens_decoder
Open
- Dominant language
- Jupyter Notebook
- Stars
- 628
- Forks
- 236
- PR merge metrics
- No merged PRs in 30d
Description
When I run the below mentioned code I get the error "Gpt2tokenizer object has no attribute _added_tokens_decoder"
configs, inference_df, iocs_df = main.go(
config_widgets
)
Contributor guide
Research direction
Start by reproducing the error with the shown main.go(config_widgets) call in the Mitremap notebook and inspect how the Gpt2tokenizer object is created. Trace the tokenizer setup to identify the compatibility failure, then verify that the notebook runs past this call without the _added_tokens_decoder error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- jupyter-notebook, python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100