[BUG] GPT2BPETokenizer (and possibly others) missing decode and offsets methods
- Dominant language
- Python
- Stars
- 17.9k
- Forks
- 4.5k
- Avg merge
- 4d 3h
- Merged PRs (30d)
- 272
Description
I've encountered an issue when running text generation with Megatron-LM. Apologies in advance if there are any mistakes — I'm a new user.
After successfully preprocessing the data using:
```bash
python tools/preprocess_data.py --tokenizer-type GPT2BPETokenizer
```
and completing pretraining, I tried running text generation using:
```bash
tools/run_text_generation_server.py
```
However, I received the following errors:
```
AttributeError: '_GPT2BPETokenizer' object has no attribute 'decode'
AttributeError: '_GPT2BPETokenizer' object has no attribute 'offsets'
```
These errors seem to originate from:
* [`tokenizer.py`, line 323](https://github.com/NVIDIA/Megatron-LM/blob/7f7ffcf895aba1e4f5739f27236031401ee32598/megatron/training/tokenizer/tokenizer.py#L323)
* [`run_text_generation_server_345M.sh`, line 25](https://github.com/NVIDIA/Megatron-LM/blob/7f7ffcf895aba1e4f5739f27236031401ee32598/examples/inference/run_text_generation_server_345M.sh#L25)
It seems like `_GPT2BPETokenizer` may be missing `decode` and `offsets` methods required by the inference script. Any guidance on how to resolve this would be greatly appreciated.
Contributor guide
Research direction
Start by reproducing the failure with tools/preprocess_data.py using GPT2BPETokenizer and tools/run_text_generation_server.py. Inspect megatron/training/tokenizer/tokenizer.py around line 323 and examples/inference/run_text_generation_server_345M.sh around line 25; done means text generation no longer raises missing decode or offsets errors.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 62/100