NVIDIA / NVIDIA/Megatron-LM

[BUG] GPT2BPETokenizer (and possibly others) missing decode and offsets methods

Open
#1,633 5 comments 0 reactions 0 assignees View on GitHub
bug community-request good first issue module: data pipeline waiting-on-customer
Dominant language
Python
Stars
17.9k
Forks
4.5k
Avg merge
4d 3h
Merged PRs (30d)
272

Description

I've encountered an issue when running text generation with Megatron-LM. Apologies in advance if there are any mistakes — I'm a new user.

After successfully preprocessing the data using:

```bash
python tools/preprocess_data.py --tokenizer-type GPT2BPETokenizer
```

and completing pretraining, I tried running text generation using:

```bash
tools/run_text_generation_server.py
```

However, I received the following errors:

```
AttributeError: '_GPT2BPETokenizer' object has no attribute 'decode'
AttributeError: '_GPT2BPETokenizer' object has no attribute 'offsets'
```

These errors seem to originate from:

* [`tokenizer.py`, line 323](https://github.com/NVIDIA/Megatron-LM/blob/7f7ffcf895aba1e4f5739f27236031401ee32598/megatron/training/tokenizer/tokenizer.py#L323)
* [`run_text_generation_server_345M.sh`, line 25](https://github.com/NVIDIA/Megatron-LM/blob/7f7ffcf895aba1e4f5739f27236031401ee32598/examples/inference/run_text_generation_server_345M.sh#L25)

It seems like `_GPT2BPETokenizer` may be missing `decode` and `offsets` methods required by the inference script. Any guidance on how to resolve this would be greatly appreciated.

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the failure with tools/preprocess_data.py using GPT2BPETokenizer and tools/run_text_generation_server.py. Inspect megatron/training/tokenizer/tokenizer.py around line 323 and examples/inference/run_text_generation_server_345M.sh around line 25; done means text generation no longer raises missing decode or offsets errors.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
62/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.