deepspeedai / deepspeedai/DeepSpeedExamples
AttributeError: 'Namespace' object has no attribute 'tensor_model_parallel_size'
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 6.8k
- Forks
- 1.1k
- Avg merge
- 2d 16h
- Merged PRs (30d)
- 1
Description
I am trying to run the GPT2 model on Google Colab. I have placed my training data in a loose JSON format, with one JSON containing a text sample per line. On running the preprocessing script, I am getting the above error. Kindly help me with this.
complete error below:
Opening my-corpus.json
building GPT2BPETokenizer tokenizer ...
Traceback (most recent call last):
File "/content/DeepSpeedExamples/Megatron-LM-v1.1.5-ZeRO3/tools/preprocess_data.py", line 200, in
main()
File "/content/DeepSpeedExamples/Megatron-LM-v1.1.5-ZeRO3/tools/preprocess_data.py", line 154, in main
tokenizer = build_tokenizer(args)
File "/usr/local/lib/python3.7/dist-packages/megatron/tokenizer/tokenizer.py", line 48, in build_tokenizer
args)
File "/usr/local/lib/python3.7/dist-packages/megatron/tokenizer/tokenizer.py", line 59, in _vocab_size_with_padding
args.tensor_model_parallel_size
AttributeError: 'Namespace' object has no attribute 'tensor_model_parallel_size'
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with tools/preprocess_data.py around main() and build_tokenizer, then inspect megatron/tokenizer/tokenizer.py at _vocab_size_with_padding where the traceback occurs. Reproduce the reported GPT-2 preprocessing run in Google Colab and verify that it completes without the tensor_model_parallel_size AttributeError.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100