deepspeedai / deepspeedai/DeepSpeedExamples
Eextension of the issus #479, chatbot.py cannot load the bloom model
@xiexbing is already working on this.
Since May 18, 2023.
- Dominant language
- Python
- Stars
- 6.8k
- Forks
- 1.1k
- Avg merge
- 2d 16h
- Merged PRs (30d)
- 1
Description
Eextension of the issus #479, I only pass the step1_supervised_finetuning, use bloom-3b model.
And I run the following program:
python chat.py --path output/actor-models/bloom3b
I get the following error message:
Traceback (most recent call last):
File "./inference/chatbot.py", line 102, in
main(args)
File "./inference/chatbot.py", line 73, in main
generator = get_generator(args.path)
File "./inference/chatbot.py", line 34, in get_generator
model = OPTForCausalLM.from_pretrained(path,
File "/workspace/anaconda3/envs/deepspeed/lib/python3.8/site-packages/transformers/modeling_utils.py", line 2629, in from_pretrained
model = cls(config, *model_args, **model_kwargs)
File "/workspace/anaconda3/envs/deepspeed/lib/python3.8/site-packages/transformers/models/opt/modeling_opt.py", line 817, in init
self.model = OPTModel(config)
File "/workspace/anaconda3/envs/deepspeed/lib/python3.8/site-packages/transformers/models/opt/modeling_opt.py", line 749, in init
self.decoder = OPTDecoder(config)
File "/workspace/anaconda3/envs/deepspeed/lib/python3.8/site-packages/transformers/models/opt/modeling_opt.py", line 492, in init
self.layerdrop = config.layerdrop
File "/workspace/anaconda3/envs/deepspeed/lib/python3.8/site-packages/transformers/configuration_utils.py", line 260, in getattribute
return super().getattribute(key)
AttributeError: 'BloomConfig' object has no attribute 'layerdrop'
May I ask, does this mean that the programs provided by the github can only run OPT models?
If I want the program to use bloom model, how can I fix the above error?
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.