deepspeedai / deepspeedai/DeepSpeedExamples

Eextension of the issus #479, chatbot.py cannot load the bloom model

Open
#505 3 comments 0 reactions 1 assignee View on GitHub

@xiexbing is already working on this.

Since May 18, 2023.

deespeed chat new-config
Dominant language
Python
Stars
6.8k
Forks
1.1k
Avg merge
2d 16h
Merged PRs (30d)
1

Description

Eextension of the issus #479, I only pass the step1_supervised_finetuning, use bloom-3b model.
And I run the following program:

python chat.py --path output/actor-models/bloom3b

I get the following error message:

Traceback (most recent call last):
File "./inference/chatbot.py", line 102, in
main(args)
File "./inference/chatbot.py", line 73, in main
generator = get_generator(args.path)
File "./inference/chatbot.py", line 34, in get_generator
model = OPTForCausalLM.from_pretrained(path,
File "/workspace/anaconda3/envs/deepspeed/lib/python3.8/site-packages/transformers/modeling_utils.py", line 2629, in from_pretrained
model = cls(config, *model_args, **model_kwargs)
File "/workspace/anaconda3/envs/deepspeed/lib/python3.8/site-packages/transformers/models/opt/modeling_opt.py", line 817, in init
self.model = OPTModel(config)
File "/workspace/anaconda3/envs/deepspeed/lib/python3.8/site-packages/transformers/models/opt/modeling_opt.py", line 749, in init
self.decoder = OPTDecoder(config)
File "/workspace/anaconda3/envs/deepspeed/lib/python3.8/site-packages/transformers/models/opt/modeling_opt.py", line 492, in init
self.layerdrop = config.layerdrop
File "/workspace/anaconda3/envs/deepspeed/lib/python3.8/site-packages/transformers/configuration_utils.py", line 260, in getattribute
return super().getattribute(key)
AttributeError: 'BloomConfig' object has no attribute 'layerdrop'

May I ask, does this mean that the programs provided by the github can only run OPT models?
If I want the program to use bloom model, how can I fix the above error?

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.