AI4Finance-Foundation / AI4Finance-Foundation/FinGPT

It did not work when I try to convert the default model "chatglm2" to "llama2"

Open
#64 3 comments 0 reactions 1 assignee Claimed by @oliverwang15 View on GitHub
bug
Dominant language
Jupyter Notebook
Stars
21.2k
Forks
3k
Avg merge
1h 55m
Merged PRs (30d)
5

Description

Thanks for your awesome project. I reproduced the FinGPT v3.1.2 (4-bit QLoRA). It does work with the default LLM model "chatglm2" on Colab, but it comes to a halt when I wanna get better results with Llama2.

- I have changed the model as per your instructions, modifying _model_name = "THUDM/chatglm2-6b"_ to _model_name = "daryl149/llama-2-7b-chat-hf"_

- Then removed the device due to running error:
```
model = AutoModel.from_pretrained(
model_name,
quantization_config=q_config,
trust_remote_code=True,
token = access_token,
# device='cuda'
)
```

- Changed the _target_modules_ to llama:
`target_modules = TRANSFORMERS_MODELS_TO_LORA_TARGET_MODULES_MAPPING['llama']`

- Unfortunately, the final step got a _TypeError: 'NoneType' object cannot be interpreted as an integer_
```
writer = SummaryWriter()
trainer = ModifiedTrainer(
model=model,
args=training_args, # Trainer args
train_dataset=dataset["train"], # Training set
eval_dataset=dataset["test"], # Testing set
data_collator=data_collator, # Data Collator
callbacks=[TensorBoardCallback(writer)],
)
trainer.train()
writer.close()
# save model
model.save_pretrained(training_args.output_dir)
```
The detail error as follows:
```
You are adding a to the callbacks of this Trainer, but there is already one. The currentlist of callbacks is
:DefaultFlowCallback
TensorBoardCallback
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
in ()
9 callbacks=[TensorBoardCallback(writer)],
10 )
---> 11 trainer.train()
12 writer.close()
13 # save model

6 frames
in data_collator(features)
37 ids = ids + [tokenizer.pad_token_id] * (longest - ids_l)
38 _ids = torch.LongTensor(ids)
---> 39 labels_list.append(torch.LongTensor(labels))
40 input_ids.append(_ids)
41 input_ids = torch.stack(input_ids)

TypeError: 'NoneType' object cannot be interpreted as an integer
```

Could you please do me a favor resolving this issue? Looking forward to your reply!
(Platform: A100 on Google Colab)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.