AI4Finance-Foundation / AI4Finance-Foundation/FinGPT
It did not work when I try to convert the default model "chatglm2" to "llama2"
- Dominant language
- Jupyter Notebook
- Stars
- 21.2k
- Forks
- 3k
- Avg merge
- 1h 55m
- Merged PRs (30d)
- 5
Description
Thanks for your awesome project. I reproduced the FinGPT v3.1.2 (4-bit QLoRA). It does work with the default LLM model "chatglm2" on Colab, but it comes to a halt when I wanna get better results with Llama2.
- I have changed the model as per your instructions, modifying _model_name = "THUDM/chatglm2-6b"_ to _model_name = "daryl149/llama-2-7b-chat-hf"_
- Then removed the device due to running error:
```
model = AutoModel.from_pretrained(
model_name,
quantization_config=q_config,
trust_remote_code=True,
token = access_token,
# device='cuda'
)
```
- Changed the _target_modules_ to llama:
`target_modules = TRANSFORMERS_MODELS_TO_LORA_TARGET_MODULES_MAPPING['llama']`
- Unfortunately, the final step got a _TypeError: 'NoneType' object cannot be interpreted as an integer_
```
writer = SummaryWriter()
trainer = ModifiedTrainer(
model=model,
args=training_args, # Trainer args
train_dataset=dataset["train"], # Training set
eval_dataset=dataset["test"], # Testing set
data_collator=data_collator, # Data Collator
callbacks=[TensorBoardCallback(writer)],
)
trainer.train()
writer.close()
# save model
model.save_pretrained(training_args.output_dir)
```
The detail error as follows:
```
You are adding a to the callbacks of this Trainer, but there is already one. The currentlist of callbacks is
:DefaultFlowCallback
TensorBoardCallback
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
in ()
9 callbacks=[TensorBoardCallback(writer)],
10 )
---> 11 trainer.train()
12 writer.close()
13 # save model
6 frames
in data_collator(features)
37 ids = ids + [tokenizer.pad_token_id] * (longest - ids_l)
38 _ids = torch.LongTensor(ids)
---> 39 labels_list.append(torch.LongTensor(labels))
40 input_ids.append(_ids)
41 input_ids = torch.stack(input_ids)
TypeError: 'NoneType' object cannot be interpreted as an integer
```
Could you please do me a favor resolving this issue? Looking forward to your reply!
(Platform: A100 on Google Colab)
Contributor guide
Assessment
This issue has not been assessed yet.