InternLM / InternLM/InternLM-XComposer
Inference problem with my lora trained checkpoint
- Dominant language
- Python
- Stars
- 2.9k
- Forks
- 175
- PR merge metrics
- No merged PRs in 30d
Description
Firstly,thank you for the incredible results of this work.
I wrote the following inference code referring to the official document load the model after training, but it reported an error TypeError: 'NoneType' object is not callable, as my model does not have a tokenizer. Why is this?
Traceback (most recent call last):
File "/home/InternLM-XComposer/inference_lora_demo.py", line 32, in
response, his = model.chat(tokenizer, query, image, do_sample=False, num_beams=3, use_meta=True)
File "/home/miniconda3/envs/InternLM/lib/python3.9/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/nvme1/huggingface/modules/transformers_modules/internlm/internlm-xcomposer2d5-7b/4aa81f2bbf20a9ddd4137dfe847c142adf07b652/modeling_internlm_xcomposer2.py", line 631, in chat
inputs, im_mask, _ = self.interleav_wrap_chat(query, image, history=history, meta_instruction=meta_instruction, hd_num=hd_num)
File "/nvme1/huggingface/modules/transformers_modules/internlm/internlm-xcomposer2d5-7b/4aa81f2bbf20a9ddd4137dfe847c142adf07b652/modeling_internlm_xcomposer2.py", line 259, in interleav_wrap_chat
part_tokens = self.tokenizer(
TypeError: 'NoneType' object is not callable
import torch
from transformers import AutoModel, AutoTokenizer
from peft import AutoPeftModelForCausalLM
torch.set_grad_enabled(False)
adapter_path = '/home/InternLM-XComposer/finetune/output/finetune_lora_2/checkpoint-120'
model = AutoPeftModelForCausalLM.from_pretrained(
adapter_path,
device_map="auto",
torch_dtype=torch.float16,
trust_remote_code=True
).cuda().eval()
tokenizer = AutoTokenizer.from_pretrained(adapter_path, trust_remote_code=True)
model.tokenizer = tokenizer
query = 'Image1 ; Image2 ; Image3 ; I want to buy a car from the three given cars, analyze their advantages and weaknesses one by one'
image = ['./examples/cars1.jpg',
'./examples/cars2.jpg',
'./examples/cars3.jpg',]
with torch.autocast(device_type='cuda', dtype=torch.float16):
response, his = model.chat(tokenizer, query, image, do_sample=False, num_beams=3, use_meta=True)
print(response)
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.