InternLM / InternLM/InternLM-XComposer

Inference problem with my lora trained checkpoint

Open
#422 1 comment 0 reactions 1 assignee Claimed by @yhcao6 View on GitHub
Dominant language
Python
Stars
2.9k
Forks
175
PR merge metrics
No merged PRs in 30d

Description

Firstly,thank you for the incredible results of this work.

I wrote the following inference code referring to the official document load the model after training, but it reported an error TypeError: 'NoneType' object is not callable, as my model does not have a tokenizer. Why is this?

Traceback (most recent call last):
File "/home/InternLM-XComposer/inference_lora_demo.py", line 32, in
response, his = model.chat(tokenizer, query, image, do_sample=False, num_beams=3, use_meta=True)
File "/home/miniconda3/envs/InternLM/lib/python3.9/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/nvme1/huggingface/modules/transformers_modules/internlm/internlm-xcomposer2d5-7b/4aa81f2bbf20a9ddd4137dfe847c142adf07b652/modeling_internlm_xcomposer2.py", line 631, in chat
inputs, im_mask, _ = self.interleav_wrap_chat(query, image, history=history, meta_instruction=meta_instruction, hd_num=hd_num)
File "/nvme1/huggingface/modules/transformers_modules/internlm/internlm-xcomposer2d5-7b/4aa81f2bbf20a9ddd4137dfe847c142adf07b652/modeling_internlm_xcomposer2.py", line 259, in interleav_wrap_chat
part_tokens = self.tokenizer(
TypeError: 'NoneType' object is not callable

import torch
from transformers import AutoModel, AutoTokenizer
from peft import AutoPeftModelForCausalLM

torch.set_grad_enabled(False)

adapter_path = '/home/InternLM-XComposer/finetune/output/finetune_lora_2/checkpoint-120'

model = AutoPeftModelForCausalLM.from_pretrained(
adapter_path,
device_map="auto",
torch_dtype=torch.float16,
trust_remote_code=True
).cuda().eval()

tokenizer = AutoTokenizer.from_pretrained(adapter_path, trust_remote_code=True)
model.tokenizer = tokenizer

query = 'Image1 ; Image2 ; Image3 ; I want to buy a car from the three given cars, analyze their advantages and weaknesses one by one'
image = ['./examples/cars1.jpg',
'./examples/cars2.jpg',
'./examples/cars3.jpg',]

with torch.autocast(device_type='cuda', dtype=torch.float16):
response, his = model.chat(tokenizer, query, image, do_sample=False, num_beams=3, use_meta=True)
print(response)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.