InternLM / InternLM/InternLM-XComposer

Some errors:ValueError: Tokenizer class InternLMXComposerTokenizer does not exist or is not currently imported.

Open
#308 1 comment 0 reactions 1 assignee Claimed by @LightDXY View on GitHub
Dominant language
Python
Stars
2.9k
Forks
175
PR merge metrics
No merged PRs in 30d

Description

I run the quick inference demo of the github repo and follow the install pipeline without any other operation.
But I got this error.
Traceback (most recent call last):
File "/home/ssw/DemoFusion/base_models/InternLM-XComposer/projects/ShareGPT4V/demo_share4v.py", line 24, in
eval_model(args)
File "/home/ssw/DemoFusion/base_models/InternLM-XComposer/projects/ShareGPT4V/share4v/eval/run_share4v.py", line 31, in eval_model
tokenizer, model, image_processor, context_len = load_pretrained_model(
File "/home/ssw/DemoFusion/base_models/InternLM-XComposer/projects/ShareGPT4V/share4v/model/builder.py", line 114, in load_pretrained_model
tokenizer = AutoTokenizer.from_pretrained(
File "/home/ssw/anaconda3/envs/share4v/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py", line 699, in from_pretrained
raise ValueError(
ValueError: Tokenizer class InternLMXComposerTokenizer does not exist or is not currently imported.

Hope for your reply!

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.