thu-ml / thu-ml/TurboDiffusion

scripts/inference_wan2.2_i2v.sh, KeyError: 't2v'

Open
#31 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
3.7k
Forks
277
Avg merge
2h 57m
Merged PRs (30d)
2

Description

Traceback (most recent call last):
  File "/home/inspur/work_space/gen_img_video/TurboDiffusion-Space/venv_turboDif/lib/python3.12/site-packages/transformers/models/auto/configuration_auto.py", line 1360, in from_pretrained
    config_class = CONFIG_MAPPING[config_dict["model_type"]]
                   ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/inspur/work_space/gen_img_video/TurboDiffusion-Space/venv_turboDif/lib/python3.12/site-packages/transformers/models/auto/configuration_auto.py", line 1048, in __getitem__
    raise KeyError(key)
KeyError: 't2v'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/inspur/work_space/gen_img_video/TurboDiffusion-Space/TurboDiffusion/turbodiffusion/inference/wan2.2_i2v_infer.py", line 70, in <module>
    text_emb = get_umt5_embedding(checkpoint_path=args.text_encoder_path, prompts=args.prompt).to(**tensor_kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/inspur/work_space/gen_img_video/TurboDiffusion-Space/TurboDiffusion/turbodiffusion/rcm/utils/umt5.py", line 533, in get_umt5_embedding
    t5_encoder = UMT5EncoderModel(text_len=max_length, device=device, checkpoint_path=checkpoint_path)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/inspur/work_space/gen_img_video/TurboDiffusion-Space/TurboDiffusion/turbodiffusion/rcm/utils/umt5.py", line 500, in __init__
    self.tokenizer = HuggingfaceTokenizer(name=tokenizer_path, seq_len=text_len, clean="whitespace")
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/inspur/work_space/gen_img_video/TurboDiffusion-Space/TurboDiffusion/turbodiffusion/rcm/utils/umt5.py", line 66, in __init__
    self.tokenizer = AutoTokenizer.from_pretrained(name, **kwargs)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/inspur/work_space/gen_img_video/TurboDiffusion-Space/venv_turboDif/lib/python3.12/site-packages/transformers/models/auto/tokenization_auto.py", line 1109, in from_pretrained
    config = AutoConfig.from_pretrained(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/inspur/work_space/gen_img_video/TurboDiffusion-Space/venv_turboDif/lib/python3.12/site-packages/transformers/models/auto/configuration_auto.py", line 1362, in from_pretrained
    raise ValueError(
ValueError: The checkpoint you are trying to load has model type `t2v` but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date.

You can update Transformers with the command `pip install --upgrade transformers`. If this does not work, and the checkpoint is very new, then there may not be a release version that supports this model yet. In this case, you can get the most up-to-date code by installing Transformers from source with the command `pip install git+https://github.com/huggingface/transformers.git`

Hi, guys, nice work. I'm using scripts/inference_wan2.2_i2v.sh, but error comes like that, I'm sure my transformers is up-to-date, what should I do?

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running scripts/inference_wan2.2_i2v.sh and follow the traceback through turbodiffusion/inference/wan2.2_i2v_infer.py and turbodiffusion/rcm/utils/umt5.py. Inspect the checkpoint passed to get_umt5_embedding and its Transformers compatibility, then verify that the tokenizer loads and the inference script proceeds past text embedding.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai-infra-agents, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.