DAMO-NLP-SG / DAMO-NLP-SG/VCD

RuntimeError: The size of tensor a (812) must match the size of tensor b (6300) at non-singleton dimension 0

Open
#33 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
418
Forks
27
PR merge metrics
No merged PRs in 30d

Description

generated_ids = model.generate(**inputs, max_new_tokens=32,)为什么推理的时候报错,但是添加use_cache=False就可以不报错呢,,下面的报错


outputs_cd = self(
**model_inputs_cd,
return_dict=True,
output_attentions=output_attentions_wo_img,
output_hidden_states=output_hidden_states_wo_img,
)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reproducing the failure from `model.generate(**inputs, max_new_tokens=32)` and compare it with `use_cache=False`. Trace the `self(**model_inputs_cd, ...)` call and inspect the tensor shapes at the point where dimension 0 is combined. Done means identifying the incompatible inputs or cache behavior and documenting a verified correction or minimal reproduction.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.