RuntimeError: The size of tensor a (812) must match the size of tensor b (6300) at non-singleton dimension 0
- Dominant language
- Python
- Stars
- 418
- Forks
- 27
- PR merge metrics
- No merged PRs in 30d
Description
generated_ids = model.generate(**inputs, max_new_tokens=32,)为什么推理的时候报错,但是添加use_cache=False就可以不报错呢,,下面的报错
outputs_cd = self(
**model_inputs_cd,
return_dict=True,
output_attentions=output_attentions_wo_img,
output_hidden_states=output_hidden_states_wo_img,
)
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reproducing the failure from `model.generate(**inputs, max_new_tokens=32)` and compare it with `use_cache=False`. Trace the `self(**model_inputs_cd, ...)` call and inspect the tensor shapes at the point where dimension 0 is combined. Done means identifying the incompatible inputs or cache behavior and documenting a verified correction or minimal reproduction.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100