huggingface / huggingface/nanoVLM
scaled_dot_product_attention throws an error when both attn_mask and is_causal are set
- Dominant language
- Python
- Stars
- 5k
- Forks
- 510
- PR merge metrics
- No merged PRs in 30d
Description
Hi, thanks for developing a very wonderful project.
I found `torch.nn.functional.scaled_dot_product_attention` [throws an error when both attn_mask and is_causal are set ](https://docs.pytorch.org/docs/stable/generated/torch.nn.functional.scaled_dot_product_attention.html).
But, currently the language_model.py code uses both.
https://github.com/huggingface/nanoVLM/blob/6ba9082e16f1fc8c21a1f8d0c54b26c9233c8771/models/language_model.py#L141
A simple fix is to create a causal mask by yourself, but if there's other ways, I want to know.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start in models/language_model.py around line 141, where scaled_dot_product_attention is called with both attn_mask and is_causal. Reproduce the error on that path and adjust the attention-mask handling so the model no longer passes an invalid combination; done means this language-model path runs without the error when both settings are needed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100