huggingface / huggingface/nanoVLM

scaled_dot_product_attention throws an error when both attn_mask and is_causal are set

Open
#80 8 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
5k
Forks
510
PR merge metrics
No merged PRs in 30d

Description

Hi, thanks for developing a very wonderful project.

I found `torch.nn.functional.scaled_dot_product_attention` [throws an error when both attn_mask and is_causal are set ](https://docs.pytorch.org/docs/stable/generated/torch.nn.functional.scaled_dot_product_attention.html).
But, currently the language_model.py code uses both.

https://github.com/huggingface/nanoVLM/blob/6ba9082e16f1fc8c21a1f8d0c54b26c9233c8771/models/language_model.py#L141

A simple fix is to create a causal mask by yourself, but if there's other ways, I want to know.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start in models/language_model.py around line 141, where scaled_dot_product_attention is called with both attn_mask and is_causal. Reproduce the error on that path and adjust the attention-mask handling so the model no longer passes an invalid combination; done means this language-model path runs without the error when both settings are needed.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.