facebookresearch / facebookresearch/sam2
Why positional encodings in cross-attention keys but not queries?
Open
- Dominant language
- Jupyter Notebook
- Stars
- 19.9k
- Forks
- 2.5k
- PR merge metrics
- No merged PRs in 30d
Description
Hi. Thanks for the great work.
Why is the pos_enc in cross-attention only used for the keys, and not the queries? (see config files)
```
pos_enc_at_cross_attn_keys: true
pos_enc_at_cross_attn_queries: false
```
Is this design decision based on empirical results (if so, what are the ablation results?)? Is there some other reason behind it, or some intuition why this might be?
Thanks in advance,
Miguel
Contributor guide
Assessment
This issue has not been assessed yet.