facebookresearch / facebookresearch/sam2

Why positional encodings in cross-attention keys but not queries?

Open
#282 6 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
19.9k
Forks
2.5k
PR merge metrics
No merged PRs in 30d

Description

Hi. Thanks for the great work.

Why is the pos_enc in cross-attention only used for the keys, and not the queries? (see config files)
```
pos_enc_at_cross_attn_keys: true
pos_enc_at_cross_attn_queries: false
```
Is this design decision based on empirical results (if so, what are the ablation results?)? Is there some other reason behind it, or some intuition why this might be?

Thanks in advance,
Miguel

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.