facebookresearch / facebookresearch/co-tracker
About the queried_frame of tracks
- Dominant language
- Jupyter Notebook
- Stars
- 5.1k
- Forks
- 389
- PR merge metrics
- No merged PRs in 30d
Description
Hello, the cotracker3_online uses these code before `forward_window()` and multiplies the `attention_mask` and `track_feat_support_pyramid`:
```
attention_mask = (queried_frames < ind + S).reshape(B, 1, N)
```
Therefore it seems that the `queried_frame` of each track in the current window is either ≤ `ind` or greater than `ind + S`(its `track_feat_support` multiplies 0 or not), without the situation in which `ind < queried_frame < ind + S`. Isn't it?
Moreover, I notice that the `attention_mask` parameter of `forward_window()` isn't used and nothing is passed to the `mask` parameter of `forward()` in `EfficientUpdateFormer`.
Contributor guide
Research direction
Start by tracing the attention_mask handling before forward_window(), then inspect the attention_mask parameter of forward_window() and the mask parameter of EfficientUpdateFormer.forward(). Verify whether the queried-frame mask and track_feat_support_pyramid behavior match the intended current-window semantics, and document or correct the unused-mask behavior once confirmed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- jupyter-notebook
- Domain
- computer-vision
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100