facebookresearch / facebookresearch/co-tracker

About the queried_frame of tracks

Open
#156 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
5.1k
Forks
389
PR merge metrics
No merged PRs in 30d

Description

Hello, the cotracker3_online uses these code before `forward_window()` and multiplies the `attention_mask` and `track_feat_support_pyramid`:
```
attention_mask = (queried_frames < ind + S).reshape(B, 1, N)
```
Therefore it seems that the `queried_frame` of each track in the current window is either ≤ `ind` or greater than `ind + S`(its `track_feat_support` multiplies 0 or not), without the situation in which `ind < queried_frame < ind + S`. Isn't it?

Moreover, I notice that the `attention_mask` parameter of `forward_window()` isn't used and nothing is passed to the `mask` parameter of `forward()` in `EfficientUpdateFormer`.

Contributor guide

Open the contributing guide

Research direction

Start by tracing the attention_mask handling before forward_window(), then inspect the attention_mask parameter of forward_window() and the mask parameter of EfficientUpdateFormer.forward(). Verify whether the queried-frame mask and track_feat_support_pyramid behavior match the intended current-window semantics, and document or correct the unused-mask behavior once confirmed.

Written by the indexing model from the issue text.

Assessment

Tech stack
jupyter-notebook
Domain
computer-vision
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.