facebookresearch / facebookresearch/co-tracker
cannot backward tracking in online model with specific queries in middle frame
- Dominant language
- Jupyter Notebook
- Stars
- 5.1k
- Forks
- 389
- PR merge metrics
- No merged PRs in 30d
Description
Hi,
I am using your online model for point tracking. and I want to process a video. I want to set up specific queries in one of the middle frame and try to use backward tracking. here is my code:
`video = torch.tensor(frames).permute(0, 3, 1, 2)[None].float().to(device)
cotracker = torch.hub.load("facebookresearch/co-tracker", "cotracker2_online").to(device)
cotracker(video_chunk = video, is_first_step = True, queries=maxima_coords[None], grid_query_frame=grid_query_frame, backward_tracking=True)
vis = Visualizer(save_dir="./co-tracker/result", pad_value=120, linewidth=2)
for ind in range(0, video.shape[1] - cotracker.step, cotracker.step):
pred_tracks, pred_visibility = cotracker(video_chunk=video[:, ind : ind + cotracker.step * 2], queries=maxima_coords[None], grid_query_frame=grid_query_frame, backward_tracking=True)
vis.visualize(video, pred_tracks, pred_visibility)`
and here is the bug i got:
---> 60 pred_tracks, pred_visibility = cotracker(video_chunk=video[:, ind : ind + cotracker.step * 2], queries=maxima_coords[None], grid_query_frame=grid_query_frame, backward_tracking=True)
61
62 vis.visualize(video, pred_tracks, pred_visibility)
~/anaconda3/lib/python3.9/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
1128 if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
1129 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1130 return forward_call(*input, **kwargs)
1131 # Do not call functions when jit is used
1132 full_backward_hooks, non_full_backward_hooks = [], []
~/anaconda3/lib/python3.9/site-packages/torch/autograd/grad_mode.py in decorate_context(*args, **kwargs)
25 def decorate_context(*args, **kwargs):
26 with self.clone():
---> 27 return func(*args, **kwargs)
28 return cast(F, decorate_context)
29
TypeError: forward() got an unexpected keyword argument 'backward_tracking'
Contributor guide
Research direction
Start by checking the forward signature and implementation of the torch.hub-loaded cotracker2_online model, focusing on the provided backward_tracking and grid_query_frame arguments. Reproduce the TypeError with the supplied video-chunk loop, then confirm that the intended middle-frame backward-tracking workflow is supported and covered by a runnable example or regression test.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- computer-vision, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 28/100