facebookresearch / facebookresearch/co-tracker

Inaccurate tracking in case of occlusions

Open
#27 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
5.1k
Forks
389
PR merge metrics
No merged PRs in 30d

Description

@nikitakaraevv

![image](https://github.com/facebookresearch/co-tracker/assets/55429956/8f08a177-dce2-4ef6-b9a3-10f0ad30d4b3)

I've noticed that when there's an obstruction in between frames, the model struggles to track points effectively. Is there a possibility of implementing a mechanism to temporarily remove tracking points when an obstruction is detected in the frame, and then resume tracking when the target object reappears in subsequent frames?

I've conducted several tests, and unfortunately, the results haven't been as promising as described in the [documentation](https://co-tracker.github.io/).

If there's a potential solution or workaround for this issue, I'd greatly appreciate any insights or guidance.

Thank you in advance for your assistance!
This is the Video that I obtained while testing it.
https://github.com/facebookresearch/co-tracker/assets/55429956/c93433fe-adbe-450f-899b-519237eade74

Contributor guide

Open the contributing guide

Research direction

Reproduce the reported tracking failure with the linked video and compare the result with the behavior described in the documentation. No source files or tests are named, so first locate the tracking entry point used for that example. Done means the reported occlusion case handles points consistently through disappearance and reappearance, with the behavior verified against the example.

Written by the indexing model from the issue text.

Assessment

Tech stack
jupyter-notebook
Domain
computer-vision, machine-learning
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.