facebookresearch / facebookresearch/co-tracker
Meaning of the annotation_dict entry of CoTracker3_Kubric data
- Dominant language
- Jupyter Notebook
- Stars
- 5.1k
- Forks
- 389
- PR merge metrics
- No merged PRs in 30d
Description
Hi, thanks for the great work!
I just started to play with the newly released CoTracker3_Kubric dataset. However, I found that there are three different entries associated with depth. There are `"coords_depth"` and `"depth"` in the annotation_dict (e.g. 0000.npy) of each sample , and there is also a `depth` folder where it looks like the depth of each frame is stored in different npy files (e.g. 000.npy-119.npy).
My preliminary exploration sort of indicate the `annot_dict["depth"]` uses the _absolute_ 512x512 depth map, and `annot_dict["depth"]` stores the _relative_ depth of each tracked point. However, what is the difference between `annot_dict["depth"]` and the npy fils under the `depth` folder? Could you please elaborate on the meaning of these entries?
Moreover, I wonder how did you convert the classic Kubric TFDS format into the current format. I see it in your [previous post](https://github.com/facebookresearch/co-tracker/issues/8#issuecomment-1808334019) you used `traj_annots = {"coords": data["target_points"], "visibility": data["occluded"]}`, but the other important entries like `depth` or `camera` or other important meta infos are missing. I wonder if you could show how the conversion between TFDS Kubric data and CoTracker3_Kubric data?
Thanks a lot!
Contributor guide
Assessment
This issue has not been assessed yet.