facebookresearch / facebookresearch/ImageBind

Using Depth Embeddings in NyuV2 Zero-Shot Classification

Open
#107 4 comments 2 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
9.1k
Forks
842
PR merge metrics
No merged PRs in 30d

Description

Thank you for your exceptional work and the code you've provided.
I have a question regarding the use of depth embedding in the context of NyuV2 zero-shot classification.
For the conversion of depth to disparity, I am utilizing a focal length of 518.857901 and a baseline value of 0.075.
However, the accuracy I am achieving is only 45%, which is 10% lower than what is reported in the paper.

Could you possibly advise on any additional steps that might be necessary?
Currently, I am conducting operations such as converting depth to disparity, resizing, center cropping, and normalizing.
For the normalization process, I am using mean and standard deviation values of 0.0418 and 0.0295, respectively.
Additionally, I attempted to apply DepthNorm again after converting to disparity, but it did not yield the desired results.

For the 10-th class, I am using both methods - labeling it as 'others' and selecting the class with the highest cosine similarity from the 18 specified in the paper.

Your guidance on this matter would be greatly appreciated.
Thank you.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.