facebookresearch / facebookresearch/sam-3d-objects

Errorness Transformation of Objects

Open
#56 17 comments 9 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
7.4k
Forks
878
PR merge metrics
No merged PRs in 30d

Description

Hi, thank you for this amazing work!

I have a question about the object transformation from `canonical` to `camera` space. I have an indoor scene image (from HyperSim) which looks like this:
![Image](https://github.com/user-attachments/assets/b523fdaf-dc8f-4189-87c9-1880e5d0dccf)

The results I got from the online demo is pretty good (shown in the image below)
Image

However, when I tested the released code and followed `demo_multi_object.ipynb`, the result (rendered GIF) looks very differently to the online demo.
Image

Also, when I transform the output glb mesh with the estimated scale, rotation, and translation, the estimation doesn't seem to be accurate (visualized in blender, figure below).
Image

I think it's reasonable when the object is cropped (ex. the towel near the camera, the metal ring), but I don't understand why objects that are complete (ex. the brown box, duck, red bottle, bathtub) have weird spatial relationships. Is there anything that I'm doing wrong.

Again, thanks for your work, and I'll be grateful if you can provide any suggestions.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.