facebookresearch / facebookresearch/spider

Questions about reproducing OMOMO result

Open
#78 0 comments 2 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
544
Forks
60
PR merge metrics
No merged PRs in 30d

Description


Hi, thanks for releasing SPIDER.

I'm trying to use SPIDER to retarget humanoid+object motion and reproduce the OMOMO results in Table 4 of the paper, but the performance is not satisfactory.

I can successfully run the provided example data `spider/example_datasets/processed/omomo/unitree_g1/humanoid_object/move_largebox` from https://huggingface.co/datasets/retarget/retarget_example. However, when I try to expand to more OMOMO motion data, I cannot find the exact OMOMO split / preprocessed humanoid-object data used for the paper experiments. Therefore, I downloaded a kinematics-retargeted subset of OMOMO from https://huggingface.co/datasets/omniretarget/OmniRetarget_Dataset, specifically the robot-object part.

The results are often not satisfactory: the robot frequently deviates from the reference and usually cannot lift or move the box correctly. A few motions succeed, but most fail.

Some failure examples:

https://github.com/user-attachments/assets/5d3fb634-b721-4ded-ada1-7a0a69ec70e0

https://github.com/user-attachments/assets/3396786d-cba5-4d34-b2ef-5b653866c6aa

These runs use examples/run_mjwp.py with the default solver parameters (+override=humanoid_object). I also tried using examples/run_mjwp_fast.py, or turning on "terminate with object threshold", but these do not fundamentally solve the issue.

Could you provide some guidance on how to reproduce the OMOMO results from the paper experiments? Specifically, is the exact data split/preprocessed data available, or are there any specific solver parameters I should tune?

Thanks for any help.

Contributor guide

Open the contributing guide

Research direction

Start with examples/run_mjwp.py and the humanoid_object solver configuration, then compare the provided move_largebox example under spider/example_datasets/processed/omomo with the downloaded OmniRetarget robot-object data. Reproduce the reported failures and check whether the exact OMOMO split or preprocessing used for Table 4 is documented or available. Done means identifying the matching data and parameters, or documenting why the alternate data cannot reproduce the paper results.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
robotics
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.