facebookresearch / facebookresearch/spider
Questions about reproducing OMOMO result
- Dominant language
- Python
- Stars
- 544
- Forks
- 60
- PR merge metrics
- No merged PRs in 30d
Description
Hi, thanks for releasing SPIDER.
I'm trying to use SPIDER to retarget humanoid+object motion and reproduce the OMOMO results in Table 4 of the paper, but the performance is not satisfactory.
I can successfully run the provided example data `spider/example_datasets/processed/omomo/unitree_g1/humanoid_object/move_largebox` from https://huggingface.co/datasets/retarget/retarget_example. However, when I try to expand to more OMOMO motion data, I cannot find the exact OMOMO split / preprocessed humanoid-object data used for the paper experiments. Therefore, I downloaded a kinematics-retargeted subset of OMOMO from https://huggingface.co/datasets/omniretarget/OmniRetarget_Dataset, specifically the robot-object part.
The results are often not satisfactory: the robot frequently deviates from the reference and usually cannot lift or move the box correctly. A few motions succeed, but most fail.
Some failure examples:
https://github.com/user-attachments/assets/5d3fb634-b721-4ded-ada1-7a0a69ec70e0
https://github.com/user-attachments/assets/3396786d-cba5-4d34-b2ef-5b653866c6aa
These runs use examples/run_mjwp.py with the default solver parameters (+override=humanoid_object). I also tried using examples/run_mjwp_fast.py, or turning on "terminate with object threshold", but these do not fundamentally solve the issue.
Could you provide some guidance on how to reproduce the OMOMO results from the paper experiments? Specifically, is the exact data split/preprocessed data available, or are there any specific solver parameters I should tune?
Thanks for any help.
Contributor guide
Research direction
Start with examples/run_mjwp.py and the humanoid_object solver configuration, then compare the provided move_largebox example under spider/example_datasets/processed/omomo with the downloaded OmniRetarget robot-object data. Reproduce the reported failures and check whether the exact OMOMO split or preprocessing used for Table 4 is documented or available. Done means identifying the matching data and parameters, or documenting why the alternate data cannot reproduce the paper results.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- robotics
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100