ByteDance-Seed / ByteDance-Seed/SimArt
Unexpected Results on the PartNet-Mobility SINGAPO 77-Test Split
- Dominant language
- Python
- Stars
- 121
- Forks
- 3
- PR merge metrics
- No merged PRs in 30d
Description
Hello,
Thank you very much for releasing the model weights and making your work publicly available.
I tested the released model on the PartNet-Mobility SINGAPO 77-test split, but the results were substantially worse than expected, as illustrated in the video below. In particular, I observed noticeable errors in both the predicted part decomposition and the estimated kinematic axes.
Since a considerable portion of SIMART’s training data appears to come from PartNet-Mobility, I expected the released model to generalize reasonably well to PartNet-Mobility-style objects. I may have overlooked an important preprocessing step, coordinate convention, inference setting, or checkpoint configuration.
Could you please clarify whether the released checkpoint is expected to reproduce the reported performance on PartNet-Mobility-style cases? If so, would you be willing to share the recommended preprocessing and inference configuration, or any evaluation scripts used for these examples?
Thank you again for your time and for releasing the project.
PM-48721:
https://github.com/user-attachments/assets/b434dcd6-80fb-4cda-a03a-aab8dfff8d8b
PM-48513:
https://github.com/user-attachments/assets/deec33ce-49ca-4ef8-ba8b-f8f21c18c930
PM-48797:
https://github.com/user-attachments/assets/8c944d36-b243-4427-b024-a554a0fc13aa
Contributor guide
No contributing guide indexed for this repository
Research direction
No repository file, test, or entry point is named. Start by locating the released checkpoint's preprocessing and inference entry points, then reproduce the SINGAPO 77-test split results and compare them with the reported expectations. Done means determining whether the checkpoint should support these cases and documenting the required configuration or evaluation procedure.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- computer-graphics, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100