ByteDance-Seed / ByteDance-Seed/SimArt

Unexpected Results on the PartNet-Mobility SINGAPO 77-Test Split

Open
#1 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
121
Forks
3
PR merge metrics
No merged PRs in 30d

Description

Hello,
Thank you very much for releasing the model weights and making your work publicly available.
I tested the released model on the PartNet-Mobility SINGAPO 77-test split, but the results were substantially worse than expected, as illustrated in the video below. In particular, I observed noticeable errors in both the predicted part decomposition and the estimated kinematic axes.

Since a considerable portion of SIMART’s training data appears to come from PartNet-Mobility, I expected the released model to generalize reasonably well to PartNet-Mobility-style objects. I may have overlooked an important preprocessing step, coordinate convention, inference setting, or checkpoint configuration.

Could you please clarify whether the released checkpoint is expected to reproduce the reported performance on PartNet-Mobility-style cases? If so, would you be willing to share the recommended preprocessing and inference configuration, or any evaluation scripts used for these examples?
Thank you again for your time and for releasing the project.

PM-48721:
https://github.com/user-attachments/assets/b434dcd6-80fb-4cda-a03a-aab8dfff8d8b

PM-48513:
https://github.com/user-attachments/assets/deec33ce-49ca-4ef8-ba8b-f8f21c18c930

PM-48797:
https://github.com/user-attachments/assets/8c944d36-b243-4427-b024-a554a0fc13aa

Contributor guide

No contributing guide indexed for this repository

Research direction

No repository file, test, or entry point is named. Start by locating the released checkpoint's preprocessing and inference entry points, then reproduce the SINGAPO 77-test split results and compare them with the reported expectations. Done means determining whether the checkpoint should support these cases and documenting the required configuration or evaluation procedure.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
computer-graphics, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.