GuyTevet / GuyTevet/motion-diffusion-model
Issues with x_t (noise) prediction and some question about the training loss.
- Dominant language
- Python
- Stars
- 4.1k
- Forks
- 462
- PR merge metrics
- No merged PRs in 30d
Description
1. Based on Issues #19 and the config, Geometric losses are not used. Instead, they are implemented through the 263-dimensional motion representation. Therefore, geometric losses will never be used; they are just redundant code. Is my understanding correct?
2. Why use x_0 as the training target? Is there an explanation for this? Currently, most mainstream diffusion models predict xt instead.
3. When I set the code to predict x_t as the training target, I found that the final results were very poor (
Training for 23,000 steps, use linear beta scheduler, the character keeps shaking and doesn't perform the action well). What could be the reason for this? I only made the change to set `self.model_mean_type = ModelMeanType.EPSILON` in the training and sampling code. Could such a simple modification cause any issues?
predict x_t sample results:
https://github.com/user-attachments/assets/e1727020-2b23-491b-8d70-09dd2fe040b6
https://github.com/user-attachments/assets/3d960100-df7a-4209-85b2-79512fd3a9e2
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by tracing the training and sampling code around `self.model_mean_type = ModelMeanType.EPSILON`, and compare it with the configured x_0 target and the 263-dimensional motion representation. Review Issue #19 and the config to determine whether geometric losses are redundant, then reproduce the 23,000-step linear-beta result and document the cause of the shaking and poor action quality.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100