GuyTevet / GuyTevet/motion-diffusion-model
We don't have to encode_text in each denoise timestep (but compute them at the beginning), do we?
- Dominant language
- Python
- Stars
- 4.1k
- Forks
- 462
- PR merge metrics
- No merged PRs in 30d
Description
https://github.com/GuyTevet/motion-diffusion-model/blob/8139dda55d90a58aa5a257ebf159b2ecfb78c632/model/mdm.py#L151C8-L151C8
```
class MDM(nn.Module):
......
def forward(self, x, timesteps, y=None):
"""
x: [batch_size, njoints, nfeats, max_frames], denoted x_t in the paper
timesteps: [batch_size] (int)
"""
bs, njoints, nfeats, nframes = x.shape
emb = self.embed_timestep(timesteps) # [1, bs, d]
force_mask = y.get('uncond', False)
if 'text' in self.cond_mode:
enc_text = self.encode_text(y['text'])
emb += self.embed_text(self.mask_cond(enc_text, force_mask=force_mask))
```
Contributor guide
No contributing guide indexed for this repository
Research direction
Start in model/mdm.py at MDM.forward and inspect the denoising caller to determine whether the same text is passed across timesteps. Verify how encode_text and the conditioning mask are used before deciding where embeddings could be reused. Done means text conditioning is computed once per sampling run without changing model outputs.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Refactor
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100