how to use "video_pred()" function to predict image
- Dominant language
- Python
- Stars
- 381
- Forks
- 35
- PR merge metrics
- No merged PRs in 30d
Description
def video_pred(self, data):
data = self.preprocess(data)
embed = self.encoder(data)
states, _ = self.dynamics.observe(
embed[:6, :5], data["action"][:6, :5], data["is_first"][:6, :5]
)
recon = self.heads["decoder"](self.dynamics.get_feat(states))["image"].mode()[:6]
reward_post = self.heads["reward"](self.dynamics.get_feat(states)).mode()[:6]
init = {k: v[:, -1] for k, v in states.items()}
prior = self.dynamics.imagine_with_action(data["action"][:6, 5:], init)
openl = self.heads["decoder"](self.dynamics.get_feat(prior))["image"].mode()
reward_prior = self.heads["reward"](self.dynamics.get_feat(prior)).mode()
# observed image is given until 5 steps
model = torch.cat([recon[:, :5], openl], 1)
truth = data["image"][:6]
model = model
error = (model - truth + 1.0) / 2.0
return torch.cat([truth, model], 2)
Contributor guide
No contributing guide indexed for this repository
Research direction
Start from the shown video_pred function and inspect the expected structure of its data argument, including image, action, and is_first. Clarify the intended inputs, outputs, and prediction workflow, then document a reproducible usage example and expected result.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Documentation
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 28/100