allenai / allenai/unified-io-2.pytorch
Training over MoD paradigms mentioned in the paper.
- 主要语言
- Python
- 星标
- 78
- 派生
- 8
- PR 合并指标
- 30 天内没有已合并 PR
描述
For training part, as mentioned in the computing loss section -
```
from torch.nn import functional as F
from uio2.preprocessing import build_batch
preprocessed_example = preprocessor(
text_inputs="What is 1+1?", text_targets="2", target_modality="text")
batch = build_batch([preprocessed_example], device=model.device)
out = model(batch)
total_loss = 0
for modality, (logits, targets, mask) in out.items():
losses = F.cross_entropy(
logits.view(-1, logits.shape[-1]), targets.view(-1).to(torch.long), reduction="none")
total_loss += (losses.reshape(logits.shape[:2])*mask)/mask.sum()
print(total_loss)
```
be it any modality (text, image, or audio), the MoD paradigm being used in the preprocessor is just [S] - causal language modelling. I don't see other paradigms being used in the preprocessor. How is the training objective over different MoD paradigms achieved then?
贡献指南
这个仓库没有索引到贡献指南
评估
这个 Issue 还没有评估数据。