graykode / graykode/nlp-tutorial
5.1 Transformer may have wrong position embed
- 主要语言
- Jupyter Notebook
- 星标
- 14.9k
- 派生
- 3.9k
- PR 合并指标
- 30 天内没有已合并 PR
描述
1. in"class Encoder": enc_outputs = self.src_emb(enc_inputs) + self.pos_emb(torch.LongTensor([[1,2,3,4,0]]))
I think it may be: enc_outputs = self.src_emb(enc_inputs) + self.pos_emb(torch.LongTensor([[0,1,2,3,4]]))
2. in"class Decoder": dec_outputs = self.tgt_emb(dec_inputs) + self.pos_emb(torch.LongTensor([[5,1,2,3,4]]))
I think it may be: dec_outputs = self.tgt_emb(dec_inputs) + self.pos_emb(torch.LongTensor([[0,1,2,3,4]]))
贡献指南
调研方向
Locate the Encoder and Decoder classes in the relevant Jupyter Notebook and inspect how their positional-index tensors are constructed. Compare the current indices with the intended sequence-position convention, then run the notebook's Transformer example to verify the corrected behavior.
由索引模型根据 Issue 内容生成。
评估
- 技术栈
- pytorch
- 领域
- machine-learning
- Issue 类型
- 缺陷
- 难度
- 2/5
- 预计耗时
- 1-3 小时
- 活跃度
- 停滞
- 描述清晰度
- 基本清楚
- 新手友好度
- 45/100