allenai / allenai/longformer

linear combination of losses

未关闭
#143 2 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
主要语言
Python
星标
2.2k
派生
285
PR 合并指标
30 天内没有已合并 PR

描述

Hi,

For the loss computation of hotpotqa, you mentioned
> We combine span, question classification, sentence, and paragraphs losses and train the model in a multitask way using linear combination of losses.

With `linear combination of losses`, do you mean multiplying a weight hyper-parameter to each loss? would the model learn to assign all the weights to be 0 to minimize the loss? Or did you set the weights to be constant?

Besides, for the span extraction loss, call the `or_softmax_cross_entropy_loss_one_doc` function as trivialqa?
for question type (yes/no/span) loss, call `torch.nn.CrossEntropyLoss()`
for supporting sentences and paragraphs loss, call `torch.nn.BCELoss(reduction='mean')`
Is this correct?

Thank you

贡献指南

这个仓库没有索引到贡献指南

评估

这个 Issue 还没有评估数据。

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。