allenai / allenai/longformer

Does the multi-task training include predicting relevant paragraphs?

オープン
#156 コメント 2 件 リアクション 0 件 担当者 0 名 GitHub で見る
主要言語
Python
スター
2.2k
フォーク
286
PR マージ指標
30日以内にマージされた PR はありません

説明

Hi,
The page mention in section 6.1
> We use a two-stage model that first selects the most relevant paragraphs then passes them to a second stage for answer extraction. Both stages concatenate question and context into one sequence, run it through Longformer, then use task-specific prediction layers. We train the models in a multi-task way to predict relevant paragraphs, evidence sentences, answer spans and question types (yes/no/span) jointly.

My understanding is that the first stage concatenates question and the whole context as input to Longformer, and only predict relevant paragraphs as a binary classification task for each paragraph. The second stage concatenates question and the predict relevant paragraphs as input, train another Longformer model in a multi-task way to predict **relevant paragraphs,** evidence sentences, answer spans and question types (yes/no/span) jointly.

Does the multi-task training in stage2 still need to include predicting relevant paragraphs?

Thank you

コントリビューションガイド

このリポジトリのコントリビューションガイドは索引されていません

評価

この issue はまだ評価されていません。

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。