allenai / allenai/longformer

Does the multi-task training include predicting relevant paragraphs?

Offen
#156 2 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
Vorherrschende Sprache
Python
Sterne
2.2k
Forks
286
PR-Merge-Kennzahlen
Keine gemergten PRs in 30 T.

Beschreibung

Hi,
The page mention in section 6.1
> We use a two-stage model that first selects the most relevant paragraphs then passes them to a second stage for answer extraction. Both stages concatenate question and context into one sequence, run it through Longformer, then use task-specific prediction layers. We train the models in a multi-task way to predict relevant paragraphs, evidence sentences, answer spans and question types (yes/no/span) jointly.

My understanding is that the first stage concatenates question and the whole context as input to Longformer, and only predict relevant paragraphs as a binary classification task for each paragraph. The second stage concatenates question and the predict relevant paragraphs as input, train another Longformer model in a multi-task way to predict **relevant paragraphs,** evidence sentences, answer spans and question types (yes/no/span) jointly.

Does the multi-task training in stage2 still need to include predicting relevant paragraphs?

Thank you

Beitragsleitfaden

Für dieses Repository ist kein Beitragsleitfaden indexiert

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.