allenai / allenai/scibert

no N/A label in relation classification

Abierto
#123 0 comentarios 0 reacciones 0 asignados Ver en GitHub
Lenguaje dominante
Python
Estrellas
1.7k
Forks
232
Métricas de merge de PR
Sin PR fusionados en 30 d

Descripción

I noticed that in the datasets of relation classification, there is no entity pair of 'N/A' label (no relation between two entities). In the other words, entities without any relations are not considered in training and inference. For example, as I check in the dataset of SCIIE, I find only the six relations are considered: EVALUATE-FOR, PART-OF, USED-FOR, FEATURE-OF, CONJUNCTION, COMPARE, HYPONYM-OF. So basically in the evaluation, SciBERT takes a pair of entities that is assumed to to have an existing relation and classifies.

But obviously SCIIE have a lot of entity pairs that do not have any of the six relations, and they are not considered at inference time in this case. This is very different from other works that extract relations with gold entities, which do not assume a pair of input entities would have an existing relation beforehand. Specifically, they make inference for all possible pairs of entities, where the relation is classified over ( defined relations types + N/A relation) .

Could you clarify why N/A relation is not included in the training and the evaluation of SciBERT? Or am I missing something?

Guía de contribución

No hay ninguna guía de contribución indexada para este repositorio

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.