dice-group / dice-group/gerbil

GsinKB A2KB

Open
#335 4 comments 0 reactions 0 assignees View on GitHub
type:question
Dominant language
Java
Stars
231
Forks
55
PR merge metrics
No merged PRs in 30d

Description

I am currently in the process of evaluating previous work and comparing their scores to the Gerbil platform (to observe discrepancies). In most of the previous work that I found which used the AIDA datasets, emerging entities were ignored as no ground truth was available. Now I read that in some cases, these out of KB entities were added, which could cause a discrepancy in the scores as previous work had no method of comparing the two. As such, for the D2KB task, I found it very convenient that I was able to report GSinKB scores as these scores, as far as I know, ignore such occurrences.

Now I am one step further and I would like to begin the evaluation of A2KB, but there is no GSinKB score available when using the cloud platform you provided. What is the reason for not incorporating the GSinKB scores for the A2KB task and is there a way that I could enable such scores?

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.