Fact extraction project

Need to think about the next step after sentence classification, i.e. fact extraction

1. An assessment of how our methods perform wrt annotation
   Set up a workflow

• Extract sentences with BioBert (current production pipeline)
  -- Feed data to an LLM (GPT-4o) asking to extract an annotation
• Use full text and ask an LLM (GPT-4o) to extract an annotation
  -- A curator creates an annotation from text

What are the differences between the three methods?
How do we evaluate/score the results? Annotation based?

2. An assessment of how our methods perform on literature from other organisms
   Focus on co-published species?
   Prioritize SGD>ZFIN>Xenbase
   How to get the list of copublished papers from postgres
   A more direct comparison with other methods, e.g. rule-based methods such as Textpresso category searches or RLIMS-P (they have an api)

Start by reviewing the current production sentence-classification pipeline using BioBert and the proposed GPT-4o workflows. Define how curator annotations, full-text extraction, cross-organism literature, PostgreSQL paper lists, Textpresso searches, and the RLIMS-P API would be compared; done means an agreed evaluation workflow and scoring approach.