google-deepmind / google-deepmind/streamingqa

How to generate data formats for OBQA setting?

Open
#1 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
51
Forks
0
PR merge metrics
No merged PRs in 30d

Description

I was trying to reproduce the experiment for OBQA but I found that retrieval models need dataset format like NQ or TrivalQA. They have format like this:

> {
> 'id': '0',
> 'question': 'What element did Marie Curie name after her native land?',
> 'target': 'Polonium',
> 'answers': ['Polonium', 'Po (chemical element)', 'Po'],
> 'ctxs': [xxx,]
> }

where 'ctxs' contain the same number of chunks related to the question. I wonder how to generate such data for StreamingQA since it only provides one index of gold passage.
Thanks a lot !

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.