google-deepmind / google-deepmind/streamingqa
How to generate data formats for OBQA setting?
Open
- Dominant language
- Python
- Stars
- 51
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Description
I was trying to reproduce the experiment for OBQA but I found that retrieval models need dataset format like NQ or TrivalQA. They have format like this:
> {
> 'id': '0',
> 'question': 'What element did Marie Curie name after her native land?',
> 'target': 'Polonium',
> 'answers': ['Polonium', 'Po (chemical element)', 'Po'],
> 'ctxs': [xxx,]
> }
where 'ctxs' contain the same number of chunks related to the question. I wonder how to generate such data for StreamingQA since it only provides one index of gold passage.
Thanks a lot !
Contributor guide
Assessment
This issue has not been assessed yet.