allenai / allenai/contrast-sets

DROP Original Test Dataset

Open
#21 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
60
Forks
9
PR merge metrics
No merged PRs in 30d

Description

The DROP contrastive set contains the original question, perturbed question and answer to the perturbed question but not the answer to the original question. A quick look at the python file to compute consistency for the DROP dataset tells that we need to feed in the predictions made for the original test set as an input. Could you please point me to the original DROP test set that was used to generate the contrast set?

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.