EleutherAI / EleutherAI/elk

Can this code suppot free-form QA task?

Open
#298 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
225
Forks
30
PR merge metrics
No merged PRs in 30d

Description

I noticed that the datasets supported in the code are all multiple-choice and classification types, such as IMDB, QNLI, and BoolQ. Can the code in this repository support free-form types of tasks, such as TriviaQA and NQ? For free-form QA tasks, the possible answer space is infinite. How should the CCS method allocate X+ and X- labels for such tasks?

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.