facebookresearch / facebookresearch/HalluLens
Consistency of prompts at Task 1
Open
- Dominant language
- Python
- Stars
- 84
- Forks
- 19
- PR merge metrics
- No merged PRs in 30d
Description
https://github.com/facebookresearch/HalluLens/blob/80307ac6bc9fd396a38b7a0de4196b931611b728/tasks/shortform/precise_wikiqa.py#L19C1-L37C1
The prompt mentions correct, incorrect, and unverifiable as definitions, but the last instruction asks for 'Yes or No', which seems inconsistent.
Contributor guide
Assessment
This issue has not been assessed yet.