baidu / baidu/Dialogue

the douban testset

Open
#13 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
442
Forks
93
PR merge metrics
No merged PRs in 30d

Description

hi,
I have questions about douban test data.
1. the pos/neg label is not 1:9 ?
2. I find that some context has all ten neg responses but no pos response, for such case, the denominator will be 0, thus how to calculate p1/10 result by using douban_evaluation.py?
3. I find that the testset len in data.pkl in douban is 6670, not equal to 10000 in test.txt, how to deal with this? is there any special point of these 6670 context?

thanks

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading douban_evaluation.py and comparing the Douban entries in data.pkl with test.txt, focusing on the label ratio, contexts containing only negative responses, and the differing test-set lengths. Done means the expected data relationship and the correct handling of the p1/10 calculation are clearly documented or confirmed.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data, machine-learning
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.