the douban testset
- Dominant language
- Python
- Stars
- 442
- Forks
- 93
- PR merge metrics
- No merged PRs in 30d
Description
hi,
I have questions about douban test data.
1. the pos/neg label is not 1:9 ?
2. I find that some context has all ten neg responses but no pos response, for such case, the denominator will be 0, thus how to calculate p1/10 result by using douban_evaluation.py?
3. I find that the testset len in data.pkl in douban is 6670, not equal to 10000 in test.txt, how to deal with this? is there any special point of these 6670 context?
thanks
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reading douban_evaluation.py and comparing the Douban entries in data.pkl with test.txt, focusing on the label ratio, contexts containing only negative responses, and the differing test-set lengths. Done means the expected data relationship and the correct handling of the p1/10 calculation are clearly documented or confirmed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data, machine-learning
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100