Evaluating precision@k/recall@k for users with <k "known positives"
- Dominant language
- Python
- Stars
- 5.1k
- Forks
- 724
- PR merge metrics
- No merged PRs in 30d
Description
Hi,
I have a question about the precision@k and recall@k evaluation functions. I can see from the source code that precision is calculated as:
precision = np.squeeze(np.array(ranks.sum(axis=1))) / k
Wouldn't this calculation underestimate performance if I have a users who have voted/rated
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reviewing the precision@k and recall@k evaluation functions and the shown ranks.sum(axis=1) / k calculation. Determine the expected behavior for users with fewer than k known positives and whether the current evaluation matches that definition; done means the behavior has a documented resolution or a targeted regression test if it is incorrect.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- numpy, python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100