Metaculus / Metaculus/metaculus
Update "Community Prediction" LeaderboardEntry meaning
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 113
- Forks
- 34
- Avg merge
- 16h 34m
- Merged PRs (30d)
- 30
Description
Currently, the "Community Prediction" gets a leaderboard entry which corresponds to the "recency_weighted" aggregation. This is true even for tournaments that exclusively use "unweighted" as the default aggregation.
This proposes to solve the issue by calculating the "Community Prediction" entry score by accumulating the scores of the default_aggregation_method by question. Sometimes this would be the recency_weighted aggregation, sometimes the unweighted one.
This would mean that we remove non "Community Prediction" entries from leaderboards.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Locate the LeaderboardEntry implementation and references to default_aggregation_method, then trace how leaderboard scores are calculated for Community Prediction. Done means the entry uses each question's default aggregation method and non-Community Prediction entries are removed from leaderboards; add or update tests for both aggregation modes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100