Metaculus / Metaculus/metaculus

Leaderboard Updates - participation viewing modes

Open
#3,185 1 comment 1 reaction 1 assignee View on GitHub

@aseckin is already working on this.

Since Jul 26, 2025.

High Priority
Dominant language
TypeScript
Stars
113
Forks
34
Avg merge
16h 34m
Merged PRs (30d)
30

Description

Looks like it might be time for some more carefully thought out updates to the Leaderboard.

Primary objective:
Add a control so that the user can switch between Leaderboard modes:

  • view all participants
  • view only human participants
  • view only bot participants

Secondary objective:
make it clear which version of the leaderboard is "official":

  • if only humans are eligible for medals, how do we make it clear that the "all participants" version isn't the one that determines medals?
  • when you switch away from the "official" one, how do we indicate that the visible ranks are not what determine medals? etc.
  • see below for example ranking

First, we need a design spec. Thus assigning ticket to @aseckin


Next, we'll need some implementation, probably assigned to lsabor or ncarazon (avoiding premature tags)

Eng thoughts on implementation:
There is a bot_status field on Leaderboard and bot_leaderboard_status on `Project model which we should use to determine the initial view when leaderboard loads:

  • "exclude_and_hide" -> view only human participants
  • "exclude_and_show" (default) ->view all participants
  • "include" ->view all participants
  • "bots_only" ->view only bot participants
    Although, maybe "exclude_and_show" (default) ->view only human participants since they are the only ones who get medals in the real leaderboard

Ranking:
let's say the setting is default ("exclude_and_show"), and there are 5 participants with the following scores:

  1. human user 1 -> score=100
  2. bot user 1 -> score=90
  3. human user 2 -> score=80
  4. bot user 2 -> score=70
  5. human user 3 -> score=60
    If I'm viewing the "only human participants" version, then I'd expect:
    If the leaderboard isn't finalized yet:
Rank 1: human user 1 | score=100 | medal = None
Rank 2: human user 2 | score=80  | medal = None
Rank 3: human user 3 | score=60  | medal = None

After finalization:

Rank 1: human user 1 | score=100 | medal = Gold
Rank 2: human user 2 | score=80  | medal = Silver
Rank 3: human user 3 | score=60  | medal = Bronze

if I switch to the "all participants" version:
if the leaerboard isn't finalized yet:

Rank 1: human user 1 | score=100 | medal = None
Rank 2:   bot user 1 | score=90  | medal = None | somehow signal ineligibility
Rank 3: human user 2 | score=80  | medal = None
Rank 4:   bot user 2 | score=70  | medal = None | somehow signal ineligibility
Rank 5: human user 3 | score=60  | medal = None

if the leaderboard is finalized:

Rank 1: human user 1 | score=100 | medal = Gold
Rank 2:   bot user 2 | score=90  | medal = None | somehow signal ineligibility
Rank 3: human user 2 | score=80  | medal = Silver
Rank 4:   bot user 2 | score=70  | medal = None | somehow signal ineligibility
Rank 5: human user 3 | score=60  | medal = Bronze

So we'll need to implement a ranking-overwrite method on the front end to write the correct ranks when toggling

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.