Metaculus / Metaculus/metaculus

Tournament leaderboards should report attainable coverage

Open
#3,757 1 comment 0 reactions 1 assignee View on GitHub

@lsabor is already working on this.

Since Nov 5, 2025.

Medium Priority Stale
Dominant language
TypeScript
Stars
113
Forks
34
Avg merge
16h 34m
Merged PRs (30d)
30

Description

I.e. the coverage column should be sum(coverage) / sum(attainable_coverage) over all questions, including the ones you didn't forecast (where your coverage is 0).

Where attainable_coverage = (effective_close_time - open_time) / (scheduled_closed_time - open_time).

The "My score" section also needs some extra info to explain things:

  • the "Coverage" column becomes "Coverage (max)", where each row shows your coverage, and the maximum attainable coverage in parantheses: "58% (84%)".
  • the totals at the bottom have both:
    • "total coverage" = avg(coverage)
    • "total attainable coverage" = avg(attainable_coverage)
    • "effective coverage" = sum(coverage) / sum(attainable_coverage), i.e. should be the same as the value in the leaderboard.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.