codalab / codalab/codabench

How to make leaderboard public only for one phase and hide scores in My Submissions tab?

Open
#2,088 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Enhancement
Dominant language
Python
Stars
176
Forks
74
Avg merge
5d 3h
Merged PRs (30d)
21

Description

Hello Codabench team,

Thank you for maintaining Codabench. I am currently setting up a competition with two distinct phases and would appreciate some guidance to ensure the configuration behaves as intended.

Intended setup

I would like to have two phases:

  1. Public test phase

    • Participants receive performance feedback.
    • The leaderboard is visible and updated normally.
  2. Blind test phase

    • No performance feedback is shown to participants.
    • Scores and detailed results should not appear in the My Submissions area.
    • The leaderboard should remain hidden during this phase and only be made public after the competition ends.
    • Submissions are used only to compute the final ranking.
Questions
  1. Is it possible to make the leaderboard public only for the public test phase, while keeping it hidden for the blind test phase?
  2. Is there a supported way to completely hide scores from participants in the My Submissions area for the blind test phase?

I have already tried disabling detailed results and enabling output hiding options for the blind phase, but participants can still see score-related information in My Submissions. I am unsure whether this behavior is expected or if I am misconfiguring something.

Any clarification or example configuration would be greatly appreciated.

Thank you very much for your time and support.

Best regards

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the competition phase settings and the My Submissions behavior described in the issue, focusing on leaderboard visibility, detailed results, and output hiding. Reproduce the two-phase configuration if possible and determine whether the requested controls are supported or need documentation; done means providing a clear configuration example or documenting the limitation.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.