Owner

codalab

1 indexed repository · View on GitHub

codabench
Codabench is a flexible, easy-to-use and reproducible benchmarking platform. Check our paper: https://hubs.li/Q01fwRWB0

Python · 176 stars

Beginner friendly Everything

Open All

Improve collaboration between co-organizers Open

Enhancement

codalab/codabench#2180 · 1 comment · 2 assignees · Opened Feb 11, 2026
Bulk download results Open

Enhancement Post-it

Difficulty 3/5 1-2 days Newbie friendliness 45/100

codalab/codabench#2160 · Opened Feb 5, 2026
Running User Submissions on Kubernetes Open

Enhancement

Difficulty 5/5 Over a week Newbie friendliness 25/100

codalab/codabench#2152 · Opened Feb 5, 2026
Ability to email participants filtered by submission or registration status Open

Enhancement

Difficulty 4/5 3-5 days Newbie friendliness 35/100

codalab/codabench#2109 · 1 comment · Opened Jan 20, 2026
No metada file Open

V1-compatibility

Difficulty 3/5 1-2 days Newbie friendliness 25/100

codalab/codabench#2108 · 5 comments · Opened Jan 19, 2026
How to make leaderboard public only for one phase and hide scores in My Submissions tab? Open

Enhancement

Difficulty 4/5 3-5 days Newbie friendliness 25/100

codalab/codabench#2088 · 1 comment · Opened Dec 16, 2025
Cleaning data from deleted competitions Open

Enhancement

codalab/codabench#2059 · 5 comments · 1 assignee · Opened Nov 11, 2025
Add Configurable File Size Limits for Competition Submissions Open

Discussion

Difficulty 4/5 3-5 days Newbie friendliness 35/100

codalab/codabench#2049 · 2 comments · Opened Oct 22, 2025
Request for arm64 Docker image and build method Open

Setup

Difficulty 5/5 Over a week Newbie friendliness 25/100

codalab/codabench#2029 · 2 comments · Opened Oct 11, 2025
CircleCI tests improvement Open

Enhancement Post-it

Difficulty 4/5 3-5 days Newbie friendliness 45/100

codalab/codabench#2023 · 2 comments · Opened Oct 7, 2025
Problem with leaderboard Open

Competition-specific UI

Difficulty 3/5 1-2 days Newbie friendliness 35/100

codalab/codabench#2000 · 8 comments · Opened Sep 9, 2025
InquiryQuestion about permission validation logic relying on custom checks instead of Django's native system Open

Discussion

Difficulty 5/5 Over a week Newbie friendliness 25/100

codalab/codabench#1996 · 4 comments · Opened Sep 4, 2025
Port mapping for public network visiting of private network CodaBench instance Open

Setup

Difficulty 4/5 3-5 days Newbie friendliness 35/100

codalab/codabench#1990 · 1 comment · Opened Aug 28, 2025
Creating a Django group to limit who can create competitions Open

Discussion

Difficulty 3/5 1-2 days Newbie friendliness 25/100

codalab/codabench#1961 · 2 comments · Opened Aug 5, 2025
Improve Codabench deployment documentation Open

Documentation Setup

Difficulty 4/5 3-5 days Newbie friendliness 35/100

codalab/codabench#1917 · 10 comments · Opened Jul 9, 2025
Blacklists and Whitelists feature for Admins and Organizers Open

Enhancement

Difficulty 5/5 Over a week Newbie friendliness 25/100

codalab/codabench#1908 · Opened Jul 4, 2025
All issues and suggestions related to "factsheet" Open

Bug Enhancement

Difficulty 5/5 Over a week Newbie friendliness 25/100

codalab/codabench#1904 · Opened Jul 3, 2025
[Question] Multiple grouper columns in the leaderboard Open

Enhancement V1-compatibility

Difficulty 5/5 Over a week Newbie friendliness 25/100

codalab/codabench#1901 · 4 comments · Opened Jul 1, 2025
Codabench Datasets Open

Enhancement Post-it

Difficulty 5/5 Over a week Newbie friendliness 25/100

codalab/codabench#1898 · 7 comments · Opened Jun 27, 2025
Competition tags, filters, search bar Open

Enhancement P1

Difficulty 5/5 Over a week Newbie friendliness 25/100

codalab/codabench#1895 · 2 comments · Opened Jun 25, 2025
Rabbit and Flower sites http without https option? Open

Setup

Difficulty 4/5 3-5 days Newbie friendliness 25/100

codalab/codabench#1849 · 6 comments · Opened May 14, 2025
Django doesn't recognize HTTPS to set protocol correctly in emails Open

Documentation Setup

Difficulty 3/5 1-2 days Newbie friendliness 45/100

codalab/codabench#1844 · 2 comments · 1 reaction · Opened May 6, 2025
Limiting signups to specific domains Open

Enhancement

Difficulty 4/5 3-5 days Newbie friendliness 38/100

codalab/codabench#1827 · 10 comments · Opened Apr 14, 2025
Add documentation about mail server configuration for online development Open

Documentation Post-it

Difficulty 2/5 1-3 hours Newbie friendliness 45/100

codalab/codabench#1812 · 2 comments · Opened Apr 9, 2025
Time Zone Issue in Edit Phase Dialog: Start Time and End Time Cannot be Saved Correctly Open

Bug

Difficulty 4/5 3-5 days Newbie friendliness 35/100

codalab/codabench#1796 · 10 comments · Opened Mar 25, 2025
UI on Mobile Devices: Text Overlap Issue Open

UI

Difficulty 4/5 3-5 days Newbie friendliness 25/100

codalab/codabench#1795 · 4 comments · Opened Mar 25, 2025
Add link between pages Open

Enhancement UI

Difficulty 1/5 Under an hour Newbie friendliness 25/100

codalab/codabench#1737 · 2 comments · Opened Jan 28, 2025
Influx of spam participant joiners Open

Discussion

Difficulty 4/5 3-5 days Newbie friendliness 25/100

codalab/codabench#1733 · 6 comments · Opened Jan 17, 2025
Possibility to highlight submissions Open

Enhancement UI

Difficulty 5/5 Over a week Newbie friendliness 25/100

codalab/codabench#1723 · 1 comment · Opened Jan 8, 2025
Overview Analytics: y axis is not scaling properly Open

Enhancement UI

Difficulty 3/5 1-2 days Newbie friendliness 38/100

codalab/codabench#1720 · Opened Dec 30, 2024
Participant cannot leave organization independently Open

Beginner Enhancement

Difficulty 4/5 3-5 days Newbie friendliness 35/100

codalab/codabench#1714 · Opened Dec 22, 2024
Submit independently to tasks Open

Discussion Enhancement

Difficulty 5/5 Over a week Newbie friendliness 20/100

codalab/codabench#1711 · 2 comments · Opened Dec 19, 2024
Warning when queue is congested Open

Enhancement P1

Difficulty 4/5 3-5 days Newbie friendliness 35/100

codalab/codabench#1693 · 1 comment · Opened Dec 5, 2024
Feature Request: Support for Separate Leaderboards or Parallel Phases Open

Enhancement

Difficulty 5/5 Over a week Newbie friendliness 25/100

codalab/codabench#1679 · 1 comment · Opened Nov 21, 2024
Storage Analytics never completes Open

Bug

Difficulty 4/5 3-5 days Newbie friendliness 30/100

codalab/codabench#1661 · 4 comments · Opened Nov 13, 2024
Multiple leaderboards discussion Open

Discussion V1-compatibility

Difficulty 5/5 Over a week Newbie friendliness 25/100

codalab/codabench#1553 · 4 comments · Opened Jul 25, 2024
Wrong leaderboard ordering and `N/a` scores discussion Open

Bug Discussion P1

Difficulty 5/5 Over a week Newbie friendliness 25/100

codalab/codabench#1540 · 8 comments · 1 reaction · Opened Jul 18, 2024
Export terms accepted by participants Open

Enhancement

Difficulty 4/5 3-5 days Newbie friendliness 35/100

codalab/codabench#1538 · Opened Jul 17, 2024
Ability to create registration forms Open

Enhancement

Difficulty 5/5 Over a week Newbie friendliness 25/100

codalab/codabench#1498 · Opened Jun 21, 2024
Feature request - Bulk "Cancel Submissions" Open

Enhancement UI

Difficulty 4/5 3-5 days Newbie friendliness 38/100

codalab/codabench#1487 · 4 comments · Opened Jun 18, 2024
Feature request: leaderboard visibility should be configurable by phase Open

Enhancement

Difficulty 4/5 3-5 days Newbie friendliness 42/100

codalab/codabench#1396 · 1 comment · Opened Apr 10, 2024
The "Dump with files" option produces duplicate files Open

Bug

Difficulty 4/5 3-5 days Newbie friendliness 35/100

codalab/codabench#1393 · 16 comments · Opened Apr 9, 2024
Can't change rabbit credentials Open

Bug

Difficulty 3/5 1-2 days Newbie friendliness 25/100

codalab/codabench#1337 · 3 comments · Opened Feb 16, 2024
Compute worker issues Open

Bug P1 Workers / Queues

Difficulty 5/5 Over a week Newbie friendliness 20/100

codalab/codabench#1205 · 10 comments · Opened Oct 31, 2023
Azure storage issue Open

Bug Setup

Difficulty 4/5 3-5 days Newbie friendliness 35/100

codalab/codabench#1192 · 2 comments · Opened Oct 21, 2023
Refresh needed in the page "My submissions" and unstable live logging Open

Enhancement UI

Difficulty 5/5 Over a week Newbie friendliness 25/100

codalab/codabench#1097 · 7 comments · Opened Aug 21, 2023
Option to cap logs Open

Discussion Enhancement

Difficulty 4/5 3-5 days Newbie friendliness 35/100

codalab/codabench#875 · Opened May 12, 2023
Storage and clean up Open

Enhancement Post-it

Difficulty 5/5 Over a week Newbie friendliness 25/100

codalab/codabench#713 · 9 comments · Opened Sep 29, 2022
Public submissions should be downloadable from the leaderboard Open

Enhancement UI

Difficulty 3/5 1-2 days Newbie friendliness 35/100

codalab/codabench#609 · 1 comment · Opened Dec 18, 2020
Add "Ingestion during scoring" option to the "Create Task" modal Open

Post-it

Difficulty 3/5 1-2 days Newbie friendliness 64/100

codalab/codabench#387 · 8 comments · Opened Mar 4, 2020

codalab

Get new issues in your inbox