Cloud-CV/EvalAI

View on GitHub

:cloud: :rocket: :bar_chart: :chart_with_upwards_trend: Evaluating state of the art in AI

Stars: 2k
Forks: 984
Open beginner issues: 0
Indexed issues: 206
Avg merge: 2h 54m
Merged PRs (30d): 14

Dominant language: Python
License: No license data
Last GitHub push: Sep 9, 2026
Latest indexed: Sep 15, 2026
Contributing guide: Contributing guide
Code of conduct: Code of conduct
Beginner labels: good first issue

Open All

206 open issues indexed

Add feature to invite users to host team using URL Open

backend GSOC-2025 medium-difficulty new-feature priority-high

Cloud-CV/EvalAI#1347 · 20 comments · 0 reactions · 1 assignee · Indexed Sep 15, 2026
Add constraints for setting challenge phase start and end date Open

backend easy_to_fix enhancement good first issue GSOC-2025 priority-high

Cloud-CV/EvalAI#1558 · 16 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Change the privacy level of files on AWS S3 Open

backend critical enhancement GSOC-2019 medium-difficulty priority-high

Cloud-CV/EvalAI#1647 · 10 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Add a progress bar showing in the percentage of file uploaded during a submission Open

enhancement good first issue GSOC-2025 new-feature new-frontend priority-low

Cloud-CV/EvalAI#2116 · 12 comments · 0 reactions · 1 assignee · Indexed Sep 15, 2026
Add remaining integration tests for the worker Open

Cloud-CV/EvalAI#2371 · 0 comments · 0 reactions · 1 assignee · Indexed Sep 15, 2026
Submission worker: Add successful_submission_flag==True condition before saving submission_output Open

Cloud-CV/EvalAI#2373 · 5 comments · 0 reactions · 1 assignee · Indexed Sep 15, 2026
Add and update existing tests for challenge submission API Open

backend easy_to_fix enhancement

Cloud-CV/EvalAI#2443 · 14 comments · 2 reactions · 0 assignees · Indexed Sep 15, 2026
UI Design: Log in and sign up page - To remove redundancy of the password character limit Open

Cloud-CV/EvalAI#2479 · 6 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
[Tests] Add tests for updating leaderboard data API Open

backend easy_to_fix enhancement good first issue GSOC-2025

Cloud-CV/EvalAI#2487 · 7 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
[code] Improve code structure for `submission_worker.py` and `remote_submission_worker.py`. Open

Cloud-CV/EvalAI#2542 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Remove the common errors Open

bug docs easy_to_fix good first issue GSOC-2025

Cloud-CV/EvalAI#2559 · 8 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Update the Contribution Guidelines on EvalAI-Docs Open

docs good first issue

Cloud-CV/EvalAI#2560 · 10 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Insecure requirement: Django version insecure Open

GSoC-2020 medium-difficulty priority-high

Cloud-CV/EvalAI#2564 · 15 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Use a single library for HTTP requests in jobs/utils Open

Cloud-CV/EvalAI#2570 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Improve documentation structure Open

Cloud-CV/EvalAI#2578 · 6 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Add integration tests for remote_submission_worker Open

Cloud-CV/EvalAI#2607 · 0 comments · 0 reactions · 1 assignee · Indexed Sep 15, 2026
Unused argument in update_submission_data Open

Cloud-CV/EvalAI#2609 · 2 comments · 1 reaction · 1 assignee · Indexed Sep 15, 2026
Analyze why maintainability estimate is low (D) Open

priority-low

Cloud-CV/EvalAI#2613 · 1 comment · 2 reactions · 0 assignees · Indexed Sep 15, 2026
Participant and Host can't remove their member Open

easy_to_fix enhancement frontend GSOC-2025

Cloud-CV/EvalAI#2648 · 4 comments · 0 reactions · 1 assignee · Indexed Sep 15, 2026
Host Deletion function performs no check for the type of Host Open

Cloud-CV/EvalAI#2664 · 6 comments · 1 reaction · 0 assignees · Indexed Sep 15, 2026
Tests for 'encode_data' and 'decode_data' in apps/base/utils.py Open

Cloud-CV/EvalAI#2669 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Create an API for getting submission counts based upon different statuses Open

GSoC-2020 medium-difficulty priority-high

Cloud-CV/EvalAI#2670 · 6 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Redundant worker dependencies Open

easy_to_fix enhancement GSOC-2025

Cloud-CV/EvalAI#2676 · 8 comments · 1 reaction · 1 assignee · Indexed Sep 15, 2026
Filter challenges based on name Open

backend easy_to_fix enhancement GSoC-2020 new-feature priority-high

Cloud-CV/EvalAI#2699 · 10 comments · 0 reactions · 1 assignee · Indexed Sep 15, 2026
Challenge Participation Invitation in Development Mode Open

Cloud-CV/EvalAI#2778 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Create unique URL for each phase under My Submission and All Submissions Page Open

backend enhancement frontend_v2

Cloud-CV/EvalAI#2785 · 2 comments · 0 reactions · 1 assignee · Indexed Sep 15, 2026
Reuse common code from leaderboard API Open

Cloud-CV/EvalAI#2800 · 0 comments · 1 reaction · 1 assignee · Indexed Sep 15, 2026
Check if the cluster is already created for a challenge in database Open

backend bug

Cloud-CV/EvalAI#2870 · 1 comment · 0 reactions · 1 assignee · Indexed Sep 15, 2026
Add tests for submission meta attributes pipeline. Open

Cloud-CV/EvalAI#2882 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Change the code to handle the simultaneous change in eval_script and test_annotation at the same time. Open

Cloud-CV/EvalAI#2883 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Poll to see if the worker container has started while fetching logs. Open

Cloud-CV/EvalAI#2886 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Send email to challenge hosts only when the worker reload is complete and successful Open

backend bug

Cloud-CV/EvalAI#2898 · 1 comment · 0 reactions · 1 assignee · Indexed Sep 15, 2026
Refactor aws_keys initialization. Open

Cloud-CV/EvalAI#2916 · 6 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Use custom email template for reset password feature Open

backend easy_to_fix good first issue

Cloud-CV/EvalAI#3069 · 34 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Certain items not correctly aligned in mobile view Open

Cloud-CV/EvalAI#3088 · 3 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Shifting from 'django-rest-auth' to 'dj-rest-auth' Open

Cloud-CV/EvalAI#3123 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Fix cyclic import dependency in challenges/aws_utils.py Open

Cloud-CV/EvalAI#3146 · 9 comments · 1 reaction · 1 assignee · Indexed Sep 15, 2026
`leaderboard_decimal_precision` and `is_leaderboard_order_descending` cannot be set from `challenge_config.yaml` Open

Cloud-CV/EvalAI#3184 · 5 comments · 0 reactions · 1 assignee · Indexed Sep 15, 2026
[Backend] Add feature to stop/delete code upload infrastructure Open

backend GSOC-2021 hard-to-fix

Cloud-CV/EvalAI#3231 · 6 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
[Backend] Add submission meta attributes in All submissions download API Open

backend GSOC-2021

Cloud-CV/EvalAI#3256 · 5 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
[Backend] Send email with challenge pk and queue name for a remote evaluation challenge Open

backend GSOC-2021

Cloud-CV/EvalAI#3257 · 9 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Improvements in Auth Token Card Open

Cloud-CV/EvalAI#3276 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Allow show and hide password Open

Cloud-CV/EvalAI#3277 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Profile completed percentage changing on refreshing Open

bug easy_to_fix frontend GSOC-2025

Cloud-CV/EvalAI#3278 · 3 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Host can't see private submissions toggle not available when no public submissions are available on leaderboard Open

Cloud-CV/EvalAI#3296 · 4 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Add metadata for challenges when a challenge link is shared Open

easy_to_fix good first issue GSOC-2025

Cloud-CV/EvalAI#3297 · 26 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
[Backend] Show fargate worker status on manage tab Open

backend GSOC-2021 medium-difficulty

Cloud-CV/EvalAI#3301 · 12 comments · 0 reactions · 1 assignee · Indexed Sep 15, 2026
Feature: Public Profile User + Follow Users + LeaderBoard Custom Sorting + Editorial on Past Challenges Open

Cloud-CV/EvalAI#3309 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 15, 2026
[Backend] Add start evaluation worker function call in challenge creation using github API Open

backend easy_to_fix GSOC-2021

Cloud-CV/EvalAI#3313 · 9 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
[Frontend] Fix leaderboard column css inconsistencies Open

frontend GSOC-2021

Cloud-CV/EvalAI#3331 · 5 comments · 0 reactions · 1 assignee · Indexed Sep 15, 2026

Cloud-CV/EvalAI

Get new issues in your inbox