Cloud-CV/evalai-cli

View on GitHub

:cloud: :rocket: Official EvalAI Command Line Tool

Stars: 57
Forks: 70
Open beginner issues: 0
Indexed issues: 56

Dominant language: Python
License: BSD-3-Clause
Last GitHub push: Jun 14, 2026
Latest indexed: Sep 15, 2026
Contributing guide: Contributing guide
Code of conduct: No code of conduct
Beginner labels: good first issue help wanted
PR merge metrics: No merged PRs in 30d

Beginner friendly Everything

Open All

56 open issues indexed

Python installation error Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

Cloud-CV/evalai-cli#383 · 1 comment · Opened Mar 6, 2025
Enhance Security Integration evalai-cli with OSS-Fuzz for Continuous Fuzz Testing Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

Cloud-CV/evalai-cli#382 · 1 comment · Opened Dec 14, 2024
Add proper exception handling Open

Difficulty 3/5 1-2 days Newbie friendliness 38/100

Cloud-CV/evalai-cli#363 · 1 comment · Opened Mar 21, 2024
Can not push leaderboard test docker to EvalAI platform Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

Cloud-CV/evalai-cli#359 · 1 comment · 1 reaction · Opened Jan 25, 2024
404 Client Error: Not Found for url: https://eval.ai/api/challenges/challenge/all Open

Difficulty 3/5 1-2 days Newbie friendliness 28/100

Cloud-CV/evalai-cli#347 · 3 comments · 1 reaction · Opened Sep 5, 2023
Please consider loosening dependency requirements Open

Difficulty 2/5 1-3 hours Newbie friendliness 45/100

Cloud-CV/evalai-cli#343 · 1 reaction · Opened Mar 15, 2023
Submission metadata appears to be dropped when submitting large files Open

Difficulty 2/5 1-3 hours Newbie friendliness 45/100

Cloud-CV/evalai-cli#334 · Opened Dec 4, 2021
Add command to download all submissions of a challenge Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

Cloud-CV/evalai-cli#321 · 10 comments · Opened Mar 17, 2021
Keyerror on logging errors when found in making a call to api Open

Difficulty 2/5 1-3 hours Newbie friendliness 45/100

Cloud-CV/evalai-cli#313 · 8 comments · Opened Jan 26, 2021
Add tests for get_user_access_token method Open

Difficulty 2/5 1-3 hours Newbie friendliness 48/100

Cloud-CV/evalai-cli#307 · 5 comments · Opened Jan 23, 2021
Write tests for uploading files through pre-signed urls Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

Cloud-CV/evalai-cli#278 · 1 comment · Opened Sep 6, 2020
socket.timeout error when submitting image using `evalai push` Open

priority-high

Difficulty 4/5 3-5 days Newbie friendliness 28/100

Cloud-CV/evalai-cli#275 · 3 comments · Opened May 7, 2020
Add supported python versions to documentation Open

Difficulty 3/5 1-2 days Newbie friendliness 45/100

Cloud-CV/evalai-cli#272 · Opened Feb 17, 2020
[docs] Add documentation for login command Open

Difficulty 2/5 1-3 hours Newbie friendliness 45/100

Cloud-CV/evalai-cli#271 · 3 comments · Opened Jan 27, 2020
Remove replication of code to get current host Open

Difficulty 2/5 1-3 hours Newbie friendliness 48/100

Cloud-CV/evalai-cli#264 · 2 comments · Opened Jan 19, 2020
EvalAI-cli Utils not fully tested Open

Difficulty 3/5 1-2 days Newbie friendliness 42/100

Cloud-CV/evalai-cli#244 · Opened Jan 3, 2020
[enhancement] Use different color for command usage examples within messages Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

Cloud-CV/evalai-cli#243 · 5 comments · Opened Jan 2, 2020
Remove duplicated code Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

Cloud-CV/evalai-cli#241 · 1 comment · Opened Jan 1, 2020
Add Dark Mode option on the docs Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

Cloud-CV/evalai-cli#239 · 1 comment · Opened Dec 30, 2019
Add Code of Conduct file Open

Difficulty 1/5 Under an hour Newbie friendliness 45/100

Cloud-CV/evalai-cli#238 · Opened Dec 30, 2019
[enhancement] Define wrapper functions for click's echo function Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

Cloud-CV/evalai-cli#234 · 2 comments · Opened Dec 28, 2019
[bug] Improper implementation of click causes errors while handling exceptions Open

Difficulty 2/5 1-3 hours Newbie friendliness 45/100

Cloud-CV/evalai-cli#232 · Opened Dec 28, 2019
Challenge submission error handling Open

Difficulty 3/5 1-2 days Newbie friendliness 45/100

Cloud-CV/evalai-cli#231 · 1 comment · Opened Dec 28, 2019
Fix Progress bar for uploading a docker image. Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

Cloud-CV/evalai-cli#229 · Opened Dec 27, 2019
[new feature] Create shell script to set up development environment Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

Cloud-CV/evalai-cli#225 · Opened Dec 23, 2019
Remove replication of code to get_token command. Open

Difficulty 2/5 1-3 hours Newbie friendliness 45/100

Cloud-CV/evalai-cli#223 · 10 comments · Opened Dec 22, 2019
Hardcoded AWS region Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

Cloud-CV/evalai-cli#218 · 1 reaction · Opened Dec 18, 2019
[error message] get_token command gives out wrong message Open

Difficulty 1/5 Under an hour Newbie friendliness 55/100

Cloud-CV/evalai-cli#216 · Opened Dec 18, 2019
[bug] token.json empty causes cli to crash while using get_token command Open

Difficulty 2/5 1-3 hours Newbie friendliness 35/100

Cloud-CV/evalai-cli#215 · 1 comment · Opened Dec 18, 2019
[New feature] Add feature to reset_token (logout) Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

Cloud-CV/evalai-cli#205 · 1 comment · Opened Dec 13, 2019
[PyPI docs] Link to project homepage on PyPI broken Open

Difficulty 1/5 Under an hour Newbie friendliness 45/100

Cloud-CV/evalai-cli#200 · Opened Dec 12, 2019
[Docs] readthedocs documentation build failed. Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

Cloud-CV/evalai-cli#199 · Opened Dec 12, 2019
[Docs] Custom backend configuration documentation Open

Difficulty 1/5 1-3 hours Newbie friendliness 48/100

Cloud-CV/evalai-cli#180 · 1 reaction · Opened Nov 24, 2019
[Doc] : Readme.md - Description and CLI commands to be updated. Open

Difficulty 2/5 1-3 hours Newbie friendliness 35/100

Cloud-CV/evalai-cli#179 · 1 comment · Opened Nov 23, 2019
Many files need to be updated and added in evalai-cli/docs/ Open

Difficulty 5/5 Over a week Newbie friendliness 15/100

Cloud-CV/evalai-cli#178 · 4 comments · Opened Nov 22, 2019
No code written under evalai-cli/evalai/urls.py Open

Difficulty 5/5 Over a week Newbie friendliness 15/100

Cloud-CV/evalai-cli#177 · Opened Nov 22, 2019
calling evaluate() infinite times if submission id is NoneType Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

Cloud-CV/evalai-cli#176 · Opened Nov 12, 2019
download_file arguments - bucket and key Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

Cloud-CV/evalai-cli#175 · Opened Nov 9, 2019
Some CLI commands not working Open

Difficulty 3/5 1-2 days Newbie friendliness 25/100

Cloud-CV/evalai-cli#174 · 2 comments · Opened Nov 9, 2019
[bug, #todo] hardcoded host url Open

Difficulty 2/5 1-3 hours Newbie friendliness 35/100

Cloud-CV/evalai-cli#172 · Opened Nov 6, 2019
[docs] Documentation on usage of download_file Open

Difficulty 2/5 1-3 hours Newbie friendliness 35/100

Cloud-CV/evalai-cli#171 · 3 comments · Opened Nov 6, 2019
Fix error message when submission limit is exhausted for a phase Open

Difficulty 3/5 1-2 days Newbie friendliness 25/100

Cloud-CV/evalai-cli#169 · 1 comment · Opened Jul 9, 2019
Add docs for auth token Open

Difficulty 2/5 1-3 hours Newbie friendliness 35/100

Cloud-CV/evalai-cli#160 · Opened Apr 7, 2019
Add unit tests for download_file command Open

enhancement good first issue GSOC-2019 help wanted

Difficulty 3/5 1-2 days Newbie friendliness 25/100

Cloud-CV/evalai-cli#157 · 10 comments · Opened Mar 30, 2019
Check if docker is running on host machine before using evalai push Open

bug enhancement good first issue GSOC-2019 help wanted priority-high

Difficulty 2/5 1-3 hours Newbie friendliness 45/100

Cloud-CV/evalai-cli#154 · 3 comments · Opened Mar 26, 2019
Add unit tests for evalai push command Open

enhancement good first issue GSOC-2019 help wanted

Difficulty 3/5 1-2 days Newbie friendliness 25/100

Cloud-CV/evalai-cli#145 · 3 comments · Opened Feb 15, 2019
Add command for creating a challenge Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

Cloud-CV/evalai-cli#119 · 6 comments · Opened Dec 14, 2018
Show error message while submitting when user doesn't participate in a challenge. Open

bug good first issue help wanted

Difficulty 3/5 1-2 days Newbie friendliness 45/100

Cloud-CV/evalai-cli#106 · 6 comments · Opened Sep 27, 2018
Multiple tokens Open

enhancement good first issue help wanted

Difficulty 5/5 Over a week Newbie friendliness 25/100

Cloud-CV/evalai-cli#105 · 5 comments · Opened Sep 13, 2018
Upcoming Command not working Open

Difficulty 1/5 Under an hour Newbie friendliness 50/100

Cloud-CV/evalai-cli#104 · 4 comments · Opened Sep 13, 2018

Cloud-CV/evalai-cli

Get new issues in your inbox