Cloud-CV / Cloud-CV/EvalAI

Add categories to challenges

Open
#3,892 6 comments 0 reactions 1 assignee Claimed by @Suryansh5545 View on GitHub
backend easy_to_fix frontend good first issue GSoC-2023 javascript new-feature python
Dominant language
Python
Stars
2k
Forks
984
Avg merge
2h 54m
Merged PRs (30d)
14

Description

### Description
This PR is similar to #3891.

Currently it is difficult to find challenges by categories (Workshop Challenge, Dataset Challenge, Challenge related to a Paper, New Environment Challenge), etc. The idea is to understand the reason behind the challenge being hosted (whether to test a benchmark a new dataset or test a new environment, for example). This issue is a first step towards that direction where we add the category a particular challenge belongs to and display it on the challenge page in this section:

Screenshot 2023-03-20 at 12 51 31 PM

The challenge domain must be made a part of the challenge configuration and the [EvalAI-Starters](https://github.com/Cloud-CV/EvalAI-Starters) repository.

This will also require you to add changes to the [challenge validation script](https://github.com/Cloud-CV/EvalAI/blob/19130726790e4cb79b0d2e8603dd6817b16dba69/apps/challenges/challenge_config_utils.py#L223) to ensure people put appropriate values for categories. Please create a PR with these dummies - `[Paper, Dataset, Environment, Workshop]`, before to converge to the set of domains we might want to include.

A next step for this one will be to allow users to edit the domain on the challenge page and provide a drop-down of things to select from.

### Pre-requisites/Learnings
This issue essentially requires background on a few things:

- An understanding of the [challenge configuration](https://evalai.readthedocs.io/en/latest/configuration.html).
- A high-level understanding of [EvalAI challenges](https://evalai.readthedocs.io/en/latest/host_challenge.html).
- An understanding of [our frontend](https://github.com/Cloud-CV/EvalAI/tree/master/frontend).
- How the [challenge validation](https://github.com/Cloud-CV/EvalAI/blob/19130726790e4cb79b0d2e8603dd6817b16dba69/apps/challenges/challenge_config_utils.py#L223) works.

### How to Approach
- You can start by setting up EvalAI locally. Set up the dev version locally (preferred) using the steps [here](https://github.com/Cloud-CV/EvalAI/tree/master#installation-instructions).
- Add a new field to the challenge model in Django.
- Play around with the challenge configuration and the frontend, put up screenshots on your PR.
- Update the [challenge validation script](https://github.com/Cloud-CV/EvalAI/blob/19130726790e4cb79b0d2e8603dd6817b16dba69/apps/challenges/challenge_config_utils.py#L223) to check for the newly added field.
- Reach out to us regarding any issues that you face or if you need any help with understanding any of the above.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.