Cloud-CV / Cloud-CV/EvalAI

500 Internal Server Error when validating challenge config via GitHub Actions

Open
#4,603 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
2k
Forks
984
Avg merge
2h 54m
Merged PRs (30d)
14

Description

Hi EvalAI team,

I'm currently setting up a challenge using GitHub Actions, following the [official documentation](https://github.com/Cloud-CV/EvalAI/blob/master/docs/source/host_challenge.md#host-challenge-using-github). I've pushed the challenge files to the challenge branch of my repository:
🔗 https://github.com/image-based-screen-industrial-detect/industrial-defect-challenge/tree/challenge

However, the GitHub Action fails during validation with the following error:

Following errors occurred while validating the challenge config:
500 Server Error: Internal Server Error for url: https://eval.ai/api/challenges/challenge/challenge_host_team/4014/validate_challenge_config/

Exiting the challenge_processing_script.py script after failure
Error: Process completed with exit code 1.

Exiting the challenge_processing_script.py script after failure
Error: Process completed with exit code 1.
What I’ve already verified:

The challenge_config.yaml file exists and passes YAML validation

All required files exist, including:

challenge_details/*.html

evaluation_script/evaluation.py (with a valid evaluate() function)

test_annotations/test.json

Paths in the config are correct

The repo secrets (EVALAI_HOST_URL, EVALAI_AUTH_TOKEN, HOST_TEAM_PK) are properly set

Still, the API call to validate the config results in a 500 error, which makes it hard to debug further on my side.

Could you help investigate what might be causing this issue on your backend or guide me on how to further debug?

Thank you!

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.