codalab / codalab/codabench

CircleCI tests: use the `site_worker`

Open
#2,541 0 comments 0 reactions 2 assignees View on GitHub

@IdirLISN is already working on this.

Since Sep 15, 2026.

Post-it
Dominant language
Python
Stars
176
Forks
74
Avg merge
5d 3h
Merged PRs (30d)
21

Description

Some points to improve:

  • The site_worker is not used in CircleCI, all tasks are processed by Django itself. We need to fix this.

  • CircleCI tests are skipping the multitask bundles.

Relevant code:

https://github.com/codalab/codabench/blob/develop/tests/test_submission.py

# Skip this test if in the CI
@pytest.mark.skipif(ci, reason="Works locally but fails in the CI because of CELERY_TASK_ALWAYS_EAGER = True")
[...]

Tests:

test_auth.py::test_auth[firefox] PASSED                                  [  6%]
test_account_creation.py::test_FailedAccountCreationUsername[firefox] PASSED [ 12%]
test_account_creation.py::test_account_creation[firefox] PASSED          [ 18%]
test_account_creation.py::test_loginIntoActivatedAccount[firefox] PASSED [ 25%]
test_competition.py::test_competition_upload[firefox] PASSED             [ 31%]
test_competition.py::test_manual_competition_creation[firefox] PASSED    [ 37%]
test_submission.py::test_v2_code[firefox] PASSED                         [ 43%]
test_submission.py::test_v2_results[firefox] PASSED                      [ 50%]
test_submission.py::test_v2_results_failure[firefox] PASSED              [ 56%]
test_submission.py::test_v2_miniautoml[firefox] PASSED                   [ 62%]
test_submission.py::test_v15_sncf[firefox] PASSED                        [ 68%]
test_submission.py::test_v15_iris_code[firefox] PASSED                   [ 75%]
test_submission.py::test_v15_iris_results[firefox] PASSED                [ 81%]
test_submission.py::test_v18_autowsl[firefox] PASSED                     [ 87%]
test_submission.py::test_v2_multiTask[firefox] SKIPPED (Works locall...) [ 93%]
test_submission.py::test_v2_multiTaskFactSheet[firefox] SKIPPED (Wor...) [100%]

================== 14 passed, 2 skipped in 209.36s (0:03:29) ===================

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.