codalab / codalab/codabench

Warning when queue is congested

Open
#1,693 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Enhancement P1
Dominant language
Python
Stars
176
Forks
74
Avg merge
5d 3h
Merged PRs (30d)
21

Description

If congestion in the queue detected (e.g. 10+ recent submissions in "Submitted"):

  • add a warning
  • add a confirmation pop-up when trying to make a new submission

The "Submitted" count depends on the queue of the competition (including for default queue).

Problem: sometimes it's stuck on "Submitted", even after service restart, so it may be considered congested indefinitely.


Suggestions:

  • Similarly to #1141, we could limit the "daily submissions limit" for competitions using the default queue
  • Having more workers would be beneficial now that Codabench is widely adopted
  • Should we optimize the queueing / ordering algorithm? Right now it is a simple first-in-first-out, so a single competition can block a whole queue.

Issue following #1446

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The issue names no files, tests, or entry points. Start by locating the competition queue and the handling of submissions in “Submitted,” then verify how queue congestion and stuck submissions are represented. Done means a congested queue is detected, users receive a warning and confirmation before submitting, and the behavior is covered by appropriate tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.