Stop worker after wrong answers
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1k
- Forks
- 412
- Avg merge
- 6d 10h
- Merged PRs (30d)
- 3
Description
See http://www.freelists.org/post/contestms/Stopping-workers-after-wrong-answers. Basically the proposal is to make the workers at least partially score-aware, so that they can stop an evaluation after some testcases have failed, in order to save on resources.
This is difficult to do with our design. This issue is considered in very low priority.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the linked proposal about stopping workers after wrong answers, then inspect the worker and evaluation architecture to determine where score awareness could fit. The issue names no files or tests, and the desired failure threshold is unspecified; done would mean safely stopping evaluations after enough failed test cases while preserving correct results and reducing resource use.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100