Project-MONAI / Project-MONAI/MONAILabel
Monai label server add API to return number of requests in the queue
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 891
- Forks
- 269
- Avg merge
- 15h 41m
- Merged PRs (30d)
- 1
Description
Please describe.
My model is very large and takes up to 60 seconds on my gpu. I run with --workers=1 to avoid having 2 requests run on the same gpu and get into out of memory. Now when multiple users each run a request they are in a queue that is some times too long.
Describe the solution you'd like
We should have an api to show how many requests are ahead of them in the queue and give an estimate on the wait time
This should take into account if we have multiple gpus. Basically this allows the UI to set the users expectations on the wait time
Describe alternatives you've considered
None
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue names no files, tests, or entry points. Start by locating the server's request queue and worker or GPU handling, then determine how queue position and wait estimates should be exposed. Done means an API reports requests ahead and an estimated wait time while accounting for multiple GPUs.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api, backend
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100