video download all: prevent simultaneous jobs for same study
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 12
- Forks
- 21
- Avg merge
- 5d 19h
- Merged PRs (30d)
- 5
Description
TL; DR
The video download all task can be very resource intensive, and we allow two jobs to run in parallel. We should prevent a video-download-all task from running while this task is still in progress for the same study.
Description
The video download all task can be very time/resource intensive, and researchers might click the 'download all' button again before the first job is finished. Also, although we have recently improved the video-download-all task to make it use less memory, our system is still sensitive to the CPU load from zipping a large amount of video data. We recently had a site outage due to this task running twice (in parallel) for the same study, which had a large amount of video data. We can reduce the chances of two simultaneous large jobs by preventing the task from starting again while another job is still in progress for that same study.
Implementation
On the responses -> videos page, we should be able to check celery's active/queued jobs to see if there's a video download all task for that particular study, and if so, disable the 'download all' button (perhaps with a message saying that this job is already in progress/queued).
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at the responses → videos page and the video-download-all task, then inspect how Celery exposes active or queued jobs for a study. Done means a second download-all request for the same study is prevented while the first is active or queued, and the button shows an in-progress message.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, frontend
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100