repro: add scheduler for parallelising execution jobs
Open
Nobody has claimed this yet.
A: pipelines
p2-medium
research
- Dominant language
- Python
- Stars
- 15.9k
- Forks
- 1.3k
- PR merge metrics
- No merged PRs in 30d
Description
When I try to run multiple dvc run commands, I get the following error:
$ dvc run ...
Failed to lock before running a command: Cannot perform the cmd since DVC is busy and locked. Please retry the cmd later.
This is inconvenient b/c I'd love to run multiple experiments together using dvc. Anyway we can be more smart about locking?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing concurrent dvc run commands and reading the locking behavior around the dvc run entry point. Define how jobs should be scheduled and how locking should behave during parallel execution; done means multiple experiments can run together without the current busy-lock error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- cli, data-engineering
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100