Feature Request: `dvc exp push --queued` to push queued experiments
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 15.9k
- Forks
- 1.3k
- PR merge metrics
- No merged PRs in 30d
Description
I want to run my experiments in my CI/CD pipeline, so I can use remote compute resources and also share the results with my team. My intended workflow is the following:
# Create a branch for the experiments
$ git checkout -b my-experiment
# Queue up some experiments
$ dvc exp run --name my-experiment --queue "param=1,2,3,4"
# Push the queue. This is the FEATURE REQUEST.
$ dvc exp push --queued
# Open a PR
$ git commit -am "Experiment description" --allow-empty
$ git push origin my-experiment
$ gh pr create --fill
Then a CI/CD pipeline will basically run:
# Before this command we check if there are any experiments queued
# ...
# Run all the experiments and show results
$ dvc exp run --run-all
$ dvc exp show
After reading and deciding an experiment to apply, I run locally:
# Pull experiment results and apply the best one
$ dvc exp pull
$ dvc apply best-experiment
$ git commit -am "Apply best parameters"
$ git push origin my-experiment
# After my team reviews the PR
$ gh pr merge
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing the existing dvc exp push command and the queued experiment workflow created by dvc exp run --queue. Define how --queued should transfer queued experiments so the CI commands dvc exp run --run-all and dvc exp show can consume them, then verify the workflow through the documented branch and pull-request sequence.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- git, python
- Domain
- ci-cd, cli, machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100