exp run: showing unhelpful error message for invalid pipeline
Open
Nobody has claimed this yet.
A: experiments
bug
p2-medium
- Dominant language
- Python
- Stars
- 15.9k
- Forks
- 1.3k
- PR merge metrics
- No merged PRs in 30d
Description
Background in https://github.com/iterative/dvc/issues/8973.
Here is the setup from that issue:
# train.py
from random import random
from dvclive import Live
NUM_EPOCHS = 10
with Live(save_dvc_exp=True) as live:
live.log_param("NUM_EPOCHS", NUM_EPOCHS)
for epoch in range(NUM_EPOCHS):
step = live.read_step()
live.log_metric("metric", random())
live.log_param("my_step", step)
live.next_step()
# dvc.yaml
stages:
train:
cmd: python train.py
deps:
- train.py
outs:
- dvclive:
checkpoint: true
metrics:
- dvclive/metrics.json
plots:
- dvclive/plots
When running dvc exp run, the error message looks like:
Expected
dvc repro for a similar setup gives an error like:
ERROR: The output paths:
'dvclive'('train')
'dvclive/metrics.json'('train')
overlap and are thus in the same tracked directory.
To keep reproducibility, outputs should be in separate tracked directories or tracked individually.
The same error from dvc repro should be shown in dvc exp run.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the issue with the provided train.py and dvc.yaml setup, then compare dvc exp run with dvc repro. Trace where each command formats the overlapping tracked-output error. Done means dvc exp run reports the same useful overlap message shown by dvc repro.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- cli
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100