treeverse / treeverse/dvc

exp run: showing unhelpful error message for invalid pipeline

Open
#8,974 2 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

A: experiments bug p2-medium
Dominant language
Python
Stars
15.9k
Forks
1.3k
PR merge metrics
No merged PRs in 30d

Description

Background in https://github.com/iterative/dvc/issues/8973.

Here is the setup from that issue:

# train.py
from random import random

from dvclive import Live

NUM_EPOCHS = 10


with Live(save_dvc_exp=True) as live:

    live.log_param("NUM_EPOCHS", NUM_EPOCHS)

    for epoch in range(NUM_EPOCHS):
        step = live.read_step()
        live.log_metric("metric", random())
        live.log_param("my_step", step)
        live.next_step()
# dvc.yaml
stages:
  train:
    cmd: python train.py
    deps:
      - train.py
    outs:
      - dvclive:
          checkpoint: true
    metrics:
      - dvclive/metrics.json
    plots:
      - dvclive/plots

When running dvc exp run, the error message looks like:

216764996-eb2efe78-b5d0-4c47-b79e-1053280e322f

Expected

dvc repro for a similar setup gives an error like:

ERROR: The output paths:
'dvclive'('train')
'dvclive/metrics.json'('train')
overlap and are thus in the same tracked directory.
To keep reproducibility, outputs should be in separate tracked directories or tracked individually.

The same error from dvc repro should be shown in dvc exp run.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the issue with the provided train.py and dvc.yaml setup, then compare dvc exp run with dvc repro. Trace where each command formats the overlapping tracked-output error. Done means dvc exp run reports the same useful overlap message shown by dvc repro.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.