treeverse / treeverse/dvc

add: repeated --glob includes generated DVC metadata

Open
#11,086 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
15.9k
Forks
1.3k
PR merge metrics
No merged PRs in 30d

Description

Bug Report

Description

Running the same dvc add --glob command twice fails when the pattern also matches the .dvc files created by the first run. The first run tracks the data successfully; the second expands both the original data paths and their generated stage files, then tries to add a stage file as data.

Reproduce
git init -q repo
cd repo
dvc init -q
mkdir data
printf "one\n" > data/one
printf "two\n" > data/two

dvc add --glob "data/*"
dvc add --glob "data/*"

The second command exits with status 1:

ERROR: DVC file 'one.dvc' cannot be an output.

This also prevents the common update workflow where new files are added to the directory and the same glob command is rerun to update existing matches and add the new ones.

Expected

Wildcard expansion for dvc add --glob should ignore DVC metadata (*.dvc, dvc.yaml, and dvc.lock) because those paths cannot be DVC outputs. Explicitly passing a metadata path should remain invalid and retain the existing error.

Analysis

dvc.repo.add.find_targets() currently passes every filesystem glob match to get_or_create_stage(). On the second invocation, generated stage files are therefore treated as new output targets and rejected by output-path validation.

Environment

DVC main at 56e59829, Python 3.13, macOS arm64.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with dvc.repo.add.find_targets() and follow how it calls get_or_create_stage() during dvc add --glob. Run the provided two-command reproduction, then verify that repeated wildcard expansion ignores .dvc, dvc.yaml, and dvc.lock while explicitly passing metadata still produces the existing error.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.