repro: --allow-missing skips changed dependencies inside directory outputs
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 15.9k
- Forks
- 1.3k
- PR merge metrics
- No merged PRs in 30d
Description
Bug Report
Description
When one stage outputs a directory and another depends on a file inside it, dvc repro --allow-missing --dry can report everything up to date even though that file changed upstream. This happens when the directory is missing from the workspace, even with its current .dir manifest available in the local cache.
This makes the command unreliable for CI freshness checks without checking out the data.
Reproduce
With DVC 3.67.1 installed, run in Bash:
set -eu
repro_dir=$(mktemp -d)
cd "$repro_dir"
git init -q
dvc init -q
dvc config --local cache.dir .dvc/cache
echo old > source.txt
dvc stage add -n produce -d source.txt -o data \
'mkdir -p data && cp source.txt data/file.txt'
dvc stage add -n consume -d data/file.txt -o result.txt \
'cp data/file.txt result.txt'
dvc repro
echo new > source.txt
dvc repro produce
mv data data.saved
dvc repro --allow-missing --dry
Actual output from the last command:
Stage 'produce' didn't change, skipping
Stage 'consume' didn't change, skipping
Data and pipelines are up to date.
Restoring the same directory makes DVC correctly detect that consume needs reproduction:
mv data.saved data
dvc repro --allow-missing --dry
Expected
The dry run should identify consume as needing reproduction in both cases. Its recorded dependency hash differs from the file's hash in the updated directory manifest; the file contents should not need to be checked out to detect this.
Note: Related to #9530, fixed by #9533 for dependencies whose paths exactly match upstream outputs. This reproduction covers a remaining case: a dependency on a file inside an upstream directory output. It still reproduces on DVC 3.67.1.
Environment information
Output of dvc doctor (personal paths redacted)
DVC version: 3.67.1 (pip)
-------------------------
Platform: Python 3.13.12 on Linux-6.8.0-138-generic-x86_64-with-glibc2.39
Subprojects:
dvc_data = 3.18.3
dvc_objects = 5.2.0
dvc_render = 1.0.2
dvc_task = 0.40.2
scmrepo = 3.6.2
Supports:
http (aiohttp = 3.14.3, aiohttp-retry = 2.9.1),
https (aiohttp = 3.14.3, aiohttp-retry = 2.9.1)
Config:
Global: [redacted]/.config/dvc
System: /etc/xdg/dvc
Cache types: hardlink, symlink
Cache directory: zfs on fastnvme/workspace-tmp
Caches: local
Remotes: None
Workspace directory: zfs on fastnvme/workspace-tmp
Repo: dvc, git
Repo.site_cache_dir: [redacted]
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by running the Bash reproduction with DVC 3.67.1 and inspect the dependency-hash handling used by dvc repro --allow-missing --dry for files inside directory outputs. Done means the final dry run identifies consume as needing reproduction when data is absent but its updated .dir manifest is available in the cache.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- cli, data
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 70/100