dvc exp run --temp: dvc-tracked dependencies are not checked out
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 15.9k
- Forks
- 1.3k
- PR merge metrics
- No merged PRs in 30d
Description
Bug Report
Issue name
dvc exp run --temp: dvc-tracked dependencies are not checked out
Description
dvc exp run --temp will not dvc checkout the dependency file.txt from file.txt.dvc if file.txt has been modified.
Reproduce
- Setup a git repo with the following minimal structure:
├── code
│ └── test.py
├── data
│ └── file.txt
├── dvc.yaml
└── README.md
code/test/py just prints the contents the file.txt, and file.txt contains the single line "foo". file.txt is gitignored, all other files are git tracked. Content of dvc.yaml is:
❯ cat dvc.yaml
stages:
generate_data:
cmd: python3 code/test.py
deps:
- data/file.txt
dvc init && dvc add data/file.txt && git add data/file.txt.dvc data/.gitignoredvc exp run --drydvc exp run --temp-> prints "foo", as expected.rm data/file.txt && dvc exp run --temp-> checks out file.txt and prints "foo" as expected.
'data/file.txt.dvc' didn't change, skipping
Running stage 'generate_data':
> python3 code/test.py
foo
touch data/file.txt && dvc exp run --temp-> fails to dvc check out file.txt and runs with empty file.txt instead
WARNING: 'data/file.txt' is empty.
Running stage 'generate_data':
> python3 code/test.py
Expected
dvc exp run --temp should not copy file.txt to the temporary folder, since it is not git tracked, and dvc checkout file.txt from the local cache, just as it does when file.txt is not present at all.
Environment information
dvc==3.27.0
Output of dvc doctor:
❯ dvc doctor
DVC version: 3.27.0 (pip)
-------------------------
Platform: Python 3.8.12 on Linux-5.15.0-87-generic-x86_64-with-glibc2.17
Subprojects:
dvc_data = 2.18.2
dvc_objects = 1.0.1
dvc_render = 0.5.3
dvc_task = 0.3.0
scmrepo = 1.4.0
Supports:
http (aiohttp = 3.8.1, aiohttp-retry = 2.8.3),
https (aiohttp = 3.8.1, aiohttp-retry = 2.8.3),
s3 (s3fs = 2023.6.0, boto3 = 1.26.76),
ssh (sshfs = 2023.4.1)
Config:
Global: /home/ralf/.config/dvc
System: /etc/xdg/xdg-ubuntu/dvc
Cache types: <https://error.dvc.org/no-dvc-cache>
Caches: local
Remotes: None
Workspace directory: ext4 on /dev/nvme0n1p5
Repo: dvc, git
Repo.site_cache_dir: /var/tmp/dvc/repo/cae49d9183304046b1da8acdeb50f42f
Additional Information (if any):
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the issue with the listed dvc exp run --temp commands, especially after touching data/file.txt, then trace the temporary experiment setup and checkout handling. Done means a modified DVC-tracked dependency is restored from the local cache before the stage runs, matching the behavior when the file is absent.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- cli, tooling
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100