pull: KeyError crash (or silently skipped target) when mixing .dvc-file targets with granular paths inside a tracked directory
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 15.9k
- Forks
- 1.3k
- PR merge metrics
- No merged PRs in 30d
Description
Bug Report
Description
dvc pull with a target list that mixes a .dvc-file target with granular paths inside a tracked directory misbehaves in two ways, depending on workspace state:
- Crash: if the tracked directory contains any untracked (drifted) file, pull dies with
ERROR: unexpected error - ('datadir', 'f1.txt')— an uncaughtKeyErrorfromcheckout's_check_can_delete. - Silent skip: in a clean fresh-clone state (no workspace, no cache), the same command succeeds (exit 0) but only materializes the
.dvc-file target; the granular target is never checked out.
Both target forms are documented as supported in the pull command reference ("paths to tracked files or directories (including paths inside tracked directories), .dvc files, and stage names"), and each form works correctly on its own — including a single invocation whose targets are all data paths, which is the workaround.
Reproduce
#!/usr/bin/env bash
set -uo pipefail
cd "$(mktemp -d)"
mkdir remote repo && cd repo
git init -q && git config user.email r@e.pro && git config user.name repro
dvc init -q
dvc remote add -d localremote ../remote
mkdir datadir
printf one > datadir/f1.txt; printf two > datadir/f2.txt; printf three > datadir/f3.txt
printf single > single.csv
dvc add datadir single.csv
git add -A && git commit -qm init
dvc push -q
echo "=== Variant 1: fresh-clone state -> granular target silently skipped ==="
rm -rf datadir single.csv .dvc/cache
dvc pull single.csv.dvc datadir/f1.txt # exit 0, "1 file fetched and 1 file added"
ls datadir/f1.txt 2>/dev/null || echo "BUG: datadir/f1.txt was not checked out"
dvc pull -q # restore everything
echo "=== Variant 2: one untracked file inside the tracked dir -> crash ==="
printf extra > datadir/extra-drift.txt
dvc pull single.csv.dvc datadir/f1.txt # ERROR: unexpected error - ('datadir', 'f1.txt')
echo "=== Variant 3 (control): same drift, data-path targets only -> works ==="
dvc pull single.csv datadir/f1.txt # "Everything is up to date."
Expected
Either the mixed target list works (both targets checked out, like variant 3), or a proper error message — not an internal KeyError and not a silent partial result.
Traceback (variant 2)
Traceback (most recent call last):
...
File ".../dvc/repo/pull.py", line 42, in pull
File ".../dvc/repo/checkout.py", line 160, in checkout
_check_can_delete(diff.files_delete, new, self.root_dir, self.fs)
File ".../dvc/repo/checkout.py", line 91, in _check_can_delete
cache_fs, cache_path = index.storage_map.get_cache(entry)
File ".../dvc_data/index/index.py", line 519, in get_cache
File ".../dvc_data/index/index.py", line 508, in get_storage
info = self[entry.key]
File ".../dvc_data/index/index.py", line 446, in __getitem__
KeyError: ('datadir', 'f1.txt')
Analysis
_check_can_delete anticipates the "entry known but no cache storage" case — it wraps the lookup in try/except ValueError (catching StorageKeyError). But when targets span different artifacts, the filtered index views seem to produce files_delete entries whose keys are entirely absent from the (filtered) storage_map, so StorageMapping.__getitem__ raises a bare KeyError, which escapes to the generic "unexpected error" handler.
Environment
DVC version: 3.67.1 (pip)
-------------------------
Platform: Python 3.12.12 on Linux-7.0.12-101.fc43.x86_64-x86_64-with-glibc2.42
Subprojects:
dvc_data = 3.18.3
dvc_objects = 5.2.0
dvc_render = 1.0.2
dvc_task = 0.40.2
scmrepo = 3.6.2
Supports: http, https, s3 (s3fs = 2026.4.0)
Originally hit with an S3 (Cloudflare R2) remote pulling two standalone-file artifacts plus 10 granular paths inside a 70-file directory artifact; the local-remote repro above shows the remote type is irrelevant.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with dvc/repo/pull.py and dvc/repo/checkout.py, especially checkout and _check_can_delete, then reproduce the mixed-target commands from the issue. Compare the filtered index and storage-map behavior for .dvc-file and granular targets. Done means both targets are checked out without a KeyError or silent partial result, with regression coverage for the reported variants.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- cli
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 68/100