dvc fetch reports 'Everything is up to date' when .dir manifest is missing from local cache

DVC version

3.67.1 (conda), Python 3.10.19, Linux

Description

When data.dvc references a .dir manifest hash that is not present in the local cache, dvc fetch reports "Everything is up to date" and does not download the manifest or its referenced files from the remote. The subsequent dvc checkout then fails with FileNotFoundError.

When this happens

This occurs in multi-machine DVC workflows when data.dvc is updated via git pull to reference a new .dir hash that was never fetched to the local machine. Specifically:

1. Machine A: runs dvc add data, dvc push, git push (new .dir hash pushed to S3)
2. Machine B: already has an older .dir in its local cache from a previous version
3. Machine B: git pull updates data.dvc to reference the new .dir hash
4. Machine B: dvc pull fails — fetch reports "up to date", checkout fails

The root cause appears to be that DVC's fetch logic relies on reading the .dir manifest to enumerate which individual files need to be transferred. When the .dir manifest itself is the missing file, DVC cannot enumerate anything, collects 0 entries, and concludes nothing needs to be fetched.

Verbose output (abbreviated)

DEBUG: Preparing to transfer data from 's3://<bucket>/files/md5' to '<project>/.dvc/cache/files/md5'
DEBUG: Preparing to collect status from '<project>/.dvc/cache/files/md5'
DEBUG: Collecting status from '<project>/.dvc/cache/files/md5'
DEBUG: failed to create '<project>/data/<file1>' from '<project>/.dvc/cache/files/md5/xx/xxxxxxxxx' - [Errno 2] No such file or directory
DEBUG: failed to create '<project>/data/<file2>' from '<project>/.dvc/cache/files/md5/yy/yyyyyyyyy' - [Errno 2] No such file or directory
Everything is up to date.
ERROR: failed to pull data from the cloud - Checkout failed for following targets: data

Evidence that files exist on remote

Verified via boto3 that both the .dir manifest and individual files are present in the S3 remote (uploaded ~2h before the pull attempt).

Expected behavior

dvc fetch should detect that the .dir manifest is missing from the local cache, download it first, then enumerate and download any missing individual files.

Workaround

Manually download the missing files from S3 into the correct cache paths, then run dvc checkout:

# Download .dir manifest and missing files directly
aws s3 cp s3://<bucket>/files/md5/xx/<hash>.dir .dvc/cache/files/md5/xx/<hash>.dir
aws s3 cp s3://<bucket>/files/md5/yy/<hash> .dvc/cache/files/md5/yy/<hash>
dvc checkout

Reproduce the multi-machine workflow with a data.dvc file referencing a missing .dir manifest, then trace the dvc fetch status and dvc checkout paths involving .dvc/cache. Verify the behavior against a remote containing the manifest and referenced files. Done means fetch downloads the missing manifest first, then the missing files, and checkout succeeds without manual cache copies.