Add `--num` option to `dvc fetch`
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 15.9k
- Forks
- 1.3k
- PR merge metrics
- No merged PRs in 30d
Description
Problem
When merging two branches in a DVC repository it is possible to encounter missing cache data from previous commits. This results in errors such as:
ERROR: unexpected error - [Errno 2] No such file or directory: '~/my_repo/.dvc/cache/files/md5/xx/xxx.dir'
To resolve this, the current workaround is to run:
dvc fetch --all-commits
on both branches to ensure all required data is available locally. However, for long-lived branches like main, this can result in downloading a very large amount of data, much of which is not needed.
Proposed solution
Introduce a --num <N> option to dvc fetch, similar to the existing option in dvc gc, which limits operations to the most recent N commits.
Example:
dvc fetch --num 10
This would fetch cache data only for the latest 10 commits in the current branch, reducing unnecessary downloads.
I would be interested in contributing an implementation if this feature is considered useful.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the CLI implementation for dvc fetch and the existing --num option in dvc gc. Trace how --all-commits selects commits and how fetch downloads their cache data. Done means dvc fetch --num N limits fetching to the latest N commits in the current branch, with coverage for the new option.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- cli
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100