treeverse / treeverse/dvc

Be able to permanently delete a file from DVC, including from history

Open
#4,903 5 comments 7 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

feature request p2-medium
Dominant language
Python
Stars
15.9k
Forks
1.3k
PR merge metrics
No merged PRs in 30d

Description

There are use-cases where you want to completely purge a file permanently from DVC history. For instance, if given a database of personal data, and the user requests to have their data deleted, it should be deleted in such a manner that it can not be recovered.

I understand that dvc gc is capable of modifying the DVC cache such that the history is also modified. However, it seems that at the moment, there is no way to specify which file/directory you want deleted; so it deletes everything that is not being currently tracked. This might not be preferred because there could be deleted files in your database that you still want to be part of your history.

It would be great if a dvc gc-like feature is added through which you can also specify a file/dir name so that only that will get deleted from the history.

Conversation regarding this in the forum:
https://discuss.dvc.org/t/deleting-a-dvc-tracked-file-from-history/556

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at the dvc gc entry point and review the linked forum discussion about deleting tracked files from history. Define how a file or directory target should be selected without removing unrelated deleted history. Done means the targeted data cannot be recovered while other retained history remains available.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli, data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.