ipfs / ipfs/boxo

filestore: update and revalidate all abspaths on re-add

Open
#360 2 comments 0 reactions 0 assignees View on GitHub
kind/enhancement P3
Dominant language
Go
Stars
316
Forks
163
Avg merge
4d 12h
Merged PRs (30d)
8

Description

This isn't super critical, as I can work around it, but I believe this would be a huge performance/usability improvement. I also hope this is the right repository for this feature.

(I believe) Filestore works like this:

1. You add file, it adds hashes normally, but also stores abspath (the full filepath)
2. You remove file from disk, still exists until GCd and dereferenced from MFS/pins
3. You add file again from new path, if old file wasn't GCd, old abspath is still stored
4. Everything breaks (if you try to access the CID, pin it, add it to MFS, pin a directory with the CID, etc, you'll get errors *sometimes*)

The solution I propose is to simply store the abspath for the new file as well, and when an error arises on one filepath, try the next one. If we store a status, could select the last successful abspath automatically.

Currently dealing with large directories of files, containing subdirectories and files that routinely get copied/renamed/deleted, I run into this issue *all the time*, where the CID I have published to IPNS fails to load certain files because their original file is gone/renamed, but the reference still exists in filestore (so the blocks need to be removed manually).

FWIW, I'm still on go-ipfs v0.7.0 for now, but filestore didn't get any changes in v0.8.0 (AFAIK).

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.