Support for higher latency storage (NFS/SMB mounts)
- Dominant language
- Go
- Stars
- 17.1k
- Forks
- 3.2k
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 11
Description
### Checklist
- [X] My issue is specific & actionable.
- [X] I am not suggesting a protocol enhancement.
- [X] I have searched on the [issue tracker](https://github.com/ipfs/kubo/issues?q=is%3Aissue) for my issue.
### Description
I would like to suggest an addition to the storage backend. The reason I come to suggest this, is [the unfortunate implosion of my node.](https://www.reddit.com/r/ipfs/comments/13ixa9z/my_node_died_how_do_i_debug_this/)
Since I use NFSv4 to mount additional storage to my homeserver, since it's quite limited in storage from the get-go, I also chucked my IPFS repo on it. And after a while, I noticed that it was running into an issue where it either couldn't access the disk usage cache in a JSON file from time to time - and when I added a particularily large file, I probably ended it. The last error mentioned inconsistencies in the LevelDB storage.
This has had me thinking and occupied my mind for quite a while. To me, IPFS is a great way to share files with friends. Unlike NextCloud, I can just drop a file in there, then share a link with the CID - and remove it when done. Simple! But if my repo keeps corrupting because of the FS layer (and not coming back - it still hasn't, I have let it run since that post the entire time), I will have to think of another solution...
This is pretty much what I would like to propose:
- Implement a storage backend that can tolerate temporary inavailability (usually a second or two).
- The backend should keep a memory-local cache - or a separate on-disk cache in a different location - of the recently used data. Maybe a straight-up LRU-style one?
- It should run verification once in a while - and if severe, drop the related segment, if plausible. Yes, it'd be a lossy storage in this regard, but rather than halting the entire node, it'd possibly be better to just drop the portion. Best-case scenario, it's elsewhere on the network anyway, so it could also be recovered.
Another side-effect of a storage backend with those features would be to possibly build a small SAN and connect several IPFS instances to it, allowing some form of load balancing in a more low-cost situation. A (pretty unrealistic and dumb) setup could be a couple of Raspberry Pis connected to a NAS holding the actual repo, thus splitting tasks between the Pis.
Hope this sparks some ideas and is helpful! :)
Kind regards,
Ingwie.
PS.: I have temporarily moved to use my NextCloud instance and since encountered that it has issues generating share links. Fun times...
Contributor guide
Assessment
This issue has not been assessed yet.