ipfs / ipfs/kubo

add `--sparse` option to `get`, `pin add` subcommands, for broken file

Open
#5,188 7 comments 2 reactions 0 assignees View on GitHub
kind/feature status/deferred
Dominant language
Go
Stars
17.1k
Forks
3.2k
Avg merge
3d 18h
Merged PRs (30d)
11

Description

#### Type: Enhancement

#### Description:
Sometimes I have a file in my local repo that's missing a single block, that's nowhere on the network either. When I `ipfs get `, it stalls at the position where that block is in the file, forever. Media files usually tolerate a single missing block just fine, so I'd like a `--sparse` option that would simply skip unavailable blocks and leave it unwritten in a sparse file or nulled in filesystems that don't support sparse files. *Unavailable* would mean that the block stalls (0bps downstream) longer than X seconds or takes Y% (~1000%) times longer than the other blocks to retrieve. Maybe X and Y could be configurable in the daemon's settings.

Similarly, a `--sparse` or maybe `--partial` option to `ipfs pin add` would skip over the missing blocks and pin the rest, while still returning some non-zero error, because the entire file could not be pinned.

The other motivation for this besides playing media files is that when I lose pieces of a file and want to recover them from a torrent, I have no easy way to get the trailing good blocks out of the ipfs datastore. I presume the API allows me to fetch each block individually and write it into the file myself, but I think this facility should be part of the CLI. Likewise, when pinning a file, I sometimes want to transfer over as much as possible in the background, while working on recovering the last piece to complete the pin.

Contributor guide

Open the contributing guide

Research direction

Start by reading the existing `ipfs get` and `ipfs pin add` command paths and their handling of unavailable blocks. Clarify how sparse output, timeout thresholds, configuration, and partial pin errors should work. Done means both subcommands have an agreed `--sparse` or `--partial` behavior, with appropriate tests and documented non-zero failure handling.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
cli
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.