`ipfs pin add` on large trees is extremely slow even when most of the content is already pinned
- Dominant language
- Go
- Stars
- 17.1k
- Forks
- 3.2k
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 11
Description
#### Version information:
go-ipfs version: 0.4.13-
Repo version: 6
System version: amd64/linux
Golang version: go1.9.4
#### Type:
Enhancement
#### Description:
`ipfs pin add` is slow on large data sets, because it needs to walk the entire tree in order to verify that it exists on the node (and, if not, fetch it from elsewhere) before the pin is considered successful. This is expected.
However, it looks like when pinning something *that itself contains other pinned objects*, it re-checks all of the contents, even though it should know that the contents are already present.
Reproduction:
The call to `ipfs pin add` will take as long as the original `ipfs add` did, or nearly so, even though all that data is already pinned and known to be present.
Ideally, `pin add` -- or whatever underlying function it calls to make sure the data is all available in the node -- should be smart enough to realize that when it encounters a recursively pinned object, it doesn't need to traverse that object's children, as they are guaranteed to be present.
Contributor guide
Assessment
This issue has not been assessed yet.