ipfs / ipfs/kubo

Adding large blocks/directories (4MiB+) should cause error

Open
#5,282 0 comments 0 reactions 0 assignees View on GitHub
help wanted kind/bug status/deferred
Dominant language
Go
Stars
17.1k
Forks
3.2k
Avg merge
3d 18h
Merged PRs (30d)
11

Description

As mentioned in https://github.com/ipfs/notes/issues/296#issuecomment-407605712

I guess there is some hard limit on "do not accepts blocks over this size"

> The generated directories are large and don't work, example:
QmY6jEuqY3U5N9yxeBs6uoM63QnV2gjewXop9U4cQKauX8 has a distfiles directory with hash QmWvMmUjrH6neokv6RTqdgaYZSgxzr6hrjqaHosnDh6o7C which is 4821289 bytes in one block, I have republished that raw block as QmNzePsWN7kCF9fvbFBAnoVERWbCiVFjVEmr5EFqXFLtxL, Another example is QmfSAWwPTx5Kmre4bDF3zbkDuYXeNo9X7eTLvRUxyfr52B which can be found in QmPctnSoGCY8j19Lhxez3wqH2Ld55DzSqjnrbvNUuxoYgw. Using Sharding as described in #212 might solve this, but still the client should give error (or at least warning) when this happens.
I guess that is the size here that is the issue? But should go-ipfs allow to create so large blocks? (It would be better for it to throw an error until there is a fix) I get a feeling there already must be an issue created about this?, but can't find a good match (that is still open)

What I recommend is that the client errors out if a block larger then that max limit is added.
For those cases where this needs to be supported an extra flag can be added `--allow-oversized-blocks`

If an error can't be added then a Warning could be enough.

This message could also point to an issue disscussing this, or directions recommendingt to enable sharding.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.