Adding large blocks/directories (4MiB+) should cause error
- Dominant language
- Go
- Stars
- 17.1k
- Forks
- 3.2k
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 11
Description
As mentioned in https://github.com/ipfs/notes/issues/296#issuecomment-407605712
I guess there is some hard limit on "do not accepts blocks over this size"
> The generated directories are large and don't work, example:
QmY6jEuqY3U5N9yxeBs6uoM63QnV2gjewXop9U4cQKauX8 has a distfiles directory with hash QmWvMmUjrH6neokv6RTqdgaYZSgxzr6hrjqaHosnDh6o7C which is 4821289 bytes in one block, I have republished that raw block as QmNzePsWN7kCF9fvbFBAnoVERWbCiVFjVEmr5EFqXFLtxL, Another example is QmfSAWwPTx5Kmre4bDF3zbkDuYXeNo9X7eTLvRUxyfr52B which can be found in QmPctnSoGCY8j19Lhxez3wqH2Ld55DzSqjnrbvNUuxoYgw. Using Sharding as described in #212 might solve this, but still the client should give error (or at least warning) when this happens.
I guess that is the size here that is the issue? But should go-ipfs allow to create so large blocks? (It would be better for it to throw an error until there is a fix) I get a feeling there already must be an issue created about this?, but can't find a good match (that is still open)
What I recommend is that the client errors out if a block larger then that max limit is added.
For those cases where this needs to be supported an extra flag can be added `--allow-oversized-blocks`
If an error can't be added then a Warning could be enough.
This message could also point to an issue disscussing this, or directions recommendingt to enable sharding.
Contributor guide
Assessment
This issue has not been assessed yet.