ipfs / ipfs/boxo

chunker: infinite loop in the Rabin chunker

Open
#353 12 comments 0 reactions 1 assignee Claimed by @ribasushi View on GitHub
Dominant language
Go
Stars
316
Forks
163
Avg merge
4d 12h
Merged PRs (30d)
8

Description

At Infura we got some of our nodes ending in an infinite loop in the Rabin chunker, with the CPU at 100% until something bad happen (the context doesn't reach there so loop keep spinning even though the request is long gone).

Here is a profiling:
![profile001](https://user-images.githubusercontent.com/294669/70930882-8468b980-2036-11ea-97b7-327b9b474a8c.png)

Original requests are `add` with the following parameters: `[chunker=rabin-262144-524288-1048576,encoding=json,hash=sha2-256,inline-limit=32,pin=false,progress=true,stream-channels=true,]`. Some also have `trickle=true` but that doesn't seem to matter.

Profiling lead to an infinite loop in https://github.com/whyrusleeping/chunker/blob/master/chunker.go#L209-L336 but it's unclear to me what is happening. `if err == io.ErrUnexpectedEOF { err = nil }` is suspicious but that kinda fly over my head.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.