celestiaorg / celestiaorg/go-header
bug: sync stuck due to networking error
- Dominant language
- Go
- Stars
- 21
- Forks
- 33
- Avg merge
- 5d 20h
- Merged PRs (30d)
- 7
Description
reproduce on mocha network
1. setup a new full node to sync,
2. close the network(turn off wifi, pull out wire, etc)
3. wait for all peers to fail
4. open the network
5. the sync goroutine will never recover
the sync goroutine is stucking
if network is down, the peers in peerQuene will always be in a decreasing state due to errors that not a ErrNotFound one(network connect fail etc). and eventually run out of the havePeer channel. in this time GetRangeByHeight alway wait for hasPeer channel while getRangeByHeight wait for the result channel
a candidate fix is push back the peer state for errEmptyResponse error. https://github.com/celestiaorg/go-header/pull/238
but another fix is to add timeout for ```GetRangeByHeight```
```go
func (s *Syncer[H]) requestHeaders(
ctx context.Context,
fromHead H,
to uint64,
) error {
amount := to - fromHead.Height()
// start requesting headers until amount remaining will be 0
for amount > 0 {
size := header.MaxRangeRequestSize
if amount < size {
size = amount
}
to := fromHead.Height() + size + 1
s.metrics.rangeRequestStart()
//to fix , add timeout for this context
headers, err := s.getter.GetRangeByHeight(ctx, fromHead, to)
s.metrics.updateGetRangeRequestInfo(s.ctx, int(size)/100, err != nil)
s.metrics.rangeRequestStop()
if err != nil {
return err
}
if err := s.storeHeaders(ctx, headers...); err != nil {
return err
}
amount -= size // size == len(headers)
fromHead = headers[len(headers)-1]
}
return nil
}
```
Contributor guide
Assessment
This issue has not been assessed yet.