erigontech / erigontech/erigon
Caplin: to set boundaries - no how much goroutines can be spawned by for loop
- Dominant language
- Go
- Stars
- 3.6k
- Forks
- 1.5k
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 465
Description
Found many patterns like:
```
wg := WaitGroup{}
for {
wg.Add(1)
go func() {
defer wg.Done(1)
}
}
wg.Wait()
```
Examples:
`getDutiesProposer`
`connectWithAllPeers`
`batchVerifyAttestations`
`VerifyAgainstIdentifiersAndInsertIntoTheBlobStore`
This pattern spawns unlimited amount of goroutines. All "unlimited" things are bad - because:
- because we have limited cpus/ram/disk/etc...
- even if today some loop doesn't spawn much goroutines - future PR's or future bugs - may change it and we will not notice
- some "for loops" are run in-parallel - means it's will be hard for us to reproduce "worst case scenario" - where N loops spawning much goroutines in same time.
------
let's replace it by next pattern:
```
g := &errgroup.Group{}
g.SetLimit(runtime.GOMAXPROCS(-1))
for {
g.Go(func() error {
})
}
if err := g.Wait(); err != nil {
return err
})
```
Can set any limit you like.
Contributor guide
Research direction
Start by locating the named entry points: getDutiesProposer, connectWithAllPeers, batchVerifyAttestations, and VerifyAgainstIdentifiersAndInsertIntoTheBlobStore. Inspect each loop for unbounded goroutine creation and determine the appropriate bounded-concurrency approach. Done means the identified patterns have explicit limits and their existing completion and error behavior remains validated.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- backend
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100